Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > Developer Center > Converting NSString to float and/or int values

Converting NSString to float and/or int values
Thread Tools
Junior Member
Join Date: Aug 2000
Location: Montreal, QC, Canada
Status: Offline
Reply With Quote
Nov 1, 2003, 06:06 PM
 
I'm writing an application, but I'm having a problem getting an unexpected result.

One function returns an 11 character NSString, which I can easily convert to an int or float value. The number is in Hertz (Hz) for frequency. The problem is I want to express it in MegaHertz (MHz) so I divide it by 1000000 and then display this number in my interface in a NSTextField.

Sometimes when I display the number I get strange results, which I blame on a floating point error.

I start with 00118125500, convert it to a float of 118125500 and the divide by 1000000 and I sometimes get 118.125499. When I later try processing this number later in my application I get all sorts of problems.

Any ideas what I can do to elimate this problem.

Thanks
     
Fresh-Faced Recruit
Join Date: May 2003
Status: Offline
Reply With Quote
Nov 1, 2003, 07:26 PM
 
Code:
double
I am now going to tell the computer what he can do with a life-times supply of chocolate.
     
Octo  (op)
Junior Member
Join Date: Aug 2000
Location: Montreal, QC, Canada
Status: Offline
Reply With Quote
Nov 1, 2003, 07:51 PM
 
Okay I'm just going to slink under a rock now.

I could of sworn that I had tried that and had problems with that.

Thanks
     
   
Thread Tools
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Top
Privacy Policy
All times are GMT -5. The time now is 07:21 PM.
All contents of these forums © 1995-2011 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.7 © 2000-2011, Jelsoft Enterprises Ltd., Content Relevant URLs by vBSEO 3.3.2