Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Software - Troubleshooting and Discussion > Developer Center > Problem with decimal numbers in Cocoa

Problem with decimal numbers in Cocoa
Thread Tools
Fresh-Faced Recruit
Join Date: Apr 2001
Location: Uppsala
Status: Offline
Reply With Quote
May 1, 2001, 05:35 AM
 
Having decided to learn myself Cocoa I wrote a simple calculator in Java and everything worked fine except for whenever I tried to use decimal numbers. The result is almost right but not exactly, e.g. 1.1 + 1 would become 2.099999... Oh, well I thought, must be something wrong with my Java-knowledge. So I tried to write a simple program in Objective-C that just adds two numbers and to my surprise the same thing happens, i.e. 1 +1 becomes 2 but 1.1 + 1 becomes 2.099999904632568. I include my source code below and I hope someone can shed some light on it as I really haven't got a clue what could be wrong...

---test.h

#import <Cocoa/Cocoa.h>

@interface test : NSObject
{
IBOutlet id number1;
IBOutlet id number2;
IBOutlet id sum;
}
- (IBAction)calculate id)sender;
@end

---test.m

#import "test.h"

@implementation test

- (IBAction)calculate id)sender
{
float value1, value2, calculation;
value1 = [number1 floatValue];
value2 = [number2 floatValue];
calculation = value1 + value2;
[sum setFloatValue:calculation];
}
     
Admin Emeritus
Join Date: Oct 2000
Location: Boston, MA
Status: Offline
Reply With Quote
May 1, 2001, 06:23 AM
 
Try using 'double' instead of 'float'
"Against stupidity, the gods themselves contend in vain" (Schiller)
     
pgw3  (op)
Fresh-Faced Recruit
Join Date: Apr 2001
Location: Uppsala
Status: Offline
Reply With Quote
May 1, 2001, 07:04 AM
 
Originally posted by parallax:
Try using 'double' instead of 'float'
Thanks for the suggestion, it worked. But why is it like that? Can't one trust float and is there ever a reason to use float instead of double? I thought that one could use float if one only needs a few decimals.

     
Dedicated MacNNer
Join Date: Apr 2001
Location: Bethesda, MD
Status: Offline
Reply With Quote
May 1, 2001, 08:36 AM
 
That's just the nature of representing a decimal fraction with binary
numbers. Using double just pushes the error out to fewer decimal places.
It's like trying to represent 1/3 in decimal. At some point you have to
cut it off.

The standard thing to do is decide how much precision you want and then
round off your numbers.
     
   
Thread Tools
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Top
Privacy Policy
All times are GMT -5. The time now is 09:40 PM.
All contents of these forums © 1995-2011 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.7 © 2000-2011, Jelsoft Enterprises Ltd., Content Relevant URLs by vBSEO 3.3.2