Welcome to the MacNN Forums.

If this is your first visit, be sure to check out the FAQ by clicking the link above. You may have to register before you can post: click the register link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below.

You are here: MacNN Forums > Community > MacNN Lounge > G5 benchmarks debunked

G5 benchmarks debunked
Thread Tools
n~s
Forum Regular
Join Date: Jan 2003
Status: Offline
Reply With Quote
Jun 24, 2003, 10:05 AM
 
Kind of an interesting challenge to Apple's claim that the new G5 machines are the fastest available. According to this site, Apple disabled some Intel features for the comparison.

Also mentions "mac fanatics" and quotes a few MacNN readers.

http://www.haxial.com/spls-soapbox/apple-powermac-G5/

(Thanks /.)
You are not your signature.
     
bamburg dunes
Mac Enthusiast
Join Date: Jun 2003
Location: Kalifornia
Status: Offline
Reply With Quote
Jun 24, 2003, 10:21 AM
 
There's a very lengthy discussion going on in the Ars Technica forum too, seems like Apple has been doing the old reality distortion factor thing again.
Still great machiens though.
PIXAR Animation Studios
     
bewebste
Senior User
Join Date: Mar 2000
Location: Ithaca, NY
Status: Offline
Reply With Quote
Jun 24, 2003, 10:24 AM
 
Yeah, if what he's saying is true, and features such as SSE2 and hyperthreading weren't enabled for the SPEC tests, then that is definitely misleading. I wonder though if Apple told Veritest to do these things, or if Veritest did them on its own. Bad either way, but it's possible that Apple wasn't the one with the bad procedures.

Personally, I don't give a flying hoot about the theoretical SPEC benchmarks, all I care about is real world performance, and I couldn't help but notice this person didn't say one word about the rest of the tests in which the G5 stomped the Pentium. Granted, Apple is only going to show the tests in which it wins, but I thought all three of the other bakeoffs were certainly revelant markets for Apple to show their speed in.
     
Millennium
Clinically Insane
Join Date: Nov 1999
Status: Offline
Reply With Quote
Jun 24, 2003, 10:25 AM
 
Frankly, anything said about Macs by He Who Was Hinks should be taken with a grain of salt; he has an axe to grind. To be honest, I don't consider any of these benchmarks valid, because they are not running the same operating system, and this can skew the results.

Both HHWH's and Apple's tests expose weaknesses in the G5 chip. However, there is something to notice about this: despite the fact that the G5 was running at 2 GHz and the Dells at 3 GHz, the difference in speed was not proportional to the clockrates; Apple did much better than it should have, according to clock speed alone.

IBM has promised that by next year, we will have caught up in terms of clockrate. Let us run our tests then -preferably with Linux on all the machines involved- and see the truth for ourselves.
You are in Soviet Russia. It is dark. Grue is likely to be eaten by YOU!
     
Eug Wanker
Posting Junkie
Join Date: Jun 2003
Location: Dangling something in the water… of the Arabian Sea
Status: Offline
Reply With Quote
Jun 24, 2003, 10:25 AM
 
I posted my comments about this here.

Dual 2.0 GHz G5 ~ Dual 2.66 GHz Xeon???
Apple did much better than it should have, according to clock speed alone.
I'm not sure what you mean by that. AMD's run at something like 2.2 GHz for instance, and are competitive with 3+ GHz P4 chips.
     
benb
Registered User
Join Date: Nov 2002
Location: Far from the internet.
Status: Offline
Reply With Quote
Jun 24, 2003, 10:50 AM
 
"Both Apple and Dell are guilty of using misleading prices. For example, Apple gives the price of the low-end G5 as "$1999", and the high-end G5 as "$2999". In other words, they have subtracted $1 from a $3000 computer to make it seem cheaper, which is absolutely ridiculous. This demonstrates that both Apple and Dell are willing to mislead people when stating their prices."

Misleading prices...?
     
AB^2=BCxAC
Grizzled Veteran
Join Date: Mar 2003
Location: New York
Status: Offline
Reply With Quote
Jun 24, 2003, 11:37 AM
 
Well, no matter what Apple did to skew the results for marketing (which is typical, so I'm not surprised), there were some nice things that G5 owers can be proud of when they finally get their machines.

8GB of 400 DDR-RAM on a 1Ghz System Bus, if you splurge. That's pretty nice to have for a professional video system set up. Even the 800Mhz frontside bus on the low end is at least respectable.

And there alot of off the shelf features that usually inflate the price of any PC, because they aren't typical off the shelf options:

Digital Audio connectors - standard.
Gigabit Ethernet - standard
DVD-R 4x - standard
Firewire 800 + USB 2 ports (finally!) - standard
Two DVI (including the ADC) monitor plugs - standard

I'm not an apologist, but I figure if Apple wants to keep the HD's and optical drives limited (probably to be conservative with internal temperatures and the resulting size and sound) then there are ways to work around it that aren't big deals. Realistically, even the workstations at Pixar don't have to have everything built inside. All the big HD space is networked to servers.

I'm saying the spec lying disappoints me, but it won't shy me off getting myself one.
"I stand accused, just like you, for being born without a silver spoon." Richard Ashcroft
     
starman
Clinically Insane
Join Date: Jun 2000
Location: Union County, NJ
Status: Offline
Reply With Quote
Jun 24, 2003, 11:46 AM
 
Yeah, cuz the Haxial guy's not biased .

Mike

Home - Twitter - Sig Wall-Retired - Flickr
     
n~s  (op)
Forum Regular
Join Date: Jan 2003
Status: Offline
Reply With Quote
Jun 24, 2003, 12:00 PM
 
Originally posted by starman:
Yeah, cuz the Haxial guy's not biased
What, and Steve Jobs isn't?

I still think this is an incredible computer and doctoring the benchmarks doesn't stop the G5 from being a competitive machine.

Besides, we've all been chanting "CPU speed isn't everything" for so long when Apple's performance didn't match up to high-end offerings from Intel or AMD.
You are not your signature.
     
ambush
Banned
Join Date: Apr 2002
Location: -
Status: Offline
Reply With Quote
Jun 24, 2003, 12:08 PM
 
This Haxial guy's a bit too proud. If you read his GUI opinion, you'll realize that he thinks he's The One To Rule Them All or something, he thinks he's reinventing the GUI.

Anyway, I find it funny when he says how he hates zealot, because [if you had not noticed] he's in fact a zealot himself! He took what? less than 24 hours to write all this! This is insane.

Anyway I used to like his opinions but I understood how he just wants to be popular and stuff.

Maybe it's because of iChat AV that he's so frustrated? Blah AH!
     
The Placid Casual
Professional Poster
Join Date: May 2001
Location: Switzerland
Status: Offline
Reply With Quote
Jun 24, 2003, 12:17 PM
 
The article 'debunked' nothing.

Apple stated in detail how they performed the tests. They just didn't use highly optiised Intel compilers... The tests were very fair.

Also in 'real' world terms, no one can deny that the x86 machines were trounced.

There is also some interesting discussion here... at AI. Very enlightening...
     
ambush
Banned
Join Date: Apr 2002
Location: -
Status: Offline
Reply With Quote
Jun 24, 2003, 12:19 PM
 
Originally posted by benb:
"Both Apple and Dell are guilty of using misleading prices. For example, Apple gives the price of the low-end G5 as "$1999", and the high-end G5 as "$2999". In other words, they have subtracted $1 from a $3000 computer to make it seem cheaper, which is absolutely ridiculous. This demonstrates that both Apple and Dell are willing to mislead people when stating their prices."

Misleading prices...?
Lol. Misleading prices.

So he's saying that like, when we're buying a computer (which is something relatively important, if we compare it to something like buying - say - a t-shirt.) we think that 2999 is actually a *lot* lower than 3000.

Sorry but since I'm 5 years old, I know this little marketing trick. And buying a computer is not like buying a tshirt - you have to calculate if you will have enough money first and calculate how much money you're going to have after.

Not to mention EVERY industry does it. It's very rare to see products that cost 10$.

So he's just inventing another reason here.
     
Nonsuch
Professional Poster
Join Date: Jan 2001
Location: Riverside IL, USA
Status: Offline
Reply With Quote
Jun 24, 2003, 12:24 PM
 
It doesn't matter much, does it? I think anyone old enough to purchase a computer knows to take phrases like "the fastest desktop computer in the world" with a grain of salt. We'll see what kind of benches Tom's Hardware or Tech TV comes up with once the machines hit the streets; the results may not be as impressive, but I'm sure the G5s will do well, well enough that most evenhanded observers will concede there really is no longer a significant speed gap.
Find out just what any people will quietly submit to and you have found out the exact measure of injustice and wrong which will be imposed upon them.

-- Frederick Douglass, 1857
     
nvaughan3
Mac Elite
Join Date: Jul 2002
Location: St. Joseph, MI
Status: Offline
Reply With Quote
Jun 24, 2003, 12:26 PM
 
So he's just inventing another reason here.

He knocked on dell for this too, though I dont know why he even brought it up in the first place. Simply dumb.


http://www.overclockers.com/tips00408/

overclockers.com chimes in on the doctoring.
"Americans love their country and fear their government. Liberals love their government and fear the people."

""Gun control is a band-aid, feeling good approach to the nation's crime problem. It is easier for politicians to ban something than it is to condemn a murderer to death or a robber to life in prison. In essence, 'gun control' is the coward's way out.""
     
moki
Ambrosia - el Presidente
Join Date: Sep 2000
Location: Rochester, NY
Status: Offline
Reply With Quote
Jun 24, 2003, 12:29 PM
 
Originally posted by n~s:
Kind of an interesting challenge to Apple's claim that the new G5 machines are the fastest available. According to this site, Apple disabled some Intel features for the comparison.
Apple didn't diable anything; VeriTest did.

Regardless, this is why benchmarks are ridiculous. You will find that other computer manufacturers do the exact same thing, and that compilers available for x86 optimize specifically for benchmarks.

It's also useful to note that gcc for x86 is far more optimized than gcc for the PPC.

None of this is relevant to the fact that the G5 is one hell of a fast machine. Who is "top" depends on what compiler is used, and what optimizations are used... and it something best left to the marketing department.
Andrew Welch / el Presidente / Ambrosia Software, Inc.
     
nforcer
Mac Elite
Join Date: Oct 2001
Status: Offline
Reply With Quote
Jun 24, 2003, 01:26 PM
 
The basis of the article is that the benchmarking Apple did unfairly favors their G5. The fact of the matter is, the only companies that can actually test the chip as it is are IBM and Apple, so most of the rest of what is mentioned is speculation on the writer's part. Some of information presented seems very reasonable to a non-expert like myself. But the writer takes a stance that acts as if Apple is the only company ever to tilt the table a little bit in their favor. So in the end, he ends up looking either very biased against Apple in particular or very naive.

The sections the writer has on "Misleading Prices", "Crazy Mac Fanaticism", and "Answering Hate Mail" are out of place and basically extend on this guy's previous personal rants. I don't know of any company that doesn't or hasn't priced their products in such a fashion listed under "Misleading Prices" at some point or another. It is a common marketing tactic. And the "Mac Fanatics" and "Answering Hate Mail" sections are a complete troll... without a doubt more "Crazy PC Fanatics" exist. But why bother with that little detail? And what kind of replies does the guy expect if he is going to MISLEAD us and SNEAK those little troll pieces in at the end? (Notice how easy it is to apply his mentality to anything?)

This article would have been better if it were more generalized. What the author is really trying to say is that benchmarks can't be fair because there are so many differing configurations. Each configuration supports different features of different CPUs, and given the right configuration, it is possible to show that almost any CPU is faster than another for the specifically tested tasks.

Of course, the main thing that matters is real world performance. The best "benchmarking" tools available are the applications we use everyday. But even if we "lose", and the G5 does turn out to be a little slower than Apple indicates, we still don't have to put up with Windows. And that is why I use a Mac
     
itai195
Addicted to MacNN
Join Date: May 2001
Location: Cupertino, CA
Status: Offline
Reply With Quote
Jun 24, 2003, 01:45 PM
 
Probably the most over the top article I've seen is the one at AMDZone, where the author seems to really go out of his way to make sure Apple doesn't get credit for anything. I also don't understand why the author likes Dell so much -- to my knowledge Dell doesn't use AMD processors at all, at least not in their PCs.
     
InegoMontoya
Fresh-Faced Recruit
Join Date: Jun 2003
Status: Offline
Reply With Quote
Jun 24, 2003, 01:49 PM
 
Originally posted by itai195:
Probably the most over the top article I've seen is the one at AMDZone, where the author seems to really go out of his way to make sure Apple doesn't get credit for anything. I also don't understand why the author likes Dell so much -- to my knowledge Dell doesn't use AMD processors at all, at least not in their PCs.
AMD is really trying to get Dell to put the opteron in their "new" server line. The advantage would be that 32-bit apps would run just fine - as opposed to the Itanium2. Dell is apparantly giving AMD serious consideration. The opteron could make or break AMD, so they are railing hard against any 64-bit competitors...especially those that allow backwards compatibility. So the AMDZone thing doesn't surprise me.
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Jun 24, 2003, 02:57 PM
 
Well, Apple's SPEC scores are certainly not the ones obtained by Intel or Dell themselves.

But the G5 almost doubles the performance per CPU compared to a G4, so this is a giant leap already. We are catching up, the speed gap issue is being actively addressed.

Two points have been mentioned that Apple didn't have to mention.

1. Within 12 months, the G5 will be clocked at up to 3 GHz. Steve basically promised it -- the people are gonna remember, and he wouldn't promise that if he wouldn't be convinced that Apple and IBM can keep it.

2. The IBM guy specifically mentions that the next generation PPC is already sampled. (I assume he is referring to the Power5 that is supposed to debut in 2004.) For chrissake, they have just released the G5 and they are implying that the G6 is already far-progressed.

So I would think that the PPC980 will follow a lot sooner after the Power5 than the PPC970 did (after all, the first Power4 systems shipped in 2000, it took three years!).
I don't suffer from insanity, I enjoy every minute of it.
     
Nonsuch
Professional Poster
Join Date: Jan 2001
Location: Riverside IL, USA
Status: Offline
Reply With Quote
Jun 24, 2003, 04:08 PM
 
Slashdot has an interesting thread on this topic. One early post catches my eye:

At work, we just bechmarked the Dell systems a month ago and got very similar results to Apple for the "base" rate. The article seems to be quoting the "peak" rate for the Dells. It's not valid to compare peak rates yet because gcc 3.3 and os 10.3 aren't really fully optimized yet.

The article also complains that using the NAGWare compilers is not a valid test since they're too slow. But I think the NAGWare compiler is a more vallid comparison than intel's compiler because most real-world computing is done with NAGWare because it fully implements the F95 spec and is more portable. In addition NAGWare is well tested for accuracy and it also very much cheaper.

The Dell benchmark numbers are pure fantasy. They never occur in real-world use.
Contrasts quite hilariously with this passage from the overclockers article:

Intel and AMD occasionally fib a little in their benchmark testing, but when they do, it's usually a matter of a handful of percentage points, and if you look closely (or sometimes even not-so-closely) you can find the differences fairly easily.

They never do these kind of things."



(I also enjoy his referring to Dell's own benchmarks as the "official" benchmarks.)
Find out just what any people will quietly submit to and you have found out the exact measure of injustice and wrong which will be imposed upon them.

-- Frederick Douglass, 1857
     
Earth Mk. II
Mac Elite
Join Date: Feb 2001
Location: Washington, DC
Status: Offline
Reply With Quote
Jun 24, 2003, 04:48 PM
 
What effect would disabling hyperthreading - the one real valid point the soapbox guy has - have on a benchmark test, anyway?

Isn't hyperthreading only an advantage when more than one thread is running on the CPU? Ideally, the benchmarking thread would be the only one running a CPU until it finishes, right?

Im just trying to wrap my head around this... but I don't think the Xeon system was significantly crippled, if at all. And even if it was while the PPC performance was extremely enhanced - who's really surprised?

FYI: it seems the system config was done under Apple's supervision.

Refrence on Hyperthreading [ArsTechnica]
/Earth\ Mk\.\ I{2}/
     
Toyin
Mac Elite
Join Date: Nov 2000
Location: Boston
Status: Offline
Reply With Quote
Jun 24, 2003, 06:22 PM
 
Not being very technical I have some questions.

Why are the Dell numbers correct and the Apple numbers wrong?
Would optimizing for Altivec and 64bit significantly increase the G5s performance on these tests?

It seems like the PC world is just angry that Apple has finally put out a cutting edge box.

Before OSX the argument was, OS9 sucks, OS9 crashes and can't multitask, OS9 isn't configurable, OS9 isn't for 'power' users...etc
After OSX the argument became, Apples are overpriced and underpowered.

Apple has put out a comparable machine and the arguments are getting thin. Now it's that Apple is cheating on benchmark tests. Yet smoked dual 3.06ghz xenons in several real world tests.

Are the G5s the fastest personal computer? Who knows, but it's definitely an adequate competitor and finally worth the $$$$ that Apple charges.
-Toyin
13" MBA 1.8ghz i7
"It's all about the rims that ya got, and the rims that ya coulda had"
S.T. 1995
     
OreoCookie
Moderator
Join Date: May 2001
Location: Hilbert space
Status: Offline
Reply With Quote
Jun 24, 2003, 07:05 PM
 
Originally posted by Toyin:
Not being very technical I have some questions.

Why are the Dell numbers correct and the Apple numbers wrong?
Would optimizing for Altivec and 64bit significantly increase the G5s performance on these tests?

It seems like the PC world is just angry that Apple has finally put out a cutting edge box.

Before OSX the argument was, OS9 sucks, OS9 crashes and can't multitask, OS9 isn't configurable, OS9 isn't for 'power' users...etc
After OSX the argument became, Apples are overpriced and underpowered.

Apple has put out a comparable machine and the arguments are getting thin. Now it's that Apple is cheating on benchmark tests. Yet smoked dual 3.06ghz xenons in several real world tests.

Are the G5s the fastest personal computer? Who knows, but it's definitely an adequate competitor and finally worth the $$$$ that Apple charges.
First of all, there are official SPECmarks found here. There is a base and a peak value for each; the base value allows for limited optimizations whereas peak pretty much allows anything. Any vendor may hand in SPECresults; the computers have to be available at least one month after the submission. So in this sense, you cannot take some `fantasy configuration' and sell something quite different. Very often you read SPECint/SPECfp (estimate) which is a kind of inofficial SPECscore.

64 bit doesn't have to be faster than 32 bit. Take a look at the (official) SPECmarks of the Opteron -- they are higher in 32 bit mode.

One of the reasons is that the compilers are not optimized to such a high degree, because it takes some time for them to mature.

Same goes for the G5. IBM is putting some effort into the GCC, so we could hope the best. AFAIK the GCC doesn't autovectorize (i. e. automatically uses AltiVec as it sees fit). The Fortran compiler does.

Apple has done a bit too much tweaking with the results of the competitors. Both, the P4 and the Xeon have more than 1000 SPECint/SPECfp.

What does that mean: on that particular benchmark, the G5 is slower. The good news is that the G5 approximately doubles the performance of a G4 (per CPU!). Apple has done a lot of catching up with the G5.
I don't suffer from insanity, I enjoy every minute of it.
     
MindFad
Posting Junkie
Join Date: Sep 2001
Status: Offline
Reply With Quote
Jun 24, 2003, 07:38 PM
 
The demo of the real world apps kicking the hell out of the dual Xeon didn't excite anyone? And Apple being at probably dual 3GHz G5s within 12 months?
     
voodoo
Posting Junkie
Join Date: Mar 2001
Location: Salamanca, España
Status: Offline
Reply With Quote
Jun 24, 2003, 08:06 PM
 
I am psyched! This is a dream come true!!

The 970 is here!!!
I could take Sean Connery in a fight... I could definitely take him.
     
Earth Mk. II
Mac Elite
Join Date: Feb 2001
Location: Washington, DC
Status: Offline
Reply With Quote
Jun 24, 2003, 08:21 PM
 
FWIW,

Dell tipped the scales in their own favor on the "official" spec, by using a 3rd party library when compiling the CINT2000 and CFP2000 benches.

Apple also used a customized malloc().

So neither benchmark represents real-world results, whoop-de-do... what published benchmark does?


And yes - the demo of the apps kicking the dual Xeon's arse was quite fun to watch. Though I bet that has more to do with memory bandwidth and latency than anything else. The Dell uses PC2100 ECC DDR-RAM, while the G5's use PC3200 non-partiy DDR-RAM. The error checking on the Dell's RAM slows memory accesses down, let alone the narrower front-side and memory busses on the Dell(!) compared to the G5's.

Raw CPU power is important, but it isn't everything, it seems.
/Earth\ Mk\.\ I{2}/
     
FulcrumPilot
Mac Elite
Join Date: Mar 2001
Location: Vladivostok.ru
Status: Offline
Reply With Quote
Jun 24, 2003, 08:32 PM
 
whoop-deee-doooo!! controversy = more publicity!! its a win-win situation in the end if you ask me!

Here is Apple's response, posted in slashdot:

http://apple.slashdot.org/apple/03/0...id=126&tid=181

Quotable quote about all this fuss (from /. once again)
"And lastly, in the end, who cares? Unless you are regularly running 4 hour jobs from a console it is irrelevent. It is more important that you are productive with the interface and that is personal choice. Few consumer tasks (and even programming tasks) require that power - and the stuff that does is generally handled by specialised hardware. Then if they have the fastests today they won't tomorrow."
- bm_luethke (253362)
( Last edited by FulcrumPilot; Jun 24, 2003 at 08:41 PM. )
_,.
a solitary firefly flies at nite
into the darkness an endless flight
a million flashes of delight.
     
MikeM33
Mac Elite
Join Date: Feb 2002
Location: North-Eastern New Jersey
Status: Offline
Reply With Quote
Jun 24, 2003, 09:38 PM
 
I think, in the end, it really doesn't matter how these new G5's match-up against the AMD/Xeon/Intel chips. For us devoted Mac users it's a revolutionary set-up. If we're not "Faster" yet, we're definitely getting closer to catching-up, and possibly surpassing the competition. With "Big Blue" (IBM) behind things we should see some screaming performance boosts in the next year or two.

I think we're seeing the tip of the iceberg right now.

MikeM
     
Nonsuch
Professional Poster
Join Date: Jan 2001
Location: Riverside IL, USA
Status: Offline
Reply With Quote
Jun 24, 2003, 11:11 PM
 
Wonder if Haxial -- who uses Macs and really likes them, really he does -- will have the decency to post Joswiak's rebutal. And I wonder if many MacNNers have taken steps to encourage him to do so.
Find out just what any people will quietly submit to and you have found out the exact measure of injustice and wrong which will be imposed upon them.

-- Frederick Douglass, 1857
     
Superchicken
Addicted to MacNN
Join Date: Mar 2002
Location: Winnipeg
Status: Offline
Reply With Quote
Jun 25, 2003, 02:14 AM
 
Man I wonder how well Apple is gona fair now that its' got Big Blue doing the CPU work. I could see them possibly getting a 10% market share within the next few years. Who knows after that point, Apple could be bought out by IBM and run as a subsidary or enter into a partner ship where they both build boxes and both end up making the same on each machine. IE Apple charges as much for the OS as IBM charges for the processor.
     
Fyre4ce
Mac Elite
Join Date: Nov 2000
Location: Los Angeles, CA
Status: Offline
Reply With Quote
Jun 25, 2003, 02:51 AM
 
This guy also said that charging $2999 for the computer instead of $3000 is "absolutely rediculous" and Apple is "willing to mislead people when stating their prices."

For how long in our consumer culture has the least significant digit of most prices been 9? A Big Mac is $2.49, a DVD player is $299... why is this guy singling out Apple and Dell for doing this?

Calling such a practice "absolutely rediculous" makes me seriously question this gentleman's credibility.
Fyre4ce

Let it burn.
     
CRASH HARDDRIVE
Addicted to MacNN
Join Date: May 2001
Location: Zip, Boom, Bam
Status: Offline
Reply With Quote
Jun 25, 2003, 03:02 AM
 
I find all this 'fastest' stuff laughable. I've known of people who bickered over who had the fastest $3,000+ rig, and were still at it when anyone's 15 year old kid brother with $700 could whip either of their systems a scant mater of months later!

Unless you throw $3000+ around like it was $30 to most people, you're going to have the so-called 'fastest' system for all of a few months tops. So really, who cares?

All I care about is Apple seems to be back in the running with desktop systems that the price/performance ratio makes financial sense enough for me to even consider buying one. That's better news that worrying over who's bullcrap benchmarks are faster than whoevers for the matter of a few months that it will be even remotely relevant.
     
DeathMan
Mac Elite
Join Date: Aug 2001
Location: Capitol City
Status: Offline
Reply With Quote
Jun 25, 2003, 03:23 AM
 
For those of you that didn't read the rebuttle from joswiak (sp?), he says that they disabled hyperthreading to make the Intel proc perform better. He offered to compare them with hyperthreading turned on, since it would just make the G5 look that much better.

Had they run it with Hyperthreading on, this so-called mac user who mysteriously get windows screencaptures, would have complained that they left it on on purpose to slow things down.

Also, a comment on slashdot mentioned that in some benchmark tests Sun ran, they did the same thing, disabled hyperthreading to make the processor perform better.

He also said that SSE2 was NOT disabled, and that the tweaks done in behalf of the G5 were done because they would be included in the final G5, but weren't quite implemented yet.

This kid has NOTHING. Except maybe the $2999-$3000 argument.
     
maxelson
Addicted to MacNN
Join Date: Dec 2000
Location: Guidance Counselor's Office
Status: Offline
Reply With Quote
Jun 25, 2003, 07:43 AM
 
Then again... it is possible to make WAY too frigging much of these things.

I'm going to pull your head off because I don't like your head.
     
bamburg dunes
Mac Enthusiast
Join Date: Jun 2003
Location: Kalifornia
Status: Offline
Reply With Quote
Jun 25, 2003, 08:33 AM
 
SPEC benchmarks and all the rest are pretty much redundant in most cases, real world application is a far better way to gauge what sytems can do and how they benefit the user.

I remember when Intel were touting their Pentiums and trying to tell us how they trounced SGI (MIPS) in various benchmarks; but failed to mention that in real world situations, the SGI left the Pentium based machines in the dust when it came to throwing around hdtv, film and video images through the system. You have to take into consideration the machine as a whole, how the pipelines are implented, how the components work together and compsed of, and not just certain aspects.

Even today my 5 year old Octane will kick the ass off any Mac or PC (most of them) when it comes to bus throughput, the cpu is only 250mhz, but that has little bearing on how I am able to composite, edit and manipulate film scans and hdtv.

For me, prefering to use Macs, the new machines are pretty much just what I need and can't wait to get my hands on one; and whether or not PC's are technologically ahead, or faster is irrelevant since the OS X environment is a much better place for me to work in than Windows, and Irix. Any difference these days between Apple and PC hardware isn't that great and certainly not enough to make me want to jump to the other side.

Here's an interesting post I saw on a website:

MONDAY BROUGHT Apple's announcement of the G5, but an event that was supposed to be a tremendous bang seems to be fading into a whimper the longer people examine it. Moreover, underneath the tremendous hype, the G5 simply isn't looking that impressive, even if we ignore the questions about its raw performance that are currently zipping around the web.

While the G5 is a huge step up from the Motorola G4, and perhaps, justifies its price against that system, it simply doesn't look that impressive when measured against current systems in the PC world. Consider some of the G5's "cutting-edge" features:



DDR400: An incremental step from the G4's DDR333 (though the system can now make full use of its DDR) but absolutely nothing new. PCs have been supporting DDR400 since the nForce2 chipset arrived late last year, and third-party manufacturers for Intel added it not long after. Even Intel has now officially adopted the standard.


1 GHz HyperTransport Bus: The speed of the bus is impressive, but its only moderately faster than busses currently available on Opteron and the Pentium 4. Apple's G5 systems offer an independent bus for each CPU (just like Opteron). Again, there's nothing revolutionary here.



Support for Serial ATA: Again, a nice feature to see the Mac adopting, but scarcely anything new. SATA has been shipping on PC motherboards (even though it was fairly useless given that no machines support it) since the launch of the KT400 early last fall.



AGP 8X: The one thing AGP 8X has in common on both the PC and the Mac is that it's practically useless at the moment. That aside, it showed up on the PC side of things about the same time SATA support did, i.e., about 10 months ago.



USB 2.0 / FireWire / FireWire 800: The Mac debuted FireWire years before it began appearing widely on PCs and FireWire 800 is still Mac-only at the moment, but USB 2.0 has been present for PCs for almost 18 months, since at least the launch of the VIA KT333 chipset.

Now, don't misunderstand - these are all great developments for the Macintosh platform and it's fabulous that Apple users have finally got systems that are comparable to PCs in terms of their I/O speeds, bus speeds, RAM speeds, etc. Yet given the doubts swirling around Apple's G5 benchmarking there's reason to believe that the G5 is not comparatively as powerful as Apple would like it to seem. Some will say that testing both CPUs in Linux and using identical software is a good idea, in order to equalize the platform, while others will say that each system should be tested on the OS it'll use most often and in a tweaked configuration, which'd be OS:X for the Apple and a flavor of Windows for the P4 / Opteron (which wasn't tested at all).

That in and of itself is telling, because you'd think Opteron (as a fellow 64-bit CPU and as a CPU being used in 64-bit Linux at the moment) would be the CPU you'd want to test against. Maybe the reason they didn't is because the G5 2 GHz isn't standing up all that well to AMD's offering.

This is all highly relevant because Apple must find a way to justify the base price of that G5 tower and it can't rely on feature sets to do so, at least not compared to the PC world. Almost every one of the Mac's "new" features has been present in the PC world for months, and there's at least one major feature - namely the Opteron's integrated memory controller - that the Mac doesn't have at all. Apple's response to the rising furore over its benchmarking practice will say a lot about what route they intend to take in promoting their new system. Hopefully, it's not a route filled with a combination of Job's RDF and FUD.
PIXAR Animation Studios
     
Eug Wanker
Posting Junkie
Join Date: Jun 2003
Location: Dangling something in the water… of the Arabian Sea
Status: Offline
Reply With Quote
Jun 25, 2003, 09:55 AM
 
Originally posted by DeathMan:
For those of you that didn't read the rebuttle from joswiak (sp?), he says that they disabled hyperthreading to make the Intel proc perform better. He offered to compare them with hyperthreading turned on, since it would just make the G5 look that much better.

Had they run it with Hyperthreading on, this so-called mac user who mysteriously get windows screencaptures, would have complained that they left it on on purpose to slow things down.

Also, a comment on slashdot mentioned that in some benchmark tests Sun ran, they did the same thing, disabled hyperthreading to make the processor perform better.

He also said that SSE2 was NOT disabled, and that the tweaks done in behalf of the G5 were done because they would be included in the final G5, but weren't quite implemented yet.

This kid has NOTHING. Except maybe the $2999-$3000 argument.
To further support your argument (forgive my cross post from the Power Mac forum), straight from the horse's mouth, HyperThreading analysis from Dell on their machines:

"Incorporated into Intel Xeon processors, Hyper-Threading technology can provide great benefits to server applications. Unfortunately, it can also degrade system performance in certain scenarios such as those simulated by the compute-intensive SPEC CPU2000 and Linpack benchmarks."



     
Nonsuch
Professional Poster
Join Date: Jan 2001
Location: Riverside IL, USA
Status: Offline
Reply With Quote
Jun 25, 2003, 10:12 AM
 
What's-his-face replied to Joswiak. I've barely started reading it and it's a scream. Highlights so far:

As for the claim that Apple "provided full disclosure of the methods used in the tests", they most likely did this only to protect themselves legally -- the legal "fine print". Most people would not be able to understand the Veritest report due to its complexity and the technical knowledge required.
The situation here is unclear. I originally said that Apple/Veritest had disabled SSE2 for FP, thereby crippling FP performance. After further investigation, it seems I was mistaken about this particular point. The GCC "-mfpmath=sse" option (which Apple used) does in fact enable SSE2 as well as SSE1. HOWEVER, a number of people have e-mailed me to say that the "-mfpmath=sse" option DECREASES the FP performance on the Pentium 4 and should not be used. I have yet to fully confirm this, but if the people e-mailing me are correct, then I was *correct* in saying that Apple crippled the FP performance of the Pentium 4, but the *reason* I gave was not correct. Can anyone point me to some evidence which suggests that the "-mfpmath=sse" option should not be used?
Hilarious stuff. Follow n~s's link above for more.
Find out just what any people will quietly submit to and you have found out the exact measure of injustice and wrong which will be imposed upon them.

-- Frederick Douglass, 1857
     
Earth Mk. II
Mac Elite
Join Date: Feb 2001
Location: Washington, DC
Status: Offline
Reply With Quote
Jun 25, 2003, 08:13 PM
 
Ok, I've scoured the net, trying to get the info needed to compare the optimizations dell used in their own benchmarks and the ones apple used in theirs.

Here's what I've discovered:

Compiler options used by dell (based on their CINT2000 Result file at spec.org): -Qipo -QxW -O3 +FDO

breakdown for each option (as best I understand it - based off this information.):
  • ipo - interprocedural optimization across multiple files and link objects [link]
  • xW - optimize code to run exclusively on Pentium IV processors (uses SSE2) [link]
  • O3 - aggressive optimization (prefetching, loop transformation, etc.) (same function as in gcc) [link]
  • FDO -(??) Feedback Directed Optimization - apparently further optimizes code during runtime. [google link]

I think Apple's compile time options are a little more widely known and understood than these, but just to be through:

(info from the VeriTest report)

Dell Precision 650 flags: -O3 -march=pentium4 -mfpmath=sse
Dell Dimension 8300 flags: -O3 -march=pentium4 -mfpmath=sse
Apple G5 flags: -fast -lstmalloc

The flags used for the Dell systems are somewhat analogous to the ones used above (SSE2 enabled, P4 specific optimizations, etc.)

for the Apple flags:
  • fast - enables G5 specific optimizations (implies -O3)
  • lstmalloc - links to faster malloc libraries

I'm not sure why faster malloc libraries were used in the G5, it may have something to do with a discrepancy in OS X and x86-Linux memory management and/or efficiency. The most important thing, I believe, is that the library is single threaded, which may increase the benchmark speed.

In the end I'd guess that the various optimizations evened out across the two platforms (PPC and IA-32), though I'm not 100% sure. The customized malloc libs may be a sticking point, but I don't know how much of an effect on the result they'd have.

The one thing I am sure of though, is that I now know alot more about compiler optimization options after researching and writing this post.
/Earth\ Mk\.\ I{2}/
     
   
 
Forum Links
Forum Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Top
Privacy Policy
All times are GMT -4. The time now is 01:29 PM.
All contents of these forums © 1995-2017 MacNN. All rights reserved.
Branding + Design: www.gesamtbild.com
vBulletin v.3.8.8 © 2000-2017, Jelsoft Enterprises Ltd.,