Welcome to the MacNN Forums.

Developer · Jul 5, 2003, 03:47 AM

http://www.macnews.de/index.php?_mcnpage=6619

Eug Wanker · Jul 5, 2003, 06:01 AM

Originally posted by Developer:
http://www.macnews.de/index.php?_mcnpage=6619

The statements there are reasonable. Cooler heads prevail.

The Placid Casual · Jul 5, 2003, 06:31 AM

Fair and even handed... a good article!

Spliffdaddy · Jul 5, 2003, 07:04 AM

He believes Apple "stacked the deck" in its own favor.

That's a nice way to describe it.

From the company that still advertises the G4 as a Pentium-crushing 'supercomputer' - we bring you the next generation of PowerMac that almost equals a P4 from last year...

Person Man · Jul 5, 2003, 07:50 AM

Originally posted by Spliffdaddy:
He believes Apple "stacked the deck" in its own favor.

Apple is behaving no different than any computer manufacturer out there when it comes to benchmarking. Every computer manufacturer is going to use the compiler and optimizations that will give them the edge over their competition.

The difference is that Apple displayed not only the results (which is where most manufacturers stop), but they even described their methods. If more hardware manufacturers detailed their methods, if would be harder for their marketing departments to make their claims.

Conscript · Jul 5, 2003, 08:39 AM

It could have been worse. Apple could have stacked the deck in their favor and then used one of Nvidia's heavily "optimized for benchmark" cards. All in all, I really don't think it's relevant comparing apples to intels anyway. There are so many intangible reasons to use an apple that are not reflected in benchmarks.

cowerd · Jul 5, 2003, 10:30 AM

Apple is behaving no different than any computer manufacturer out there when it comes to benchmarking. Every computer manufacturer is going to use the compiler and optimizations that will give them the edge over their competition.

yeah it also helps if you click on the link and read the article. to help you out here's the FULL QUOTE you were responding to:

I think the whole thing has been blown way out of proportion. When I was first watching the keynote live, I thought that Apple had stacked the deck, and I still entertain the possibility that they did so deliberately. But who cares? It's a keynote, and it's their biggest hardware introduction since the original G4, so they're going to make it look really good at all costs--there's just too much at stake for them. I mean, I hate to sound like an Apple apologist, but any deck-stacking that was done was extremely tame and forgivable in the notoriously underhanded world of competitive benchmarking. Besides, anybody who plans an expensive workstation purchase based on vendor-supplied benchmarks of unreleased hardware shouldn't be in charge of a credit card or a checkbook.

Person Man · Jul 7, 2003, 08:47 AM

Originally posted by cowerd:
yeah it also helps if you click on the link and read the article. to help you out here's the FULL QUOTE you were responding to:

I most certainly DID read the article before I responded, and I stand by my statement that Apple was behaving no different than any other company would. The big difference is that they detailed their methods.

Does Dell (or any other "big name" PC maker) use an independent hardware test firm? They might, but we'd never know it because they don't reveal their methods, just the scores. How do we know they didn't "just make it up?"

Note: there may be some who do reveal their methods, but I have not run across any myself.

istallion · Jul 7, 2003, 02:39 PM

Originally posted by Person Man:
Does Dell (or any other "big name" PC maker) use an independent hardware test firm? They might, but we'd never know it because they don't reveal their methods, just the scores. How do we know they didn't "just make it up?"

If you look through the running and submission rules, you can see how SPEC is tested. OS configs and flags are detailed in the results database. If scores are suspect, they can be investigated for non-compliance.

I think if Apple had benched their machines like the rest of the industry and submitted their results to SPEC, people would be much less suspicious.

Person Man · Jul 7, 2003, 05:13 PM

Originally posted by istallion:
I think if Apple had benched their machines like the rest of the industry and submitted their results to SPEC, people would be much less suspicious.

So Apple didn't send their results to SPEC, but they used an independent lab (are there any other manufacturers who did that?), and the lab reported the methods used in full.

Spliffdaddy · Jul 8, 2003, 12:30 AM

The problem is that Dell submitted SPEC scores for the machine that Apple tested the G5 against. Why didn't Apple simply do what ALL other manufacturers have done? That is, comply with the SPEC testing sequence and post the results.

Because the Dell would have won, that's why.

istallion · Jul 8, 2003, 12:50 AM

Originally posted by Person Man:
So Apple didn't send their results to SPEC, but they used an independent lab (are there any other manufacturers who did that?), and the lab reported the methods used in full.

All an independent lab does is ensure that methods & results aren't fabricated. Nobody in the article or the thread was claiming that. Veritest just runs the tests according to Apple's instructions. I suppose once they publish that malloc lib, they may submit the results. Although it's unsuitable for many uses. It seems like a waste, buying the software, hiring Veritest, pimping the results in the keynote and on their website, but not submitting results, unless there is some reason.

And it's silly for Apple to bench the Dell. Just as silly as it is for Dell to bench the G4 or IBM to bench the Itanium, for the reasons given in the article.

OreoCookie · Jul 8, 2003, 06:42 AM

Every manufacturer cheats more or less on benchmarks. Be it software or hardware. c't magazine for instance doesn't use the synthetic 3dmark for years, because it's been so skewed by tweaked drivers that its result are next to meaningless.

But the articles was good.

Seriously, most of the guys complaining are professional techies, they should know better than taking every word of a manufacturers benchmark

The Godfather · Jul 8, 2003, 09:34 AM

Best quote:

"Besides, anybody who plans an expensive workstation purchase based on vendor-supplied benchmarks of unreleased hardware shouldn't be in charge of a credit card or a checkbook."

Gene Jockey · Jul 8, 2003, 01:34 PM

Originally posted by Spliffdaddy:
The problem is that Dell submitted SPEC scores for the machine that Apple tested the G5 against. Why didn't Apple simply do what ALL other manufacturers have done? That is, comply with the SPEC testing sequence and post the results.

Because the Dell would have won, that's why.

Or, it could be that the Apple benchmarks are for pre-production machines. The G5s tested weren't exactly what we'll be getting next month, and Apple didn't want to submit premature results for inclusion in the database. I'm betting once these machines ship you'll see final SPEC scores and they'll be submitted. If I were in charge, it's what I'd do.

--Josh

itai195 · Jul 8, 2003, 02:01 PM

wrong thread

brinj · Jul 8, 2003, 08:34 PM

Originally posted by Gene Jockey:
Or, it could be that the Apple benchmarks are for pre-production machines. The G5s tested weren't exactly what we'll be getting next month, and Apple didn't want to submit premature results for inclusion in the database. I'm betting once these machines ship you'll see final SPEC scores and they'll be submitted. If I were in charge, it's what I'd do.

--Josh

NASA tested the G5 against a G4 and a P4

http://members.cox.net/craig.hunter/g5/
32% better than P4 ain't bad.

CubeBoy · Jul 9, 2003, 05:45 PM

Originally posted by brinj:
NASA tested the G5 against a G4 and a P4

http://members.cox.net/craig.hunter/g5/
32% better than P4 ain't bad.

32% better than the 2 GHz Pentium 4, about parity with the 2.66 GHz Pentium 4, and considerably below a 3 GHz Pentium 4. Of course, the G5 had a pretty crappy compiler but than again, the Pentium 4 wasn't using it's best compiler either.

MindFad · Jul 9, 2003, 06:07 PM

Originally posted by CubeBoy:
32% better than the 2 GHz Pentium 4, about parity with the 2.66 GHz Pentium 4, and considerably below a 3 GHz Pentium 4. Of course, the G5 had a pretty crappy compiler but than again, the Pentium 4 wasn't using it's best compiler either.

Right ... on one processor.

Anyway, I was more impressed watching the G5 kick ass in the real world application demos at the conference, particularly in the 3-D app and Photoshop. No one finds that more impressive? Or was that rigged, too?

Everyone's got an answer. Man, MacNN is gonna be so fast on my G5*.

* Cannot yet afford a G5.

Superchicken · Jul 9, 2003, 08:30 PM

good article...

GCC was chosen because it's the same thing for both platforms, they weren't just testing which compiles code fastest, they did that with Photoshop which is optomized for each platform and they kicked butt!

colpolyp · Jul 13, 2003, 01:07 PM

Even if Apple really DID cheat and in the real world a G5 would have the same performance as a P4, no.... even if the G5 had 80% of the P4's performance, I would still prefer the G5 just because I can run OSX on it!

Brazuca · Jul 13, 2003, 03:33 PM

Originally posted by CubeBoy:
32% better than the 2 GHz Pentium 4, about parity with the 2.66 GHz Pentium 4, and considerably below a 3 GHz Pentium 4. Of course, the G5 had a pretty crappy compiler but than again, the Pentium 4 wasn't using it's best compiler either.

I think you misinterpreted the results of the test.

Take a look at this thread at Arstechnica. The guy who ran the tests does a little Q&A and explains a little better the results.

CubeBoy · Jul 13, 2003, 08:28 PM

Originally posted by Brazuca:
I think you misinterpreted the results of the test.

Take a look at this thread at Arstechnica. The guy who ran the tests does a little Q&A and explains a little better the results.

I understood it fairly well, my original post was addressing the 32% better remark, which could only be attained doing a processor to processor comparison. I don't think theirs any doubt that dual 2 GHz G5s could outperform a single 3 GHz Pentium 4, the real question well be whether or not it'll outperform Dual Xeons or Dual Opterons.