[equinox]

I just saw the new mac-mini Xbench charts! Unbeliebable
Look athttp://www.macintouch.com/perfpack/comparison.html
If you look at the CPU and Thread bars the iMac G5 1,8 is slower than the G4 1,42 G4 mini! How is this possible? I checked my iMac with Xbench myself and i got 161! In terms of CPU and threads the G5 is slower than a G4! How can this be?

[Grrr]

I don't geddit? Mac Mini's don't score anything like what a G5 does. I looked at the Xbench site myself only yesterday.

[discotronic]

The low score for the iMac is with the processor set on Automatic settings. If the mini had an auto setting the score would be lower also. The true score is with the processor set to Highest. The processor set to Automatic is not a valid test of the iMac's ability since the CPU isn't being utilized to it fullest potential.

From what I see the iMac beat the mini in every single area of testing.

[Simon]

Maybe we could just conclude (once and for all) that XBench really really sucks.

G5s score worse than they should with XBench. It's been like this since day one. (Apart from the fact that XBench can't even make sure test are always performed with the same energy saving settings)

[Grrr]

Originally posted by Simon:
Maybe we could just conclude (once and for all) that XBench really really sucks.

G5s score worse than they should with XBench. It's been like this since day one. (Apart from the fact that XBench can't even make sure test are always performed with the same energy saving settings)

Oh lord, not this dead horse beating session again.. Although I agree with you on the energy saver thing, the rest is BS. There is little to nothing wrong with Xbench.

[hudson1]

Well, if XBench doesn't float your boat, how about this from the Macintouch tests:

"Using QuickTime Pro 6.5.1 and QuickTime Player, we export a high-quality 50-second DV file to MPEG-4 format. Source and target files are on the hard drive. We use the standard "Default" settings.

This is a good real-world test of system performance, and the results are surprising: Right out of the box, the eMac, Mini and lowly iBook G4 handily outperform the iMac G5 at Apple's standard settings!"

This isn't quite correct as the iBook G4 on "high" beat the iMac G5 on "auto" unless "high" is the default for the iBook.

[Simon]

Originally posted by Grrr:
There is little to nothing wrong with Xbench.

What a bunch of bull. In the last two years there have been countless debates on this forum because Xbench delivered scores which made absolutely no sense. Xbench has repeatedly given numbers that contradicted real-world benchmarks. Many very hardware-savvy people have wondered about how in the world Xbench gave results which could obviously not be true.

The fact that Xbench can't make sure the testing environment is at least roughly equal and doesn't even note what ES settings were chosen during the tests, is a major design flaw. This alone makes it beta software at the most. It by far doesn't qualify to quantitatively measure hardware. It is a dirty hack made to let people believe every gradma can easily benchmark hardware.

You can continue to hail this product as long as you want. Go ahead and buy a cheap 1.2GHz G4 iBook and believe it is completely up to par with a 1.8GHz iMac G5. If it makes you feel better, great! I'm sure Apple just charges the huge difference in price because the G5's all just about marketing. After all these years of development of very skilled departments at IBM or Apple, how could it offer any benefit over the aging G4? How silly of us (and the entire mac related press) to believe there could be any advance.

[Simon]

Originally posted by hudson1:
This isn't quite correct as the iBook G4 on "high" beat the iMac G5 on "auto" unless "high" is the default for the iBook.

The Macintouch tests are very interesting. But their Xbench results contradict about every other test they ran. I think people should take a good look at some of the other benchmark results - they give a much better impression of the Mac mini's performance.

And I think we should settle on a few common sense points:
• the Mac mini is an excellent computer
• $599 gets you a whole lot of CPU power
• with equal ES saver settings any SP G5 desktop Mac will beat any SP G4 desktop Mac in CPU tests
• if a 1.25GHz G4 beats a 1.8GHz G5 in a CPU test, the testing conditions are either not equal or the tool used to 'benchmark' sucks

The Mac mini is a switcher computer. It's meant to be a cheap way to get PC users to try out the Mac user experience. It is by absolutely no means a replacement of an iMac or iBook. The former is much more powerful (and more expensive) and the latter is made with portability in mind. They are geared towards entirely different markets.

What was the reason for this thread again?

[Sparkletron]

After all these years of development of very skilled departments at IBM or Apple, how could it offer any benefit over the aging G4? How silly of us (and the entire mac related press) to believe there could be any advance. [/B]

I'm not surprised by the benchmarks, which I believe are accurate. Much of what you're paying for in a G5 is its potential, which won't be realized until Apple releases an OS that can take advantage of the new 64-bit architecture. Take memory for example; yes you can get 8 GB of RAM but each individual non-OS process is still limited to the same maximum heap size as with the 32-bit G4. So while all that memory may be present, no single app can use it.

Hopefully Tiger will improve the G5 performance landscape.

-S

[hudson1]

Originally posted by Simon:
The Macintouch tests are very interesting. But their Xbench results contradict about every other test they ran. I think people should take a good look at some of the other benchmark results - they give a much better impression of the Mac mini's performance.

I did take a good look at the all of the results before I posted earlier.

The only test performed by Macintouch where the iMac G5 at its standard or out-of-the-box processor setting eclipsed the G4 systems was Cinebench.

Xbench and the DV-->MP4 export were both slower on the iMac G5 (std.). File duplication was either faster or slower depending on whether it was compared to a G4 with a 2.5" or 3.5" hard drive. iTunes ripping was equivalent.

There's a reason why the iMac G5's default processor setting is "standard" and not "high". Is there anyone who thinks the reason is to just keep the G4 systems looking competitive by comparison? Whatever that reason, it doesn't apply to G4 desktop systems. Therefore, the best Apple-to-Apple comparison should be with the G5's "standard" setting until Apple decides that these machines are best set and run at their "high" setting.

[Eug Wanker]

Check out my 1.7 GHz Cube's Xbench CPU score: 203. That's faster than the G5 2.0.

In other words, Xbench's CPU score has little correlation with real-world CPU performance.

[discotronic]

Originally posted by hudson1:
I did take a good look at the all of the results before I posted earlier.

The only test performed by Macintouch where the iMac G5 at its standard or out-of-the-box processor setting eclipsed the G4 systems was Cinebench.

Xbench and the DV-->MP4 export were both slower on the iMac G5 (std.). File duplication was either faster or slower depending on whether it was compared to a G4 with a 2.5" or 3.5" hard drive. iTunes ripping was equivalent.

There's a reason why the iMac G5's default processor setting is "standard" and not "high". Is there anyone who thinks the reason is to just keep the G4 systems looking competitive by comparison? Whatever that reason, it doesn't apply to G4 desktop systems. Therefore, the best Apple-to-Apple comparison should be with the G5's "standard" setting until Apple decides that these machines are best set and run at their "high" setting.

Are you saying that the PowerMac G5 needs these same settings? When comparing an iMac G5 1.8GHz to a PowerMac G5 1.8GHz should the processor be set to Automatic on the iMac so as to give the PM an unfair advantage? Since the PM doesn't have this setting what kind of results do you think will be presented? Your reasoning if very flawed. Just because the processor is set to Auto by default doesn't mean that is the way Apple has intended for the processor to run. Why else would there be a setting for Highest?

Intentionally crippling a processor for a benchmark isn't the correct way to obtain accurate test results. If you think so you are slightly on the delusional side.

[Sparkletron]

Originally posted by discotronic:
Just because the processor is set to Auto by default doesn't mean that is the way Apple has intended for the processor to run.

But that's exactly what he's saying. Otherwise why have settings at all? Why not ship all G5s running as fast as they possibly can? I think the reason has to do with noise (more power = more heat = more fans = more noise).

-S

[Simon]

OMG.

Who gives a flying fsck what setting Apple uses default on some Mac? We have a system preference to set this value to whatever we like depending on our priorities (power, heat, noise). But, when you conduct a CPU benchmark, obviously such settings have to be equal if the results should have any meaning. Comparing an eMac running at highest with an iMac G5 running at lowest is plain and simply moronic. End of story.

[hudson1]

Originally posted by Simon:
Comparing an eMac running at highest with an iMac G5 running at lowest is plain and simply moronic. End of story.

There's no such thing as running an eMac at "highest"; it only has one speed.

If you're going to claim that the only valid comparison is when you manually change the processor setting in the iMac G5 away from what Apple ships it with (and recommends for most uses), then who's to say that you shouldn't be allowed to over-clock the G4 to get better results on those machines? Sure, it's not the same thing but where do you draw the line?

[larkost]

Originally posted by Sparkletron:
I'm not surprised by the benchmarks, which I believe are accurate. Much of what you're paying for in a G5 is its potential, which won't be realized until Apple releases an OS that can take advantage of the new 64-bit architecture. Take memory for example; yes you can get 8 GB of RAM but each individual non-OS process is still limited to the same maximum heap size as with the 32-bit G4. So while all that memory may be present, no single app can use it.

Hopefully Tiger will improve the G5 performance landscape.
-S

64bit OS's/computers are not faster than 32bit OS's/computers unless you are using processes that need >32bit memory spaces (in which the 32bit OS requires segmentation, and that slows things down enormously), or you require 64bit precision (and the 32bit chip does not have 64bit hardware math). In fact depending on the exact implimentation you can often get a slight slowdown in going to 64bits because of the added data size to move around.

There is no magical speedup to be had, and 10.4 will not be a 64bit OS signifigantly more than 10.3 is. There will be some more work done, especially in the compiler, but this will not be a pervasive thing since the OS will still need to run on 32bit hardware. There is also the matter of most applications not benefiting from going 64bit.

10.4 may well speed up things on the G5 by being compiled in a way that is better optimized for the G5 chip, but this will probably have almost nothing to do with being a 64bit chip. A similar speedup for the G4's is also likely (due to general compiler optimizations).

[discotronic]

Originally posted by hudson1:
There's no such thing as running an eMac at "highest"; it only has one speed.

If you're going to claim that the only valid comparison is when you manually change the processor setting in the iMac G5 away from what Apple ships it with (and recommends for most uses), then who's to say that you shouldn't be allowed to over-clock the G4 to get better results on those machines? Sure, it's not the same thing but where do you draw the line?

The 1.8GHz in the low end PowerMac is the same processor as the 1.8GHz in the iMac. Do you think the PM is running "by default" the same as the iMac on Automatic?

I was right. You are delusional.

[hudson1]

Originally posted by discotronic:
The 1.8GHz in the low end PowerMac is the same processor as the 1.8GHz in the iMac. Do you think the PM is running "by default" the same as the iMac on Automatic?

I was right. You are delusional.

Well, I'm not going to get into some stupid name-calling excercise.

The short answer is that the "Automatic" processor mode in a PM G5 and in an iMac G5 don't appear to be the same thing. From the previous Macintouch machine comparison (Conclusion):


"In addition, the iMac G5 and the Power Mac G5 take the 'Auto' mode penalty differently: The iMac G5 suffers a greater slowdown. Why? We simply don't know what's happening at that technical level. Is it differences in bus speeds? Is is differences between the old and new "G5" chips? Is it related to thermal issues and the G5's 'slewing' to manage heat? The jury is still out."

[indigoimac]

I beat u all and got a 41 w/ my iMac, harr!

[Simon]

Originally posted by hudson1:
The short answer is that the "Automatic" processor mode in a PM G5 and in an iMac G5 don't appear to be the same thing.

Goodness gracious! Don't you understand, any setting other than highest is completely meaningless in CPU benchmarking? It doesn't matter if iMacs run with other settings at 'Automatic' than PowerMacs. It's completely irrelevant when comparing their two CPUs (which are identical BTW).

Here's the deal with ES settings: We're not setting the processor faster or slower like Apple's terminology could make you think. The processor has a nominal speed which is the 1.8 GHz in the 20" iMac or 1.25/1.42 GHz in the Mac mini. Anything you set in ES will either slow the clock down or decrease the load threshold for the clock slow down. It's not as if users had the choice of running their iMacs at 1.4, 1.8, 2.2 GHz - there is only 1.8 GHz non-stop or clock settings below 1.8 Ghz (either continuously or variable).

And since the only 100% reproducible setting across all Macs is running the CPU at its nominal frequency, this is the frequency that should be tested. There is no other ES settings for which testing between different models makes any sense, because for one we don't know exactly what is being altered when we reduce the ES settings and mainly, because processors step down in different ways - a 970fx does something else when you got to 'reduced' than a 7447.

So, in one sentence: When comparing one Mac to another in CPU benchmarks, the ES settings must be at a maximum - any other settings renders meaningless results.

And since Xbench can't even take care of this, it's one hell of a lousy miserable POS hack. And that's why every other weak somebody arrives at this board and posts some moronic thread like 'OMGWTF??!!!???? - my 700MHz eMac just knocked the socks off of my 2.5GHz G5 in Xbench - is this Apple's dirty secret???'. I'd like to see the author of Xbench made to answer every single posting like that. And then, he could get me a beer.

[hudson1]

Originally posted by Simon:
So, in one sentence: When comparing one Mac to another in CPU benchmarks, the ES settings must be at a maximum - any other settings renders meaningless results.

And since Xbench can't even take care of this, it's one hell of a lousy miserable POS hack. And that's why every other weak somebody arrives at this board and posts some moronic thread like 'OMGWTF??!!!???? - my 700MHz eMac just knocked the socks off of my 2.5GHz G5 in Xbench - is this Apple's dirty secret???'. I'd like to see the author of Xbench made to answer every single posting like that. And then, he could get me a beer.

Why are you arguing your point with me? It's Ric Ford that you should be addressing this to.

However, it's clear that we're talking about different things. If your contention is that the processor capability is what matters then yes, we should probably go digging into System Preferences and reset the Energy Savings preferences to the non-default before running the test. My point is that what matters most is how fast is the machine when configured the way Apple primarily intends it to be used. That to me means leaving the Energy Savings setting to its default state.

From Ric Ford's tests, it's obvious that a default-state iMac G5 is roughly on par with a default-state eMac G4 or Mac mini. The PowerMac series, in either ES state, are faster than any of the consumer machines. If this isn't true, then please explain why.

[Grrr]

Originally posted by Simon:
So, in one sentence: When comparing one Mac to another in CPU benchmarks, the ES settings must be at a maximum - any other settings renders meaningless results.

And since Xbench can't even take care of this, it's one hell of a lousy miserable POS hack. And that's why every other weak somebody arrives at this board and posts some moronic thread like 'OMGWTF??!!!???? - my 700MHz eMac just knocked the socks off of my 2.5GHz G5 in Xbench - is this Apple's dirty secret???'. I'd like to see the author of Xbench made to answer every single posting like that. And then, he could get me a beer.

With the way you rant on and on, i'd honestly be surprised if you are even old enough to drink beer? I mean really.. You do go on and on like some caffeine hyped know-it-all student.
Anyway, I digress.. In other words.. If users are smart enough to tweak their energy saver settings to 'highest' (and some are smart enough, believe it or not) Then Xbench is in fact, perfectly good? Right! After all, Xbench is really very simple.. It gives the computer a task, and the less time it takes to complete it, then the faster it is. Higher scores, faster computer, lower scores, slower computer! Its not rocket science, and really does not need to be any more complicated than that.
At the moment you are mostly blaming Xbench for a problem that is in fact down to users either not using the software properly, or not using their own computers properly.

[Simon]

Grrr, you are entirely missing the point. This isn't about a rant, it's about the spreading of misinformation.

Benchmarking is not simple. Especially good benchmarking. If you want to know how well a certain Mac performs, and you're not an expert, go to a site like BareFeats and see how a pro benchmarks systems. But don't use Xbench if you lack the knowledge to judge its output. It simplifies the testing procedure to the point where every grandma can benchmark a Mac, but, the results have basically no meaning. So why use the tool in the first place?

If people would read sites like BareFeats instead of trying to Xbench their own boxes, we wouldn't have to explain to new users that a dual 2.5 GHz G5 is faster than a 700 MHz eMac no matter what you do.

Xbench tries to pull a MS: Make people think difficult tasks are simple by dumbing down the interface and taking away options. Unfortunately that only renders confusion.

I'm done here. If some people want to trust Xbench when it shows a mini beating the pants off a 1.8GHz G5 in CPU tests, so be it.

[amazing]

Suppose you wanted to benchmark racehorses by testing them while they're trotting. After all, the argument might go, most of the time they trot about.

Would any reasonable person think that testing a trotting racehorse actually measures how fast they are?

[hudson1]

Originally posted by amazing:
Suppose you wanted to benchmark racehorses by testing them while they're trotting. After all, the argument might go, most of the time they trot about.

Would any reasonable person think that testing a trotting racehorse actually measures how fast they are?

Well, you're onto something but I don't think you have it quite right, IMHO.

If I wanted to test the speed of trotters, I'd leave the harness and sulky on them and see how fast they go. I wouldn't take the harness and sulky off, pretending that they're thoroughbreds when they aren't, and claim that I had a valid comparison for trotters.

There must be a reason that Apple has set the iMac G5 default for the jockey to ride on a sulky (auto mode) and not on a saddle (fast mode). No one seems to want to address this question when they state that the only valid comparisons must be done in fast mode.

[discotronic]

Originally posted by hudson1:
Well, you're onto something but I don't think you have it quite right, IMHO.

If I wanted to test the speed of trotters, I'd leave the harness and sulky on them and see how fast they go. I wouldn't take the harness and sulky off, pretending that they're thoroughbreds when they aren't, and claim that I had a valid comparison for trotters.

There must be a reason that Apple has set the iMac G5 default for the jockey to ride on a sulky (auto mode) and not on a saddle (fast mode). No one seems to want to address this question when they state that the only valid comparisons must be done in fast mode.

I am at a loss. I would say that most people who have read your take on benchmarking are also.



Wait!! I get it now!



You don't have a clue

[hudson1]

Originally posted by discotronic:
I am at a loss.

You're at a loss and I don't have a clue. I guess that puts us about "neck and neck" (to stay with the horse racing analogy).

[amazing]

Think of the target audience: Those people don't necessarily care about higher processor settings, they just want pictures of the kids, email without viruses, etc.

So, the "extra gear" on the iMac which allows you to go faster is something for geeks among us. The target audience may eventually find out about it and make use of it, at which point it will be an added feature for them as well.

The opinion that Xbench should be testing that extra gear is valid, but arguing about it is futile--changing Xbench isn't in the cards. It would be helpful in making a buying decision if Xbench tested it reliably, because it's very confusing when people think that the mini is as fast as a G5 iMac.

Mostly, if you really think about it, the underlying bone of contention is about why Apple hobbles the entry level Macs. For example, I want a 1.42 ghz 12" iBook that does monitor spanning, with an 80 GB HD. In other words, I want a 12" iBook with the specs of the mini.

That's apparently too much to ask of Apple.

[Simon]

Originally posted by amazing:
Mostly, if you really think about it, the underlying bone of contention is about why Apple hobbles the entry level Macs. For example, I want a 1.42 ghz 12" iBook that does monitor spanning, with an 80 GB HD. In other words, I want a 12" iBook with the specs of the mini.

Well, I think what you really want is a 12" PowerBook, but at the price of an iBook.

[amazing]

Originally posted by Simon:
Well, I think what you really want is a 12" PowerBook, but at the price of an iBook.

Well, perhaps--except that I really like the excellent wireless reception of the iBook and the more rugged package. What I really should have said is to wonder why you can't order a 12" iBook with the option to pay for a faster processor? I just can't stand the lower quality screen (type gives me eye fatigue and headache) of the 14" iBook and the added weight.