[kupan787]

Apple says that a dual 1 GHz xserve should get 15 GFLOPS (from their xserve website). So this 33 xserve setup should be getting 495 GFLOPS (theoretical), but it is only getting 217 GFLOPS (which comes out to 3 GFLOPS per dual proc xserve). So something is wrong.

My single proc 867MHz gets 3.3 GFLOPS using AltiVec Fractal Carbon on default settings. If they are only averaging 3 GFLOPS per dual 1GHz, that seems funny. What do people with dual 1GHz machines get when running AltiVec Fractal Carbon?

I know that the 495 GFLOPS number is theoretical, but it should be doing better than 43.8% of theoretical I would think.

[mac-at-kearsarge]

Grant that in theory you might get more, but you seem to be complaining about ONLY HAVING 217 GFLOPS!?!?!? If this is so terrible, perhaps you should trade your XServer cluster for my 600 iBook Combo?

I Know you weren't really complaining, but if I had that much processing power at my disposal, I would gladly take what I got without question it.

[alex_kac]

Its distributed computing and may have some overhead or latency. Just a guess of course - but usually to get rid of that kind of latency or overhead for massively parallel computers requires a lot of engineering.

[dot_nix]

Originally posted by kupan787:
Apple says that a dual 1 GHz xserve should get 15 GFLOPS (from their xserve website). So this 33 xserve setup should be getting 495 GFLOPS (theoretical), but it is only getting 217 GFLOPS (which comes out to 3 GFLOPS per dual proc xserve). So something is wrong.

My single proc 867MHz gets 3.3 GFLOPS using AltiVec Fractal Carbon on default settings. If they are only averaging 3 GFLOPS per dual 1GHz, that seems funny. What do people with dual 1GHz machines get when running AltiVec Fractal Carbon?

I know that the 495 GFLOPS number is theoretical, but it should be doing better than 43.8% of theoretical I would think.

Yes it should be probably doing better but there are many other factors that need to be considered. First of all, altivec fractal carbon was not built for clustering. Second, Mac OS X Server 10.2 does not YET have native clustering capabilities. Third, a third party software was used for clustering. And then the switch they used was a 3COM 100BaseT Switch while all the Xserves are standard with 1000BaseT. What I mean is that if they used a 1000BaseT Switch it would probably make a difference.

And just a note if you did actually use Altivec Fractal Carbon on an XServe Dual 1GHz and NOT CLUSTER IT and ADD EACH ONE OF THEM YOU PROBABLY WOULD GET BETTER THAN 43.8% THEORETICAL SPEED.

[BlackGriffen]

Originally posted by kupan787:
Apple says that a dual 1 GHz xserve should get 15 GFLOPS (from their xserve website). So this 33 xserve setup should be getting 495 GFLOPS (theoretical), but it is only getting 217 GFLOPS (which comes out to 3 GFLOPS per dual proc xserve). So something is wrong.

My single proc 867MHz gets 3.3 GFLOPS using AltiVec Fractal Carbon on default settings. If they are only averaging 3 GFLOPS per dual 1GHz, that seems funny. What do people with dual 1GHz machines get when running AltiVec Fractal Carbon?

I know that the 495 GFLOPS number is theoretical, but it should be doing better than 43.8% of theoretical I would think.

Aside from you math skills blowing chunks...

You're right, Apple's numbers put something that big at a 495 theoretical peak. 217 Gflops divided by 33 xserves is 6.58 Gflops per Xserve (3.29 GFLOPS per processor). Which is probably right about in line with the fact that the GHz processors have to manage extra communications that a cluster has to handle, as well as XServes waiting for data and/or handling cluster administration (if there is a master/slave system, that is).

I'd wager that Apple's numbers are rather inflated, but not nearly as bad as you made it sound.

BlackGriffen

[kupan787]

Originally posted by BlackGriffen:

Aside from you math skills blowing chunks...

Crap, sorry I probably sounded like a troll or something, but really I am not.

Ya I made a slight mistake when saying that it was 3 GFLOPS per xserve, when I meant per proc (so ya, around 6 GFLOPS for a xserve). But I don't see how my math skills were blowing chunks by this one mistake. All my other figures were fine (the % of theoretical was valid).

My question was somewhat answered. I didn't realize there would be that much overhead for clustering. And someone else mentioned the fact they were using a 100 switch instead of a 1000 switch, and altivec fractal is not built for clustering.

My point was that I would think, even after overhead, that the machines should be getting better than 43.8% of theoretical. Like maybe 60% or something. Maybe in the future we will see better numbers?

[Cipher13]

You got a 33 cluster of Xserves, and you're concerned about how they perform on a Fractal test?

Gee, well, get folding@home and see how that runs on there. If it doesn't perform, take them back.

[Evangellydonut]

Originally posted by kupan787:
My question was somewhat answered. I didn't realize there would be that much overhead for clustering. And someone else mentioned the fact they were using a 100 switch instead of a 1000 switch, and altivec fractal is not built for clustering.

My point was that I would think, even after overhead, that the machines should be getting better than 43.8% of theoretical. Like maybe 60% or something. Maybe in the future we will see better numbers?

Even Gbit ethernet isn't really the best thing for clustering. I know at least 2 local companies that makes clustering specific hardware built to be cheaper and faster than Gbit switched networks...('course both companies are pretty much all graduates from my university, but that's another story)

[Deal]

I'll bet we are going to see a whole lot of Xserve at Jan Macworld. I think they must have something to surprise us. Why?

Because Apple's Raid device was supposed to be released by the end of the year. Then rumor had it that it would be out early than expected. Now I just heard (although 3rd hand but originating from Apple) that it is delayed.

I'll bet they are holding it back. There are other Raid devices out there (so that's nothing new, even if they are using ATA drives with hardware raid). I can't imagine it taking this long unless they are pairing it's release with something GOOD.

Everybody knows Xserve has to be cluster-able. Apple has bought or collaborated with just about every piece of software needed to make Pixar style animated movies (and movie affects, etc...). How can it not be cluster-able (and I mean low over head cluster-able)?!

Whatever Apple needs to get done to floor this market, it will do. I'm no wizard when it comes to Movie affects, etc... But I know Steve wants to see Apple the best in this market. With his knowledge base at Pixar I'm sure he'll get it done.

Xserve will be (hopefully just one of) the main topic(s) at MWSF

[rogerkylin]

Originally posted by Deal:
I'll bet we are going to see a whole lot of Xserve at Jan Macworld. I think they must have something to surprise us. Why?

Because Apple's Raid device was supposed to be released by the end of the year. Then rumor had it that it would be out early than expected. Now I just heard (although 3rd hand but originating from Apple) that it is delayed.

I'll bet they are holding it back. There are other Raid devices out there (so that's nothing new, even if they are using ATA drives with hardware raid). I can't imagine it taking this long unless they are pairing it's release with something GOOD.

Everybody knows Xserve has to be cluster-able. Apple has bought or collaborated with just about every piece of software needed to make Pixar style animated movies (and movie affects, etc...). How can it not be cluster-able (and I mean low over head cluster-able)?!

Whatever Apple needs to get done to floor this market, it will do. I'm no wizard when it comes to Movie affects, etc... But I know Steve wants to see Apple the best in this market. With his knowledge base at Pixar I'm sure he'll get it done.

Xserve will be (hopefully just one of) the main topic(s) at MWSF

What do you guys mean when you say it is not cluster-able? If I am not mistaken, you can use PBS (batch software) or distributing jobs, or write code in MPI. What else is needed to make a machine cluster-able?

[kupan787]

Originally posted by Cipher13:
You got a 33 cluster of Xserves, and you're concerned about how they perform on a Fractal test?

Gee, well, get folding@home and see how that runs on there. If it doesn't perform, take them back.

I wish I had a cluster of 33 xservers. I was just refering to an article on the macnn frontpage yesterday.

[passmaster16]

Originally posted by Evangellydonut:


Even Gbit ethernet isn't really the best thing for clustering. I know at least 2 local companies that makes clustering specific hardware built to be cheaper and faster than Gbit switched networks...('course both companies are pretty much all graduates from my university, but that's another story)

I agree though, I think in bigger clusters special hardware is typically used. Check out some of the cabling used at PSC: http://www.psc.edu/publicinfo/tcs/miles_of_cable.html I had an opportunity to tour this facility last spring, no G4s but impressive nonetheless: http://www.psc.edu/publicinfo/news/2...002-06-25.html

-Josh

[Scotttheking]

That's one horribly setup cluster.
That's why it's slow.
There is no reason why it shouldn't be performing at 70-80% of what the machines could do alone, and they should be even higher at a distributed task that doesn't need real time communication.

[BlackGriffen]

Originally posted by kupan787:


Crap, sorry I probably sounded like a troll or something, but really I am not.

Ya I made a slight mistake when saying that it was 3 GFLOPS per xserve, when I meant per proc (so ya, around 6 GFLOPS for a xserve). But I don't see how my math skills were blowing chunks by this one mistake. All my other figures were fine (the % of theoretical was valid).

My question was somewhat answered. I didn't realize there would be that much overhead for clustering. And someone else mentioned the fact they were using a 100 switch instead of a 1000 switch, and altivec fractal is not built for clustering.

My point was that I would think, even after overhead, that the machines should be getting better than 43.8% of theoretical. Like maybe 60% or something. Maybe in the future we will see better numbers?

Sorry, that insult was there because I thought you were a troll. I make math mistakes like that all the time.

Assuming processing speed scales linearly with speed (which is a good first approximation when the speeds are close and the processors are the same model), you're real world results suggest that It would be more realistic to expect a maximum of 3.8 GFLOPS per proc (or only 7.6 GFLOPS per machine). Tricky thing is that those two processors have to share the same bus, so a dual processor mac will not be able to get the same performance as two independent single processor machines. Not to mention the fact that the estimate of a single GHz processor machine's performance assumes that the processor is the only limiting factor on the performance of the machine. Depending on the code, this can be true or false. So, I'd say those XServe's did about as well as could be expected, and clustered quite well, performing at 79% of what a realistic overestimation of their performance would be. Remember, like I said, it takes processor time to coordinate the work of the XServes, which our estimate did not take in to account.

In order to say more about exactly how good this performance is, I would have to compare other dual processor clusters (since 66 XServe proc's don't have as much networking overhead because half of the processors are internally coordinated). I simply do not have enough information to do this.

BlackGriffen