[jrh87]

Hello, having just tested some code on G5, I'm considering applying for money to build a small G5 cluster.

Is there anyone who reads these forums who might be able to provide me with some pointers? There are plenty of places to go if you want to build a Beowulf cluster and I'm hoping that somewhere, someone has started doing the same for the G5s.

What would it cost and what is involved to set up, say a 24 node machine made of G5's? If I scale down the VT machine, the cost is about $2200 per processor. Could I realistically expect to be able to build a 24 processer machine for about $50k?

[Eug]

You should call Dr. V at Virginia Tech. He will be outlining the VT build.

Also, I'd consider waiting a couple of months for the G5 Xserve to come out. They'll likely have G5 cluster nodes available, and it's possible they may support ECC as well. They also come with Panther Server and include server monitoring and remote management software.

Also, are you at an academic institution? You can get about a 10% educational discount if you are. Also, to keep prices down, memory should be purchased from a 3rd party, such as from Crucial.

BTW, I'd suggest posting at the Ars Mac forum. They are more oriented to this sort of stuff than people are here. You should also check out Apple's scientific computing mailing list.

P.S. What kind of code and what relative speeds are you getting with your code? I assume you're using IBM's compilers? (I'm not an engineer or whatever, but I'm interested nonetheless.)

[Catfish_Man]

Originally posted by jrh87:
Hello, having just tested some code on G5, I'm considering applying for money to build a small G5 cluster.

Is there anyone who reads these forums who might be able to provide me with some pointers? There are plenty of places to go if you want to build a Beowulf cluster and I'm hoping that somewhere, someone has started doing the same for the G5s.

What would it cost and what is involved to set up, say a 24 node machine made of G5's? If I scale down the VT machine, the cost is about $2200 per processor. Could I realistically expect to be able to build a 24 processer machine for about $50k?

It kinda depends on what you want to do. The VT cluster is pretty heavy duty stuff (tweaking the PCI-X timings to get better infiniband performance, for example), but a simpler cluster is easy (my friend and I set up a 2 node cluster in <30 minutes, most of which was spent updating the powerbook to OS9).
try here:
http://exodus.physics.ucla.edu/appleseed/appleseed.html

[jrh87]

Originally posted by Eug:
You should call Dr. V at Virginia Tech. He will be outlining the VT build.

Also, I'd consider waiting a couple of months for the G5 Xserve to come out. They'll likely have G5 cluster nodes available, and it's possible they may support ECC as well. They also come with Panther Server and include server monitoring and remote management software.

Also, are you at an academic institution? You can get about a 10% educational discount if you are. Also, to keep prices down, memory should be purchased from a 3rd party, such as from Crucial.

BTW, I'd suggest posting at the Ars Mac forum. They are more oriented to this sort of stuff than people are here. You should also check out Apple's scientific computing mailing list.

P.S. What kind of code and what relative speeds are you getting with your code? I assume you're using IBM's compilers? (I'm not an engineer or whatever, but I'm interested nonetheless.)

Thanks for ars technica tip -- Its a site I've stumbled across before but had forgotten about.

We are doing modeling of solid earth deformation. The test I ran is a multi-grid finite element code for mantle convection - the big challenge being that viscosities can vary by 6 orders of magnitude. It's not a very rigorous test by computing world standards, but it is a real world test for how I think many scientists work. The G5 I had access to is a 1.6 GHz machine with OS 10.2.8 on it, and I compiled with gcc 3.3 and -O3. A sample run (128x128 elements and 2000 cycles) ran in 67 minutes. On an old SGI we've done most of our work on, the same run took 196 minutes (using SGI's compiler and an array of optimizations) - the chip was a 360 MHz R12000. A pentium III chip on a Beowulf cluster (1.3 GHz) ran it in 160 minutes. That was gcc 2.95 and -O3. For yuks I also tried my 800 MHz iMac at home, and it took 204 minutes. So the G5's seem quite impressive, and I presume that the same G5 running OS 10.3 would do even better than what I got from my test.

In any case, the other issue for future work is memory, and 2GBytes per node is not enough if we want to do 3D simulations. Since I no longer have access to SGI's (that was at an institute I left) and here, the best computing option is the Beowulf cluster, I'm thinking that G5 cluster would be just as cost effective as the Beowulf. And since the G5 is 64 bit, we will have fewer memory limitations.

I tried emailing the contact person listed on VT's home page, but never got an answer...

[Eug]

The G5 is 64-bit but the OS isn't (fully). The OS can access > 4GB RAM, but only supports 4 GB per process. Still, that's way better than 2 GB. Furthermore, the G5 1.8 and 2.0 have 8 memory slots, making 4 GB a cheap option (8x512). (There is no 64-bit Linux available for the G5, BTW.)

And you should also take a look at IBM's XL compilers for Mac OS X. xlc may or not be faster than gcc, but xlf rules the roost for Fortran. (xlc -O5 is way faster than gcc -O3 at SPEC, but whether that translates into real-world improvements with your code is a different story. OTOH, xlf is basically always faster, and often MUCH faster.)

The beta XL downloads are free. Don't let the beta status fool you though. They are ports of the (mature) XL compilers for the IBM POWER series chips. And I'm told that for scientific code they're rock solid. (Where they may not work is when you have written code with Apple-specific proprietary functionality.)