[Scotttheking]

I'm making this as a new thread because we really went off topic in the gromacs thread.

We were discussing optimum farm machines.
I made a little spreadsheet with some data.
This is very limited, I can add more stuff as needed.

Code:
base $	system memory       	ram $	proc	proc $	motherboard	mobo $	total $	mhz per dollar	avg OC	MHz/dollar w/ OC
75	256MB crucial pc2100	32	1700	42	Biostar M7VIQ	56	205	8.2926829268	2200	10.731707317
75	256MB crucial pc2100	32	2100	73	Biostar M7VIQ	56	236	8.8983050847	2500	10.593220339
75	256MB crucial pc2100	32	2400	80	Biostar M7VIQ	56	243	9.8765432099	2600	10.699588477
75	256MB Kingston pc2700	33	2500	93	CT 7NIF2-SUMMIT	93	294	8.5034013605	3000	10.204081633
75	256MB Kingston pc2700	33	2600	96	CT 7NIF2-SUMMIT	93	297	8.7542087542		
75	256MB Kingston pc2700	33	2700	132	CT 7NIF2-SUMMIT	93	333	8.1081081081		
75	256MB Kingston pc2700	33	2800	180	CT 7NIF2-SUMMIT	93	381	7.3490813648

All prices were taken from newegg
CT = chaintech
Base cost is power supply, HSF, and a HD. If I missed anything, let me know.
Both mobos are mATX. The estimated overclock assumes the boards will OC, which may not be true. That needs to be explored.
I can add other mobos as needed.

One thing to note is that the chaintech mobo can take faster procs later, while the Biostar can't.

[reader50]

<table BORDER=1 CELLSPACING=0 CELLPADDING=1><tr><th>Base</th><th>Motherboard</th><th>type</th><th>$</th><th>NB</th><th>RAM</th><th>$</th><th>CPU</th><th>OC</th><th>$</th><th>Total</th><th>MHz/$</th><th>OC MHz/$</th></tr><tr><td>$75</td><td>Biostar M7VIQ</td><td>mATX</td><td>$56</td><td>?</td><td>256 PC2100</td><td>$32</td><td>K7 1700</td><td>2200</td><td>$42</td><td>$205</td><td>8.293</td><td>10.732</td></tr><tr><td>$75</td><td>Biostar M7VIQ</td><td>mATX</td><td>$56</td><td>?</td><td>256 PC2100</td><td>$32</td><td>K7 2100</td><td>2500</td><td>$73</td><td>$236</td><td>8.898</td><td>10.593</td></tr><tr><td>$75</td><td>Biostar M7VIQ</td><td>mATX</td><td>$56</td><td>?</td><td>256 PC2100</td><td>$32</td><td>K7 2400</td><td>2600</td><td>$80</td><td>$243</td><td>9.877</td><td>10.700</td></tr><tr><td>$75</td><td>Chaintech CT 7NIF2-SUMMIT</td><td>mATX</td><td>$93</td><td>?</td><td>256 PC2700</td><td>$33</td><td>K7 2500</td><td>3000</td><td>$93</td><td>$294</td><td>8.503</td><td>10.204</td></tr><tr><td>$75</td><td>Chaintech CT 7NIF2-SUMMIT</td><td>mATX</td><td>$93</td><td>?</td><td>256 PC2700</td><td>$33</td><td>K7 2600</td><td>?</td><td>$96</td><td>$297</td><td>8.752</td><td>?</td></tr><tr><td>$75</td><td>Chaintech CT 7NIF2-SUMMIT</td><td>mATX</td><td>$93</td><td>?</td><td>256 PC2700</td><td>$33</td><td>K7 2700</td><td>?</td><td>$132</td><td>$333</td><td>8.108</td><td>?</td></tr><tr><td>$75</td><td>Chaintech CT 7NIF2-SUMMIT</td><td>mATX</td><td>$93</td><td>?</td><td>256 PC2700</td><td>$33</td><td>K7 2800</td><td>?</td><td>$180</td><td>$381</td><td>7.349</td><td>?</td></tr></table>
Base = HD $25 (cheapest available from Scott) + HSF $20 + cheapo 250W Power Supply $30
NB = Net Boot compatible.
RAM = Crucial (PC 2100) or Kingston (PC 2700)
OC = Average stable overclock. Individual CPUs will vary.

[enola]

Here's some links to articles I found interesting regarding setting up farms:

Windows Folding Server

Jason Rabel's well known article

The SETI Monster

I have one PC that is a dedicated cruncher that's got win2k pro on it. I can experiment with that box safely. But I have some basic logistics to figure out like where to put this stuff. Even just one node leaves me perplexed as I look at the lack of space around my desk.

I thought I might try the windows folding server method first before trying to go completely linux. I'm not afriad of linux but feeling lazy about partitioning the PC and installing another os. I'm really keen on trying netboot but cripes, it looks complicated.

[Scotttheking]

If anyone wants to water their farm, feel free to head over to my place.
Hell has it's rare flood.
Uh, a main water pipe burst, I'm moving out until it's fixed. Lappy is not coming with me.

[jbcool]

Originally posted by enola:
Here's some links to articles I found interesting regarding setting up farms:

Windows Folding Server

Jason Rabel's well known article

The SETI Monster


I thought I might try the windows folding server method first before trying to go completely linux. I'm not afriad of linux but feeling lazy about partitioning the PC and installing another os. I'm really keen on trying netboot but cripes, it looks complicated.

Thanks for the links. Netboot appear to be about a complicated at learning to ride you bike for the first time. Remember that? I am going to try and set something up here in the near future I will be using Jason's article as a starting point using Mandrake and the move on to set up a second one using Debian thanks to Scott's help.

Scott thanks for the new thread.

[Welnic]

Originally posted by enola:
Here's some links to articles I found interesting regarding setting up farms:

Windows Folding Server

Jason Rabel's well known article

The SETI Monster

I have one PC that is a dedicated cruncher that's got win2k pro on it. I can experiment with that box safely. But I have some basic logistics to figure out like where to put this stuff. Even just one node leaves me perplexed as I look at the lack of space around my desk.

I thought I might try the windows folding server method first before trying to go completely linux. I'm not afriad of linux but feeling lazy about partitioning the PC and installing another os. I'm really keen on trying netboot but cripes, it looks complicated.

I set up my farm using Jason Rabel's article. I used Debian. I set it up for distributed folding which was a little harder because it writes about 8MBs of files into the temp directory, so I had to figure out how to be able to do that to the hard drive instead of the stock 1MB ram drive that this version of net booting usually uses.

I have some pictures of of my farm that I will try to post a link to next week as far as a possible physical setup.

[jbcool]

Originally posted by Welnic:
I set up my farm using Jason Rabel's article. I used Debian. I set it up for distributed folding which was a little harder because it writes about 8MBs of files into the temp directory, so I had to figure out how to be able to do that to the hard drive instead of the stock 1MB ram drive that this version of net booting usually uses.

I have some pictures of of my farm that I will try to post a link to next week as far as a possible physical setup.

Thanks Welnic!
Alway glad to have others share what they have learned before the rest of us get there.


In answer as to why I have decided to buy motherboards with on board video.

(1) I don't have to hunt for a motherboard that will boot without it.
(2) When I end up giving them away I don't net to buy a video card.

[Welnic]

The link to my farm pictures:

http://rasy.com/home/clint/farmpics/CubeFarm.html

[jbcool]

Originally posted by Welnic:
The link to my farm pictures:

http://rasy.com/home/clint/farmpics/CubeFarm.html

Nice neat setup. I would have thought there might have been a heat issue with them stacked as near to each other as they are. What motherboards are you using.

Thanks
jbcool

[enola]

How are the motherboards mounted? It doesn't look like they are sitting on a shelf of any sort. They look like they are connect by those posts but how are the posts mounted to the motherboards. Looks like a nice compact set up. Thanks for posting it!

[Welnic]

The motherboards are MSI MS-6390. I was going to get around to overclocking but still haven't really chased after being able to check the temps in linux. They seem to work fine but I am not doing anything special.

I used ethernet cards that came with boot ROM chips from disklessworkstations.com. In the pictures that is what the bottom two nodes use. The top two use PXE which what the builtin ethernet uses. It is a little harder to get working.

The heat seems to be fine. The fan looks kinda stupid, but it really does work.

Stand-offs are little hex shaped deals with a threaded hole in one end and a threaded rod on the other end. In most computer cases they are what the motherboard actually sits on, but they are usually only 1/2 inch high. That is what the bottom board is sitting on. Then there are two 2 inch standoffs between the next 2 boards, and then the top two have a 1 1/2 and a 2. So the boards are only supported by the standoffs, but that is the way that it normally is. Usually though there are 6 of them, but it wasn't feasible because if I used the other two holes I wouldn't be able to use a video card, which I need to do from time to time.

[jbcool]

Originally posted by Welnic:
The motherboards are MSI MS-6390. I was going to get around to overclocking but still haven't really chased after being able to check the temps in linux. They seem to work fine but I am not doing anything special.

I used ethernet cards that came with boot ROM chips from disklessworkstations.com. In the pictures that is what the bottom two nodes use. The top two use PXE which what the builtin ethernet uses. It is a little harder to get working.

Welnic I just fired up one of the motherboards that I got in and noticed that the do PXE. Any words of wisdom before I dive in and drown? Anything that might trip me up?

Thanks

[Scotttheking]

all documentation

That page has PXE info, and I also noticed it now has info on how to get the nforce mobos working.

[Welnic]

Originally posted by jbcool:
Welnic I just fired up one of the motherboards that I got in and noticed that the do PXE. Any words of wisdom before I dive in and drown? Anything that might trip me up?

Thanks

The only thing that I remember that was different was PXE required a more advanced version of TFTP than I had on there originally. I will try to find out what I ended up with. I think it ended up to not be that different, but the long nonsensical string that it needs in the dhcp.conf file looks really scary.

[jbcool]

Originally posted by Welnic:
The only thing that I remember that was different was PXE required a more advanced version of TFTP than I had on there originally. I will try to find out what I ended up with. I think it ended up to not be that different, but the long nonsensical string that it needs in the dhcp.conf file looks really scary.

I took a good look at things and it was well documented as to how to set it up so that ended up not being an issue. It looked like Jason did a good job on the write up also. I glanced at the windows setup and decided it was going to be a lot less trouble to get the linux version working.

I did have another question and I have an idea of how to do it but figure I should ask you since you have done it. You are writing to a temp file instead of the ram drive, how did you set that up?

Thanks

[jbcool]

Can you see me dancing? Dat's what I'm doin' now.

Enola: Thank you for the link to the windows farm server.
I took about 15 minutes to read the document and get it set up.

If I did things correct I am crunchin for Team Macnn. If not??????

For those of use that do not linux I suggest the windows server first.

There is a issue with the NIC that was giving me some type of problem with the linux box. I get that fixed and still try and figure that out.

[enola]

Originally posted by jbcool:
Can you see me dancing? Dat's what I'm doin' now.

Enola: Thank you for the link to the windows farm server.
I took about 15 minutes to read the document and get it set up.

Congratulations! Damn, it was that easy?? Well, now I know whom to ask questions if I ever get around to trying this. Refresh my memory. What motherboard are you using? Also, are you using the onboad lan or a separate NIC?

[Welnic]

Originally posted by jbcool:

snip...
I did have another question and I have an idea of how to do it but figure I should ask you since you have done it. You are writing to a temp file instead of the ram drive, how did you set that up?

Thanks

I saw this as I was leaving work and I thought I'll answer that when I get home. But my farm is at work and I have to look at the file that tells it to make the /tmp directory on the hard drive. I'll get to it tommorrow.

The setup that I used at first had a 1MB Ramdisk that the /tmp directory was on. I think that it is enough for FAH. I think I tried ECCP2 as a test, I know that worked okay. dFold needs 8MBs+. It is possible to make the Ramdisk bigger but you have to recompile the kernel, I think that putting the /tmp on the hard drive is easier.

[jbcool]

Originally posted by enola:
Congratulations! Damn, it was that easy?? Well, now I know whom to ask questions if I ever get around to trying this. Refresh my memory. What motherboard are you using? Also, are you using the onboad lan or a separate NIC?

I am using the BIOSTAR M7VIG Pro which has onboard lan (PXE) cost was $57. I chose it because the 266FSB so I could use all my old memory chips.

[Scotttheking]

This is from my rc.local file

Changes are in bold.

pr_set 69 "Creating Ramdisk"

echo "Mounting /tmp"
#RAMDISK_SIZE=`get_cfg RAMDISK_SIZE 1024`
#/sbin/mke2fs -q -m0 /dev/ram1 ${RAMDISK_SIZE}
#/bin/mount -n /dev/ram1 /tmp
mount -t nfs -o nolock 192.168.0.20:/netboot/tmp/${HOSTNAME} /tmp
rm -R /tmp/*
pr_set 70 "Setting Hostname"

HOSTNAME=`hostname`
echo "Current hostname: ${HOSTNAME}"

and from my exports file

/netboot/nodes/node-100 192.168.0.20/255.255.255.0(rw,no_root_squash)
/netboot/tmp/node-100 192.168.0.20/255.255.255.0(rw,no_root_squash)
/netboot/nodes/node-101 192.168.0.20/255.255.255.0(rw,no_root_squash)
/netboot/tmp/node-101 192.168.0.20/255.255.255.0(rw,no_root_squash)
/netboot/nodes/node-102 192.168.0.20/255.255.255.0(rw,no_root_squash)
/netboot/tmp/node-102 192.168.0.20/255.255.255.0(rw,no_root_squash)

Hopefully this explains it

Edit: Welnic pointed out a mistake I made.

[Welnic]

Now that I see it the stuff that Scott just posted is what you need. Above the bold added lines in the rc.local file you also have to comment out the ramdisk lines like Scott did.

[jbcool]

Thanks Scotttheking and Welnic.

Although seldom said " Thanks to all in all forums for the sharing of information to help other."

I can actually understand what is going on, another first.
I can start using it when I get enough power to run all the machines.
Got two farm crunchers up and running to have the circuit breaker trip and take the crunchers and two other machines down. That is why I want them running off a network drive and not a ram drive as the final setup.

Be a good week for the gang to pass me on the d-fold project.