[The_Equivocator]

I have permission to run DF on a bunch of lab computers... The only stipulations are that no one can be using the computers at the time, and I can't install any scripts or anything like that on the lab machines... only DF. So, I was thinking that I could write a script on my home computer that does the following:

1) The program opens up a text file with a list of the IPs of the lab computers
2) The program would then ssh to each of the lab machines and do the following:
a) use "finger" or "who" to see if anyone else is currently logged in to this machine.
b) if there is, disconnect and try again in 15 minutes, otherwise run DF on that machine, checking every minute or so to see if anyone has logged in. If someone has logged in, disconnect and try back in 15 minutes.

Is this possible to do? I'm worried about the ssh part. How do I get a script to use my password automatically instead of prompting me for one?

Does anyone else have any ideas for me?

[reader50]

Was hoping someone else would jump in. I'm inexperienced with shell scripts.

Can you be certain the computers will be idle during certain hours, like midnight to 5AM?

I'd be inclined to cron them during latenight hours, and not risk trying to run during idle work time. No chance of a network problem causing them to keep crunching because you can't reach them to shut down -- oops, lost the farm.

Somewhat lower production is better than brief power, followed by none.

The foldit file is actually a script, but it's part of the client distribution. Put the finger check in there on startup, just in case. Cron the foldit file for ... midnight? Cron an rm command for ... 5AM? The rm removes that check file, which will cause dfold to shut down. Same as giving the quit command. I'm not remembering which file is the one to remove though.

Other opinions welcome.

[The_Equivocator]

The computers are usually turned off at 1AM and aren't rebooted until the labs open again in the morning. As for network stability, I'm not really worried about that. Everything would be done over a LAN which rarely has any problems. I can't cron anything or do anything at all to the lab computers other than have the distribfold folder in there (it's actually located in my user directory and I don't have access to any system files and the admin doesn't want me messing with anything I would need root access to do).

The biggest hurdle I'm having is getting a script to connect to the lab computers over ssh (everything else is turned off, including telnet). When I try to ssh into any of the machines, script or no script, it prompts for a password. I've been searching the internet, but I can't figure out how to just put the password into the script so that I can do everything without user input.

[reader50]

Try checking man ssh. This may be what you are looking for:

ssh implements the RSA authentication protocol automatically. The user creates his/her RSA key pair by running ssh-keygen(1). This stores the private key in $HOME/.ssh/identity and the public key in $HOME/.ssh/identity.pub in the user's home directory. The user should then copy the identity.pub to $HOME/.ssh/authorized_keys in his/her home directory on the remote machine (the authorized_keys file corresponds to the conventional $HOME/.rhosts file, and has one key per line, though the lines can be very long). After this, the user can log in without giving the password. RSA authentication is much more secure than rhosts authentication.

It appears that with some setup work, the remote machines will not ask for a password when ssh'd from your home machine. However, I'd first read the manpage very carefully to be sure no vulnurabilities will be introduced by doing this.

[Scotttheking]

Originally posted by The_Equivocator:
I can't cron anything or do anything at all to the lab computers other than have the distribfold folder in there (it's actually located in my user directory and I don't have access to any system files and the admin doesn't want me messing with anything I would need root access to do).

Is it that you can't run scripts / cron, or you can't have root access? If you already have a user account, I don't see why you can't do it using your account.

[The_Equivocator]

Originally posted by reader50:
Try checking man ssh. This may be what you are looking for:It appears that with some setup work, the remote machines will not ask for a password when ssh'd from your home machine. However, I'd first read the manpage very carefully to be sure no vulnurabilities will be introduced by doing this.

Hmm... this seemed like it would be the perfect thing, but it does not appear to work. I did the following:

1) ran "ssh-keygen -t rsa1" without giving it a passphrase.
2) copied the contents of the file $HOME/.ssh/identity.pub on my machine to $HOME/.ssh/authorized_keys on the remote machine
3) attempted to ssh to the remote machine, but it still prompted for my password.

Did I do something wrong? I'm worried that the machines on the network might not allow this type of authentication. Has anyone else ever gotten this to work?

[Welnic]

Originally posted by The_Equivocator:
Hmm... this seemed like it would be the perfect thing, but it does not appear to work. I did the following:

1) ran "ssh-keygen -t rsa1" without giving it a passphrase.
2) copied the contents of the file $HOME/.ssh/identity.pub on my machine to $HOME/.ssh/authorized_keys on the remote machine
3) attempted to ssh to the remote machine, but it still prompted for my password.

Did I do something wrong? I'm worried that the machines on the network might not allow this type of authentication. Has anyone else ever gotten this to work?

I was intrigued by this and finally got it to work with a linux box logging on to my OSX box. I had to add "-1" to the line to force it to use protocol 1, then it logged on without needing the password.

[Welnic]

But when I tried to log on to my linux box from OSX it didn't work with the -1 switch. So I set up the protocol 2 keys and that worked, and it even worked without the -2 switch. I would guess that the box that you are trying to log in on only accepts protocol 2 logins.

I actually agree with Scott that you should be able to do what you need without root access on the boxen. Anything that you can do from your machine automatically a cron job should be able to do from the machine itself.

[The_Equivocator]

Excellent! The -1 did it! -2 doesn't work, but -1 does. Now to write the script...

As for cron, I'm pretty sure it won't work for what I am trying to do. It needs to only run when no one else is using the computers. Plus, I'm pretty sure the admin doesn't want me messing around with cron on the lab computers. Logging in and doing everything manually is fine, but I shouldn't be messing the the computers themselves in any way other than running foldit.

[mikkyo]

Are you going to run command line DF?
If so it uses a script to launch itself.
So you WILL have a script on every machine.
If I were you I would put a script in the DF folder that checks to see if anyone is logged on, and if not runs DF.
If someone logs on you terminate DF.
You can have cron call your script in the DF folder every 5 mins to see if anyone has logged on and then fire off the commands you need.
Pretty hard to just manage it all remotely, when it is so easy to make it manage itself.
Sure you could ssh to like 10 machines and do things manually, but using cron on the local machine will work even when you arent there.

[The_Equivocator]

Originally posted by mikkyo:
Are you going to run command line DF?
If so it uses a script to launch itself.
So you WILL have a script on every machine.
If I were you I would put a script in the DF folder that checks to see if anyone is logged on, and if not runs DF.
If someone logs on you terminate DF.
You can have cron call your script in the DF folder every 5 mins to see if anyone has logged on and then fire off the commands you need.
Pretty hard to just manage it all remotely, when it is so easy to make it manage itself.
Sure you could ssh to like 10 machines and do things manually, but using cron on the local machine will work even when you arent there.

I'm confused. How would I get cron to do this on a machine on which I only have user privs? It seems like a security hazard to let anyone mess with cron so that anything can be launched at anytime on any machine. Am I misunderstanding something?

[The_Equivocator]

Here's an update on what I have done thus far...

I just finished writing a multi-threaded C++ program that:

1) Spawns a thread for each computer address I have listed in a text file.
2) Each thread then forks into two processes. The child process attempts to ssh to its associated machine and run a script I have in my home folder while the parent process waits until the child process is killed.
3) When the ssh process is terminated, the parent wakes up and waits for 5 minutes, and then goes to step 2 above.

I've tested it and it works. Now I just need a shell script that does the following:

1) Checks to see if anyone is connected by calling "who" and counting the number of lines it returns. More than one is bad.
2) If no one is connected, start foldit in the background.
3) Call "who" every minute or so to see if anyone has connected. When someone does, rm the lock file and disconnect.

So, being much more familiar with C++ than shell script, I am running into some problems. Can anyone show me how to do number 1 above? I've been Googling around shell script pages but they haven't been helping.

Any help/insights on things I could be doing better would be great. Thanks!

[reader50]

Originally posted by The_Equivocator:
I'm confused. How would I get cron to do this on a machine on which I only have user privs? It seems like a security hazard to let anyone mess with cron so that anything can be launched at anytime on any machine. Am I misunderstanding something?

Every user has their own crontab. You can edit your own, but not someone else's unless you have an admin account.

When cron launches something, it is as if you launched it. Only apps you have the privileges for can be launched by you.

[The_Equivocator]

Originally posted by reader50:
Every user has their own crontab. You can edit your own, but not someone else's unless you have an admin account.

When cron launches something, it is as if you launched it. Only apps you have the privileges for can be launched by you.

Ah, that makes sense. I'll look into doing everything with cron since it seems much easier... Ah, well, shouldn't have spent all that time writing the C++ program.

So, while I look into that, anyone have any help they can give me on the shell script?

[Welnic]

This is a script that I wrote that seems to work. I tested this on OSX as my linux boxes are at work. It is set up so that I could be logged in without turning folding off. You need to change clint to whatever your login is. I don't really understand how to get the variables treated as numbers instead of strings, but it did finally work.

#!/bin/sh

whoThere=1 #this seems to make whoThere numeric
whoThere=`who | grep -vc clint` #any one else logged in outputs a number

if [ $whoThere != 0 ] #the != was a way to get it treated as a number
then

cd ~/distribfold #~ stands for home directory
if [ -f foldtrajlite.lock ] #check for the lock file
then
rm foldtrajlite.lock #remove if there
fi

elif [ -f ~/distribfold/foldtrajlite.lock ] #check to see if we are running
then

cd ~/distribfold #it seems that there had to be something here

else

cd ~/distribfold
./foldit & #start it up!

fi

After you save the script you have to change it so that it is executable:

chmod 755 UserCheck.sh

Then run crontab -e (which means edit your crontab) and insert this line:

0-59/5 * * * * cd ~/bin; ./UserCheck.sh

This assumes that you put the script in the bin directory in your home directory, just as the script assumes that the distribfold directory is in your home directory. The /5 means that it will be checked every 5 minutes, change for other times or leave it off for 1 minute intervals.

Good luck

[The_Equivocator]

Beautiful. Thanks, Welnic. Now time to do some final editing on this. Apparently, cron as a whole is disabled for users on these systems, so, I'll have to use my ssh program. I'll also have to change the script so that it runs forever in a while loop and checks every minute by itself. I should be able to handle that. Hopefully, this will all work.

Thanks again.

[Scotttheking]

Originally posted by The_Equivocator:
Apparently, cron as a whole is disabled for users on these systems, so, I'll have to use my ssh program.

Why not write your own version?

Code:
#!/bin/sh
while true
        do
        cd /home/distribfold;./foldit &
        sleep 60
        done

Just change the script line to point to that checking script.
Run this once, it'll keep running fine.
It's been running for a month of two on my netboot box, it hasn't failed yet.

[Scotttheking]

Any news on this?
Did you get it working?

Inquiring minds want to know

[Welnic]

Originally posted by Scotttheking:
Any news on this?
Did you get it working?

Inquiring minds want to know

Yeah, what he said.

[Toast]

Hmmm, I wonder how much it would need to change to get it to work with Folding@Home, that is, if it's working at all

[The_Equivocator]

I got it working beautifully. I now have a multi-threaded C++ program that SSHs to each remote machine, and only runs dfold when no one else is logged into the computer.

But, then a problem reared its ugly head. I went into the computer lab and told the sysadmin that I had everything working, but that I needed to know which computer I could run the C++ program from. The labs are set up so that there is only one computer you can ssh to from outside of the labs. You can only connect to the other computers from within the lab network.

So, he said to check with this one professor I know fairly well. He has a machine for research purposes that I could probably run it on. So I go talk to him, and he doesn't like the idea of dfold. He has so many worries about running this thing on the network that he basically convinces the sysadmin that this is a bad idea.

Unfortunately, the case is pretty much closed. They are supposedly going to bring it up at the next Computer Science department meeting, but my hopes are anything but high.

There goes my dreams of having close to 20 P4s and 20 P3s running dFold in their spare time.

[The_Equivocator]

Originally posted by Toast:
Hmmm, I wonder how much it would need to change to get it to work with Folding@Home, that is, if it's working at all

It would require a change in maybe 2 or 3 lines of code. Basically change all "cd distribfold"s and "./foldit"s to their Folding@Home counterparts.

[krove]

Couldn't you double SSH (if that's even a correct term)?

My home network uses NAT and an iMac serves as the software router. The way it is setup, I can only ssh from the outside world to it. But once I've connected via ssh to the iMac, I can ssh again to any internal machine on the network. Double ssh! That should work given that only one machine is accessible from the outside world. As long as that machine then has access to all the other machines, you shouldn't encounter any problems.

[The_Equivocator]

Originally posted by krove:
Couldn't you double SSH (if that's even a correct term)?

That's exactly what I do, but SSHing to the different machines is not really the problem. The problem is now that my college faculty are too selfish. They don't want anything running on their network that *might* cause them to do extra work. Even though the program I wrote is stable and would require no maintenance from the faculty, they are worried about what *might* happen if it crashes one of their machines.

That is half of their problem with it. The other half involves how the department should decide which distributed program should get to be the one chosen for college use. Why dFold and not another one that a different student supports?

I think the selfishness is what will win out in the end, though. There are no faculty members backing me, and I am not allowed into department meetings to give presentations or the like. They see this as possible extra work for them with no immediate benefit, even though dFold is a project that could theoretically benefit science and humanity as a whole, which is exactly what an educational institution should be concerning themselves with.

[mikkyo]

Your solution is far easier.
Forget SSH.
Just set up the scripts to self manage.
You can have the script run a check every 5 minutes to see if anyone logged on and shutdown. Then it continues checking until everyone is off. You never have to touch the things.
I would do it with 2 sh scripts.
One does the check and shutdown, then spawns the other to do the check for logoff and startup.
Or you could do everything with cron.
Have a cronjob run every 5 minutes and launch your checker/starter/killer script.
If no one on, start, if someone on stop.
It really shouldn't be too hard.

As to the selfishness issues..
Have it self manage and tell them to let you know if there is ever a problem and if so you will turn everything off (just check your scripts good first). As to why dfold and other students whims, well You made the scripts and set it all up first. You get the initiative award (your program runs :-).

[mikkyo]

Oh BTW, if you set the nice to 20, chances are even with someone logged on they will never know anything is running unless they check with ps or top (it won't interfere with their work).

[krove]

If the script is run well enough, the chances of them actually knowing that you are running any dist. computing client is remote. Just run it on maybe one machine for a while and see if anyone notices...



[/back to the real world where there are consequences]

So, this sucks... Does your campus have other computing labs that are under the control of another department? There must be a science lab, chemistry, biology, etc that would be very interested to see their computers utilized to help such endevors as distributed folding and F@H. Check it out...

[The_Equivocator]

Originally posted by mikkyo:
Your solution is far easier.

Have a cronjob run every 5 minutes and launch your checker/starter/killer script.
If no one on, start, if someone on stop.
It really shouldn't be too hard.

I tried that already, but apparently user-level cron is disabled on the lab machines. The only way to run something is to ssh into the machine and run it that way.

Plus, this gives me more power from a centralized computer, which is nice. I could even set up a cool GUI and everything that tells me the status of each computer with the ability to start and stop other computers on a whim. Think of the possibilities!

As to the selfishness issues..
Have it self manage and tell them to let you know if there is ever a problem and if so you will turn everything off (just check your scripts good first).

I told them that already. Reason does not seem to apply in these situations. I can't seem to convince them that any possible problem would be easily remedied.

[The_Equivocator]

Originally posted by krove:
So, this sucks... Does your campus have other computing labs that are under the control of another department? There must be a science lab, chemistry, biology, etc that would be very interested to see their computers utilized to help such endevors as distributed folding and F@H. Check it out...

A good idea, but I go to a fairly small college and the only computer labs worth anything are all run by the same system administrator. If I can't get this guy to see things my way, there's no hope.

[Scotttheking]

Why not write up a little description and info about why it should be done, and give that to the tech and some other professors.
Even though you aren't in the room, your info will be there.

[The_Equivocator]

Originally posted by Scotttheking:
Why not write up a little description and info about why it should be done, and give that to the tech and some other professors.
Even though you aren't in the room, your info will be there.

I have a meeting today with one of the CS profs I have not yet talked to. As far a the CS meetings go, there are only 4 people who attend. 3 profs and the sys admin. I've talked to one prof and the sysadmin so far. Hopefully I'll get better results from the last two profs.

Wish me luck.

[The_Equivocator]

Update:

So I met with the other CS professor, and he likes the whole idea! Actually, he even wants me to type a 1 page write-up about the whole thing which he will present at the next department meeting.

Phase 1: Accomplished

Phase 2 will commence at the next department meeting.

[Scotttheking]

Originally posted by The_Equivocator:
Update:

So I met with the other CS professor, and he likes the whole idea! Actually, he even wants me to type a 1 page write-up about the whole thing which he will present at the next department meeting.

Phase 1: Accomplished

Phase 2 will commence at the next department meeting.

If you need any help with the write-up, I've got one posted somewhere around here, and it probably has some useful stuff in it.

[The_Equivocator]

Originally posted by Scotttheking:
If you need any help with the write-up, I've got one posted somewhere around here, and it probably has some useful stuff in it.

Sure, that sounds great. If you find it, post a link!

Thanks!

[The_Equivocator]

For those interested, here is a copy of the one-page proposal letter I wrote (all specific names have been crossed out). The proposal will be presented by my advisor at tomorrow's department meeting. Cross your fingers!



Distributed Computing Proposal for the Computer Science Labs

The Proposal
On average, most of the computers in the Computer Science labs sit idle. While these computers are on for up to sixteen hours each day, each computer is being used for only a small fraction of that time. I propose that we take advantage of these unused hours to do something worthwhile for the scientific community through running a distributed computing project.

The Project
There is a project called Distributed Folding, which is run by the Hogue Bioinformatics Research Laboratory, the Samuel Lunenfeld Research Institute, the Mt. Sinai Hospital, and the University of Toronto Dept. of Biochemistry. The goal of the project, as stated on their website (www.distributedfolding.org), is to test and improve a relatively new protein folding algorithm that attempts to produce natural protein folds. By accomplishing this, the project hopes to learn how to predict the possible stable three-dimensional structures of any given protein. Protein structures are currently solved through rigorously slow and expensive laboratory techniques. By finding a more mathematically and computationally-driven manner of determining the stable structures of different proteins, researchers will have much more information available to them when trying to create new effective therapeutics to treat diseases such as Alzheimer's, Cystic fibrosis, and Mad Cow, all of which are related to protein defects.

Why Distributed Folding?
While Distributed Folding is definitely not the only distributed computing project dedicated to scientific research purposes, I feel that it is currently the most promising. After investigating the other possibilities, I believe that Distributed Folding is the project that is most likely to bring about important and interesting results that will be beneficial to numerous people. While it may be important to consider alternative projects as a department, half of this proposal is dedicated to the idea that we, as an institution with resources, should be running a program on our idle lab machines to further the scientific community.

Reasons For Participation
The way I see it, an institution such as XXXXXX has two main purposes: to educate its students, and to promote research. In this case, XXXXXX would be donating unused processor time to a program whose goals could very possibly benefit researchers everywhere. By participating, the Math/CS Department at XXXXX would not only be helping a worthwhile scientific project, but the department would also bring recognition to our institution as one that supports non-profit research.

Implementation
Taking many concerns into consideration, I have written a program that would completely control and automate the use of Distributed Folding, as well as any other distributed computing application we might take into account. The program will ensure the following:

The educational purposes of the labs will not be jeopardized: The program ensures that processor time is only taken from computers that are not in use. The minute that someone starts to use a computer, whether it be by physically logging into the computer or by doing so remotely, the Distributed Folding application will automatically be shut down, allowing the computer to be completely used for the educational purposes of Carleton students.

The use of this program will be secure: All network traffic is done using SSH, the most secure means of network communication.

A very small amount of disk space will be used: Each morning, when the program is launched, the application and all of the data it needs is copied to the network file server. Then, each night, as the labs are shut down, all traces of the distributed application are deleted so as not to take up space on the nightly backup tapes.




Attached to this proposal, you will find the source code for the program I have written. Thank you for taking the time to go over this, and please direct all questions and concerns to [email protected] where they will be addressed as soon as possible.