Welcome to the MacNN Forums.

Scotttheking · Jun 13, 2003, 07:04 PM

Originally posted by reader50:
Must be some new email 'bots around.

waltham-host.scansoft.com was the perpetrator
over 8K hits by noon.
Interestingly, I'm currently dealing with someone from scansoft at work, I may ask him if he can put me in touch with whoever runs that box

Oh, they are now in the firewall too

OneMacGuy · Jun 14, 2003, 07:16 PM

Originally posted by Scotttheking:
waltham-host.scansoft.com was the perpetrator
over 8K hits by noon.
Interestingly, I'm currently dealing with someone from scansoft at work, I may ask him if he can put me in touch with whoever runs that box
Oh, they are now in the firewall too

Any progress on getting it fixed? The stats page for Seti says that it is 1.4 days out of synch and has not updated at all since then as dar as I can tell.

Shaktai · Jun 14, 2003, 07:38 PM

Originally posted by OneMacGuy:
Any progress on getting it fixed? The stats page for Seti says that it is 1.4 days out of synch and has not updated at all since then as dar as I can tell.

Folding@home has also not updated in over 21 hours.

reader50 · Jun 14, 2003, 11:06 PM

Working on it.

mikkyo · Jun 15, 2003, 01:28 PM

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 52 bytes) in /distcomp/webserver/teamstats/includes/page_base.php on line 200

That machine needs more ram!
That was from trying to access Current Members of F@H.

reader50 · Jun 15, 2003, 02:53 PM

It does need more RAM.

However, the specific problem was more typical. I didn't give it a full checkout after the boxes got swamped a few days ago by the email spider. And didn't check closely enough last night.

Data blocks are processed, then replaced in the database. It takes too long to modify rows in the DB on a per-row basis, so when a block is finished processing in memory, I delete the original DB block and insert the complete processed block. Normally this works very well.

Under certain conditions (swamping) is it possible to insert a block before the previous version has been deleted. This is rare even when the boxes get swamped, but when it does happen, it can run away. Our Fold blocks usually contain around 340 rows because that is how many members we have. A couple of them had 5000+ rows due to duplicate entries. We are unable to handle a 5000+ member team at this time, and the webserver doesn't want to serve up a team that large either.

The repair scripts can and will catch such an error, and will delete the affected block. If this problem happens on a smallish team, the stats will recover themselves and no one will know it even happened, unless I sifted through the logfiles. Unfortunately, when it happens on a largish team, it can run the box into the ground before the repair scripts can get to it.

I've cleaned such a block out on Folding, and those stats are coming back. Seti may have a mush block somewhere also, I'll check that next. Box3 does the Fold/Seti processing, and that is where the problem happened, the error you posted from box1 was the site scripts trying to display an illegal and too-large block. Allowing PHP to allocate more that 64MB for a single script would allow the site scripts to display such a team, but that would possibly get box1 into trouble if a lot of people loaded that team at the same time (this is more likely to happen on larger teams - ouch). Leaving the memory limit where it will display any legal data has the useful side effect of visibly flagging any team that has gotten into trouble.

reader50 · Jun 15, 2003, 04:04 PM

Fold is fixed up.

Our Seti team had acquired an extra 1K members. Nice.

The clones have been removed, and our Seti team has been thrown back to before the error, to reprocess. I'll speed it up some, get it back to normal sooner rather than later.

Shaktai · Jun 15, 2003, 06:43 PM

Great job as always. Thanks for all the hard work.

Scotttheking · Jun 15, 2003, 08:49 PM

looks like I missed a party this weekend.

Scotttheking · Jun 16, 2003, 03:09 AM

Originally posted by mikkyo:
That machine needs more ram

I'm currently playing a game called "How much money can Scott save up", which precludes new hardware and software unless really necessary. I don't consider the servers as really necessary.

Scotttheking · 06:54 PM

Power outage at home. It's a long one, I hope no one hit a power pole. I'll post when it comes back.

Edit: Power is back (I can ping the router) but the server isn't. It usually takes 15-20 mins to fsck the drive, assuming it came back up on it's own. If not, I'll be home in 1.5 hours.

Edit2: Looks like 2 out of 5 came back.
Stats won't update, and the setiQ is down, but the main server is up.

OneMacGuy · Jun 18, 2003, 11:36 AM

Originally posted by Scotttheking:
Power outage at home. It's a long one, I hope no one hit a power pole. I'll post when it comes back.

Edit: Power is back (I can ping the router) but the server isn't. It usually takes 15-20 mins to fsck the drive, assuming it came back up on it's own. If not, I'll be home in 1.5 hours.

Edit2: Looks like 2 out of 5 came back.
Stats won't update, and the setiQ is down, but the main server is up.

As of 11:30 EST on Wed, the TEAMSTATS Seti server is 8.4 hours out of synch again. Did you decide to update less frequently?

reader50 · Jun 18, 2003, 01:06 PM

Apache had died on box3. Restarted.

Scotttheking · Jun 18, 2003, 01:54 PM

Originally posted by reader50:
Apache had died on box3. Restarted.

DAMNIT
I knew I forgot something last night.
I turned apache off to benchmark DF, and forgot to turn it back on.
Sorry

OneMacGuy · Jun 19, 2003, 06:29 AM

Originally posted by Scotttheking:
DAMNIT
I knew I forgot something last night.
I turned apache off to benchmark DF, and forgot to turn it back on.
Sorry

Sorry to keep bothering you guys but the DFold stats have not updated in about a day and a half or so.

Scotttheking · Jun 19, 2003, 10:05 AM

Originally posted by OneMacGuy:
Sorry to keep bothering you guys but the DFold stats have not updated in about a day and a half or so.

We just don't have a parser for phase two yet.