Welcome to the MacNN Forums.

MacosNerd · Jan 16, 2008, 08:35 AM

Those annoying and pesky server busy errors seemingly returned.
Yesterday I thought it was because of MWSF but I checked to see who's online and there wasn't a ton of people. Plus I got this error last night and this morning.

Oisín · Jan 16, 2008, 08:58 AM

I just got it too. MWSF seems to be having slow, but detrimental effects.

seanc · Jan 16, 2008, 01:55 PM

I got a server error and lack of images Monday, but it's been fine apart from dying during MWSF yesterday.

imitchellg5 · Jan 16, 2008, 09:04 PM

I haven't had any today. I had a ton yesterday, but I expected that.

turtle777 · Jan 20, 2008, 12:17 AM

Had a couple 2 minutes ago

-t

brassplayersrock² · Jan 20, 2008, 12:24 AM

same here. i think it was when glenn posted from the future that screwed things up.

may i present peoples evidence #1

peoples evidence two:
http://forums.macnn.com/92/networkin...wifi-security/

rickey939 · Jan 20, 2008, 01:14 AM

Burp.

besson3c · Jan 20, 2008, 03:30 AM

The "server is busy" errors are application level warnings that are generated when the traffic hits a certain threshold, I'm pretty certain. However, it is not a given that there is a relationship between MWSF and these messages. The server could very well be attacked, and I wouldn't be surprised if it were.

I pointed out several months ago the exposure of this file:

http://forums.macnn.com/phpinfo.php

This has obviously not been dealt with. With an older version of PHP being run and this being openly advertised and this file easily found (among other vulnerabilities which are easily detectable), it is probably only a matter of time before an attack is successful.

reader50 · Jan 20, 2008, 04:01 AM

You guys are missing the most important question.

"The Server is too busy" doing what?

Since it happens regularly during the day, it might be coffee breaks. But that's far from the only possibility.

seanc · Jan 20, 2008, 08:18 AM

Originally Posted by reader50

You guys are missing the most important question.

"The Server is too busy" doing what?

Since it happens regularly during the day, it might be coffee breaks. But that's far from the only possibility.

Exactly. Not having many users online at the time makes it even more odd.

brassplayersrock² · Jan 20, 2008, 05:00 PM

just got one that lasted a couple of minutes

brassplayersrock² · 05:23 PM

and another

edit:
do the forums seem a bit snappier now?

imitchellg5 · Jan 20, 2008, 07:59 PM

Originally Posted by brassplayersrock²

and another

edit:
do the forums seem a bit snappier now?

No they are really slow for me.

C.A.T.S. CEO · Jan 29, 2008, 01:06 PM

They're back, saw one at noon-ish.

MacosNerd · Jan 30, 2008, 09:25 AM

Oh man, getting a lot of them right now

xi_hyperon · Jan 30, 2008, 03:28 PM

Yup. Still an issue.

Fab · Feb 1, 2008, 10:00 AM

Got that message a few times yesterday

Dakar the Fourth · Feb 1, 2008, 10:02 AM

It's been going on intermittently for two weeks for me.

alligator · Feb 1, 2008, 11:16 PM

These errors are getting really annoying. If this keeps up, you'll lose forum members. ADD MORE BANDWIDTH NOW!

besson3c · Feb 1, 2008, 11:19 PM

I don't think it's a bandwidth issue, but yeah, it needs to be addressed.

If you guys are really in a bind and don't have an admin to look at this, PM me, perhaps we can work something out.

rickey939 · Feb 2, 2008, 12:59 AM

Just got it. I will post here every single time I see it. Literally. Every single time. Starting with this one.

turtle777 · Feb 2, 2008, 01:02 AM

Ok, so would someone from staff please tell us what's going on ?

-t

Dork. · Feb 2, 2008, 01:10 AM

I'm getting a server busy error right now, which is preventing me from posting.

turtle777 · Feb 2, 2008, 03:58 AM

WTF ?

Server busy at 3:00 am ?

Even Europe is barely up.

-t

imitchellg5 · Feb 3, 2008, 03:42 PM

I got about 8 just trying to post my U23D thread.

iMOTOR · Feb 3, 2008, 08:21 PM

I just got like the seventh one in one day.

besson3c · Feb 3, 2008, 10:08 PM

Guys, anybody can get these at any time, these are server related problems, not browser related problems.

Just wanted to make sure that this was understood if you feel that your server busy error "experience" is unique and feel compelled to post here based solely on this premise.

Tiresias · Feb 4, 2008, 01:18 PM

I got about four today. Sheesh. Get a new freaking server already.

C.A.T.S. CEO · Feb 4, 2008, 01:33 PM

Originally Posted by Tiresias

I got about four today. Sheesh. Get a new freaking server already.

Ditto.

besson3c · Feb 4, 2008, 02:16 PM

The problem is not necessarily with the server either, it could simply be a vBulletin specific issue, or vBulletin rate limit configuration issue. The errors are not being generated by the server itself, but by the vBulletin application.

I suspect that some spammers are probably pushing vBulletin to its rate limits with some automated scripts. If it were me, I'd write a script to extrapolate the number of connection attempts coming from particular IPs, send email notifications to the admin(s), and work on firewall blocking the worst of these offenders and analyzing patterns. It is likely that some of these attacks are coming from non-English speaking countries.

In fact, you can pretty much do the analysis with a single Unix command that combines awk, uniq, and sort.

turtle777 · Feb 5, 2008, 10:25 AM

Whatever it is, could some mod shed some light on it, or at least tell us they are looking into it ?

-t

mkbhatia · Feb 6, 2008, 11:03 AM

Hey guys,

I tracked down some of the issues:

1) A RSS feed from a site that went down last night caused an outage on the servers. The poorly-coded PHP fragment caused the servers to wait for the non-responding site. All sites on the server were affected. This was down for about 40 minutes before our monitors caught it. I have fixed the fragment (taken it down) and also fixed the monitoring to be much pro-active (10 min).

2) the server busy errors have been due to a hacked script that allowed spammers to send tens of thousands emails via our servers. The undeliverable emails were taking up server resources and increasing load exponentially. The emails have been removed and the script disabled (until appropriate checks are put into place).

3) As a result of the troubleshooting, there was a configuration error on the main server causing the sites not to be available for about 60 min this morning. I have fixed that and hopefully all is well.

Thanks for your patience as I try to get our admins to be better about managing this stuff and be more proactive about avoiding this type of issue in the future. Please post feedback here.

Dakar the Fourth · Feb 6, 2008, 11:07 AM

Thanks for keeping us posted.

besson3c · Feb 6, 2008, 11:16 AM

mkbhatia: Thanks man! Looks like you've been kept busy, glad to see the problem is under control...

Just wondering, have you thought about having your MTA on the server refuse to queue messages not addressed from [email protected]?

I don't mean to be another cook in the kitchen or imply that you would need assistance or something, I just happen to be a mail admin which is why I immediately thought of this

SSharon · Feb 6, 2008, 01:13 PM

Thanks for the update, I'm glad to see that the hamster's trip to the vet went well and that he is back running our servers at full speed.

Oisín · Feb 6, 2008, 03:08 PM

Well, after reading this thread earlier today (about three hours ago), the server just suddenly completely stopped responding here. Couldn’t get on at all. This is the first time I’ve been able to load any pages at all since. So either something odd happened on my end, or Monish’ efforts either didn’t fix an original bug or introduced a new one

Dakar the Fourth · Feb 6, 2008, 03:08 PM

I know, I went to shock here at work. I don't drink coffee, but apparently I need my NN.