No replies to this topic

[rapidgen]

Well we've been having users using sometimes around 30 split parts per download, and seeing as we stream files directly using php, that many processes grinds apache and the server to a halt. As a result, we've had to install a connection limiter to the server.

This way, a lot of 503 Service Temporarily Unavailable errors might occur to you but it doesn't mean you're banned or anything, it's just the connection limiter at work. The way it works is, every user is checked to see how many connections they have to the server at any given time. The server is set to block any connections that go above a preset amount. If a user exceeds this amount, they will receive a 503 error at that point. The 503 error will continue until the user either disconnects or one of the connections is closed, or times out.

This is effective for download managers, but can wreak havoc on users trying to download the 'normal' way, i.e. directly in their browser. Downloading via browser normally only allows one connection anyway, so if you already have more than the allowed amount of simultaneous connections to the site, you'll get a 503 when you try to download. The only way to get over this is to check you aren't connected any other way to the site, e.g. loads of open browser tabs.

Say for 7 max connections: (hope this explains it!) - when you are in the gen homepage, that is one connection active already plus 6 free; if you load into IDM 2 downloads, that's it (all connections will be used up) because IDM tries to split each download into nearly 30 parts per link, so your first link would start downloading with a max of 6 split parts, and the second link would give 503 until the first link cleared up some parts or finished downloading!

In a download manager, you can 'queue' your download, so that the download manager will appear to 'keep trying' every so many seconds (usually 5) for the download to start. The 503 error returned in this case will not fail the download, it will stay alive and keep trying until it gets access. In this case it would be preferable to change the 'retry' value in your download manager to say 15 seconds so as not to hammer the server with requests.

The preset values for limits at the moment are:

1024000kbps (1Gbps) Limit Overall (as we have a fast vps cloud)
20 rps limit (requests per second)
2500 connections overall

For remote clients (each user), the limit is set at 7 (no more than 7 connections per indivdual user).

So as you can see there are several factors which could result in a 503 error. For example, if any user decided to visit the site and gets a 503, then the overall limit of connections has been reached, so try again

If you have a suggestion for what a suitable connection limit, or other values might be then feel free to suggest. The values I quote above are subject to change without warning...as I'm still testing what the current limits are, and I will change based on them.

Thanks!

This post has been edited by szal: 27 February 2010 - 08:48