how stable are the servers?

a couple of years ago, distributed.net ran into a problem on thier servers. most people were still cracking 2*28 blocks of the RC5-64 project. In 1996 these blocks were a good size. they took most people about an hour to do one, and we only uploaded every 10 blocks.

200,000 people, uploading every 10 hours.

however with the advancement of computers, i can now process a 2*28 block in about 55 seconds, and we usually upload every 25 blocks

200,000 people, uploading every 20 minutes

as you can imagine, they had to increase the size of the blocks, and BEG people not to process lots of the smallest increments. their servers are donation based, and the networks were getting clogged.



How are we doing on this front? it seems we have about 2000 people actively searching, most are probably using 5000 structures as thier upload threshold. (i use 10k, as are most on this board) if my computer was average (thunderbird 1333, seems average nowadays) that would mean an update about every 35 minutes. are the servers and networks holding up well under this load?

if you can wager a guess, how many people do you think the current network could support before we would have to do one of the following.

:) increase the maximum threshold from 10K
:) Have the clients begin only uploading structures deemed usefull
:) perform major upgrades to the network


(FYI, the main resason i'm curious to this info is because i dont want CNN to advertise our project just yet if we're not able to handle the load)

as you can see, i've posted a lot since i found this project just a short week ago. i'm very interested and want to know exactly what state of affairs it's in. i followed the human genome project all the way through high school, i love that i might be able to help it out now. i hope you dont get sick of my questions.

Sean

howard upgraded the back end a few months back, i think he has posted that it is now quite capable of handling more users than we presently have, for quite a while

i was hoping his TV appearance would attract some new blood, but i am not sure it did :/

SETI is soon ending.

The Distributed Folding project is in a very strong position to attract new members from within the distributed computing community, IMO. I am talking about people who do DC as a hobby rather than the kind of people who read about a project on a news site and wander in and run the screensaver for a few weeks.

The client has a great native service install (becoming more and more important as anyone in the corporate world with half a brain who isn't already using a version of NT moves to NT by virtue of WinXP and even home users are starting to migrate over), it is not completely no-net friendly but it is far more reasonalbe in this regard than many (most) projects, the client is very stable and essentially bug-free, there are some great 3rd party applications (dfGUI to name just one) and there are great stats (Those by Dyy, reader50 and statsman).

Having been around this project, not since the beginning, but since it was still pretty new (feb. 2002) it is GREAT to see that all of the features and requirements that more experienced or hard-core DC participants require are essentially covered by this project.

Sort of off-topic but I just popped over to this forum after spending a few more hours working on a FAQ for a different, newer project and realized how mature this project has really become, especially in contrast. :)

Quote:

---

Originally posted by MAD-ness
it is not completely no-net friendly but it is far more reasonalbe in this regard than many (most) projects,

---

the only two projects that are in the same ballpark for non-netting (IMO) are G@H and ECCP(2)

G@h is still functional, although most call it a dead project

ECCP is a "guess what number i picked" project, so i am not interested

According to:
http://www.distributedfolding.org/login.html
as of Jan 12, 2003 of the 16,432 users registered, 1842 are currently busy folding away on this particular protein structure.

The computers that are folding range from single Pentium 1s (as claimed by a few folks) to large farms (Jodie's 100 Athlon 1800+'s were a good example of this) - to the top 2 producer's ability to run the client on almost all the computers at their company/university. (several hundred systems?)
On each of the two teams I've been associated with, there's been competition to gain position in team rankings - and that usually required multiple systems. I currently have 4 systems folding for me - and I'd have to add 4 times that amount to keep up with our team's leading producer.

So while we have 1842 users, they represent at least 3-6,000 computers.
And the only problem we've had was with some teams storing all their structures for weeks and uploading them all at the last minute (which overloaded the servers).
After asking that everyone upload as frequently as possible (not saving everything up until the day before the structure is switched) and upgrading the hardware and software - the servers don't seem to be hit so hard during the recent changeovers.

This is all incredibly good to hear. i'l see what i can do to get a couple hundred more people a month to start joining. i'm not in the position to be as vocal as a few years ago, but i can still drive a little attention around.

Sean

Kileran,

The 5000 structures is the default upload amount set server side. A lot of people crank that up to the max of 10,000 structures per set. With the current protein that's around an hour on my XP2000+.

The main swamp we see these days is when we change over to the new protein. That's mainly due to all the clients out there trying to get the latest update, but it usually clears up after a couple hours or so.

TTFN,

RuneStar½

I agree with Runestar that the protein switchovers do clog the servers.

I also notice that during that period I can't access the HTTP server either; I get a message saying that it's down for maintenance or something like that. I don't know if that is planned or not.

Anyway, my suggestion would be to offer a set of alternate download sites. This would allow the main project servers to continue responding normally during the update process.

I'm sure some of the project contributors could gladly offer to distribute the updates, throughout the month or only during the update periods.

Clients could have a file containing a list of valid servers they would loop through until they get successful download. That list could be updated during the update process.

I don't know if the users need to be authenticated before downloading the updates, that could cause security concerns.

This would greatly help those that do not rely on the automated updates and have to try over and over to swith to a new protein.

François

Quote:

---

Originally posted by francoisr

Anyway, my suggestion would be to offer a set of alternate download sites. This would allow the main project servers to continue responding normally during the update process.

I'm sure some of the project contributors could gladly offer to distribute the updates, throughout the month or only during the update periods.

François

---

We actually already DO this. BeefDart (I think) at RIT has generously donated his bandwidth to serve as an auto-update mirror site. Jodie did initially too... (where is she now anyways?). Any more would be welcome. Anyone with a 5Mbit (T3??) or better connection available that is. This helps reduce the load on our network when we switch proteins and everbody's auto-update kicks in. Just e-mail us at trades@mshri.on.ca if you can help out.

And yes the 'server down for maintenance' on the web site is intentional, to prevent people registering or accessing stats while the database is being modified.

Howard,

Instead of taking down the whole web section, couldn't you simply just disable the stats section of the site?

Also, maybe you could put some kind of ETA to completion counter during these times? New protein switchover started at ##:## UTC and approximate time to completion is ## mins?

RS

k, i understand devotion and all, but cant you go without the site for 1 day out of every 3 weeks :)