PDA

View Full Version : Too much Data!!!!



gpdnet
03-28-2002, 03:49 PM
This looks like a worthwile project. But....

I can compute 5000 structures in about 30-40 minutes on each of my 10 machines at work. We have a firewall internet connection with a dial on demand ISDN line. This means that the ISDN line is dialling every 3-4 minutes to upload data to the server!!! Phone bills through the roof.

If I buffer the data I accumulate 60 MBytes PER DAY of data on the system. If is is left over the weekend I get about 150 MB of data to upload to the server on a Monday morning. NO WAY...

I would really like to put all that computing power to good use. I
have tried other projects (apart from SETI) but many have problems with firewall connections or other problems.

I don't think anyone has done the sums on how much data these
clients have to upload, even in a day. My machines are around 1.4GHz Athlon machines which is fairly run of the mill these days.

Any suggestions before I have to give up???

FoBoT
03-28-2002, 05:39 PM
each project has its own characteristics, unfortunetly, this one has large return data size

for someone on a metered bandwidth plan, that really hurts :(

you would probably be better off going to one of the math projects that have very small return files (only tiny amounts of data to send back to the project)

gpdnet
03-28-2002, 06:02 PM
Thought that might be the case.

Oh well, back to finding another worthwhile cause for all this spare CPU fluff.

MAD-ness
03-28-2002, 10:01 PM
This client can now run 'no-net' pretty well, but you would still have to find a way to 'upload' the results. What I mean to say is that if you know someone with unlimited internet access, you could run the client no-net, move the results files to a computer on your internal network with a CD burner and burn them onto a CD from which you could upload them via another computer with a "fat pipe."

If you are having to use metered isdn, odds are decent that no one in your area has kickass net access either. :(

gl either way.

[Ars]KD5MDK
04-02-2002, 03:02 AM
If you crunch for a team that runs multiple projects, you could offer to trade with someone who has an equivilent amount of power who has unmetered bandwidth. This is usually done when one architecture is much better at a particular project than another. Cases are the P4 in GIMPS, the G4 in RC5, and the K6 in d.net OGR. In this case, Distributed Folding seems to like Athlons best, and most projects do. So there's not much to trade on that front. Still, you might find someone with a equal sized farm (or less, and do a partial) who is running a different project but wouldn't mind switching.

For the ultimate in low bandwidth projects, try GIMPS. But most of the other math ones are good in this respect I understand, including RC5 and ECCp. Check with teams members you might have on those projects.

(If you don't have team members, there are lots of teams looking for new members. I'm with Ars Technica, we have a lot of fun in the DC Game. Look around for others.)

KWSN_Millennium2001Guy
04-10-2002, 12:28 PM
I finally got around to reading my proxy logs and discovered that I am averaging 500 megabytes of results being uploaded to the servers each day. This equates to almost 10 million structures a day. So you can assume that if you are averaging 100 K structures a day with a reasonably fast system, you should be uploading about 5 megabytes a day per system. Not an unreasonable amount, but enough that you should be aware of the bandwidth requirements.

Ni!