Mirrors

[runestar]

I was just wondering if Howard had given any thoughts to any official mirrors for the clients?

Second, to everyone in general why not utilize P2P technology such as Fileshare or Kazaa for that purposes? Theoritically as more people download it should be a lot easier to spread the load out, assuming they stay on after downloading it and make sure it is shared...

~RuneStar½

[pointwood]

BitTorrent maybe?

[djp]

BitTorrent Definitely! :idea:

According to the website, there are now 19874 registered users out there and many will have more than one computer to update. I've got 130-something in my farm, but I plan to set-up the Auto-update Daemon mentioned on the download page. Others may have folding farms connected directly to the Internet and set to auto-update. When the update is ready, there will be a surge of 8-or-so MB download requests competing with the regular flood of pending Phase I structures and a growing trickle of Phase II structures. Thankfully, the results uploads are small files.

With such a storm of update requests, BitTorrent would be a great help in reducing the bandwidth consumption by folks manually requesting a full download of the new client. It would also reduce the bandwidth consumed by new downloads in the long run.

I wonder if the DF client could be modified easily to take advantage of BitTorrent for upgrades like mere protein changes?

[Brian the Fist]

For one thing we have much more bandwidth now than at the last update (we changed ISPs a while back dont forget, when our IP changed).

However, we have asked for download mirrors before. We only have one mirror at the moment. We welcome any other volunteers to act as mirrors, if they have lots of bandwidth available. It is trivial to set up, just using the Daemon we supply.

[pointwood]

Howard, I don't think you've quite understood the idea - to provide a normal mirror, you would need quite a lot of bandwidth - with BitTorrent (BT), you get everyone to that uses BT to download the client, to mirror it too. BitTorrent is different from the other P2P networks as you'll understand if you read more about it: http://bitconjurer.org/BitTorrent/

Also there is a Slashdot interview with the developer behind it: http://interviews.slashdot.org/article.pl?sid=03/06/02/1216202&mode=thread&tid=126&tid=185&tid=95

And in regards to security which I'm pretty sure you're quite interested in, here is a bit from the FAQ:
How do I know the download isn't corrupted?

BitTorrent does cryptographic hashing (SHA1) of all data. When you see "Download Succeeded" you can be sure that BitTorrent has already verified the integrity of the data. The integrity and authenticity of a BitTorrent download is as good as the original request to the tracker. Checking the MD5 of a file downloaded via BitTorrent is redundant. EDIT: Oh :Pokes: any news in regards to when the change to phase 2 will happen? ;)

[Matthew.M]

I have roughly 9Gb of free bandwidth a month. Would that be enough for a mirror? If so you can consider me a volunteer.

[pointwood]

Well, I have more than 9GB a month, but my upload rate is 512kb, which I believe, is not quite enough to be very useful as a mirror...

[runestar]

Originally posted by pointwood
Well, I have more than 9GB a month, but my upload rate is 512kb, which I believe, is not quite enough to be very useful as a mirror...

Well the idea is to SPREAD out the load. =)

[bwkaz]

Originally posted by pointwood
Well, I have more than 9GB a month, but my upload rate is 512kb, which I believe, is not quite enough to be very useful as a mirror... Yeah, same here. I've got no total upload amount limit per month, but the bandwidth is capped at about 64 kilobits/sec, which is pretty crappy.

But it might just be enough for something like BitTorrent. Maybe -- and if I want to bother setting it up, that is. I'd need to install python on that box, then probably some other crap too, and whether it'd be worth it to poke another hole in my firewall is questionable at best...

[IronBits]

I think when Howard says "5 Mbit", he is referring to the size of the pipe bandwidth, not total monthly allocations ;)
I have unlimited allocation, however, I don't believe I would make a good mirror because the size of my upload, your download, pipe is only 31 KB/sec, 1 at a time...
If 100 folks hit me all at once, my house would get sucked inside out, and they would be there for a week trying to get the files. :jester:

[Brian the Fist]

Yes I meant 5Mbit pipe. Basically you won't be able to mirror from your house, it'll need to be a 'real' net connection at an institution or company. As for this BitTorrent thingy, it sounds like a neat idea, however, my 'daemon' software already accomplishes this task. All a 'mirror' is is someone running my daemon with it set to grab all OSes and which I have entered in the master list of mirrors (on our web server).

[Digital Parasite]

Originally posted by Brian the Fist
As for this BitTorrent thingy, it sounds like a neat idea, however, my 'daemon' software already accomplishes this task. All a 'mirror' is is someone running my daemon with it set to grab all OSes and which I have entered in the master list of mirrors (on our web server).

The reason you need mirrors is to increase the total bandwidth available so everyone can download the files at a reasonable pace. Your deamon software only works if people sign up to be mirrors and run your daemon software. They need to have large pipes to do this.

BitTorrent spreads out the downloads among everyone who is downloading the software. So if you have 500 people hitting the server trying to download the latest client, BitTorrent will use those 500 users to start sending parts of the client amongst each other.

It is an easy way to transform the users who are downloading your program to all become mirrors simultaneously. The cryptographic hashing ensures that nobody's files get tampered with on the way.

idSoftware uses BitTorrent to distributed the free Wolfenstein - Enemy Territory FPS game since it is a 250MB+ download and thousands of people try to DL it at the same time. See here: http://zerowing.idsoftware.com/BT.html

Jeff.

[Brian the Fist]

Like I said above, we have significantly more bandwidth now (or so Im told) so lets just see what happens this time...