Results 1 to 16 of 16

Thread: Upload bandwidth

  1. #1
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619

    Upload bandwidth

    Howard, I have/had over 2000 wus I found on a computer that couldn't get out on the network.
    I am unable to upload a darn thing.

    In my efforts to zip up and move them to another server to try from there, I noticed the .zip file was over 45 megs, so I aborted that idea of sending it to another server.
    Then I realized, after the fact, they had the old zip on their computer, which truncated all the filenames to 6chars~1
    I may have lost all that work. Still working on it tho...

    Is anything being done to help with upload bandwidth?
    We are killing your Servers with these fast proteins for sure.
    What are you going to do if membership here doubles?

  2. #2
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    Yup, as I suspected, more work down the drain.
    Everything is an 'illegal file' or something, after restoring all the files from the trash can ...
    won't even take the -purgeuploadlist 1 command either.

    Guess that will teach me to watch the clients more closely.

    I'm seeing ALOT of timeouts to the server on all the clients to.

  3. #3
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    Originally posted by IronBits
    I'm seeing ALOT of timeouts to the server on all the clients to.
    I also. I've written about this before, to not much avail. Seems like I spend way too much time going around to my various (borged and otherwise) boxen an removing humongous error.log files.

    Has Canada set a tariff on uploads?

    Howzabout... setting a "minimum time between uploads" in the client?

    Maybe even making it user-selectable?

    Or some other bright idea? But hurry! I'm running out of these damn lightbulbs!

  4. #4
    Administrator PCZ's Avatar
    Join Date
    Jun 2003
    Location
    Chertsey Surrey UK
    Posts
    2,428
    IB
    Thats about a million points down the drain, it would annoy me to

    I think that it would be better if the client only uploaded results at the end of each 250 generations.
    This would be similar to how F@H works, a fast machine might only upload once or twice a day instead of 100's of times.

  5. #5
    Senior Member
    Join Date
    Jul 2002
    Location
    Kodiak, Alaska
    Posts
    432
    Originally posted by IronBits
    Yup, as I suspected, more work down the drain.
    Everything is an 'illegal file' or something, after restoring all the files from the trash can ...
    won't even take the -purgeuploadlist 1 command either.

    Guess that will teach me to watch the clients more closely.

    I'm seeing ALOT of timeouts to the server on all the clients to.
    If you have a corrupted generation (or missing generation) - and you use -purgeuploadlist until you get up to gen 245, and then use -purgeuploadlist 1 until it starts uploading - it should upload most of those 2k generations for you. (I seemed to always be 1 off when I was trying to use the -purgeuploadlist command - and had to use 30 instead of the 31 gens I was missing..)
    www.thegenomecollective.com
    Borging.. it's not just an addiction. It's...

  6. #6
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    Originally posted by PCZ
    I think that it would be better if the client only uploaded results at the end of each 250 generations.
    This would be similar to how F@H works, a fast machine might only upload once or twice a day instead of 100's of times.
    That would work. The client could still save after every generation, but the upload could be severly compacted.

  7. #7
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    I got it sorted out...
    Had to zip it up and ftp the 40meg file to my server and drag it over here on this box where everything works fine.
    1 of 1589
    2 of 1589
    3 of 1589
    ...
    289 of 1589
    ...

    Had already completed the 1 of 2,300+ to get down to 1589 this morning, before it crapped out.

    All is well, but, we still have the upload timeout and bandwidth issues to deal with, or rather, Howard does

  8. #8
    Again, if you find a suspected bug or problem, please provide a full description of the problem and any relevant messages from the error log. Our servers are not overloaded to the best of my knowledge so if you are receiving lots of timeouts, I do not know why...
    Howard Feldman

  9. #9
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    ========================[ Aug 31, 2003 3:41 PM ]========================
    ERROR: [777.000] {ncbi_socket.c, line 1258} [SOCK::s_Connect] Failed pending connect to www.distributedfolding.org:80 (Unknown) {errno=No such file or directory}
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to www.distributedfolding.org:80 failed: Unknown
    ERROR: [777.000] {ncbi_socket.c, line 1258} [SOCK::s_Connect] Failed pending connect to www.distributedfolding.org:80 (Unknown) {errno=No such file or directory}
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to www.distributedfolding.org:80 failed: Unknown
    ERROR: [777.000] {ncbi_socket.c, line 1258} [SOCK::s_Connect] Failed pending connect to www.distributedfolding.org:80 (Unknown) {errno=No such file or directory}
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to www.distributedfolding.org:80 failed: Unknown
    ERROR: [777.000] {ncbi_http_connector.c, line 101} [HTTP] Too many failed attempts, giving up
    ERROR: [777.000] {ncbi_socket.c, line 1173} [SOCK::s_Connect] Failed SOCK_gethostbyname(bioinfo.mshri.on.ca)
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to bioinfo.mshri.on.ca:80 failed: Unknown
    ERROR: [777.000] {ncbi_socket.c, line 1173} [SOCK::s_Connect] Failed SOCK_gethostbyname(bioinfo.mshri.on.ca)
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to bioinfo.mshri.on.ca:80 failed: Unknown
    ERROR: [777.000] {ncbi_socket.c, line 1173} [SOCK::s_Connect] Failed SOCK_gethostbyname(bioinfo.mshri.on.ca)
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to bioinfo.mshri.on.ca:80 failed: Unknown
    ERROR: [777.000] {ncbi_http_connector.c, line 101} [HTTP] Too many failed attempts, giving up
    ERROR: [000.000] {foldtrajlite2.c, line 2101} Tue Sep 02 15:27:03 2003
    Unable to check server status
    ERROR: [777.000] {ncbi_socket.c, line 1258} [SOCK::s_Connect] Failed pending connect to www.distributedfolding.org:80 (Unknown) {errno=No such file or directory}
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to www.distributedfolding.org:80 failed: Unknown
    ERROR: [777.000] {ncbi_socket.c, line 1258} [SOCK::s_Connect] Failed pending connect to www.distributedfolding.org:80 (Unknown) {errno=No such file or directory}
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to www.distributedfolding.org:80 failed: Unknown
    ERROR: [777.000] {ncbi_socket.c, line 1258} [SOCK::s_Connect] Failed pending connect to www.distributedfolding.org:80 (Unknown) {errno=No such file or directory}
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to www.distributedfolding.org:80 failed: Unknown
    ERROR: [777.000] {ncbi_http_connector.c, line 101} [HTTP] Too many failed attempts, giving up
    ERROR: [777.000] {ncbi_socket.c, line 1258} [SOCK::s_Connect] Failed pending connect to www.distributedfolding.org:80 (Unknown) {errno=No such file or directory}
    ERROR: [777.000] {ncbi_connutil.c, line 801} [URL_Connect] Socket connect to www.distributedfolding.org:80 failed: Unknown
    ERROR: [777.000] {ncbi_socket.c, line 1258} [SOCK::s_Connect] Failed pending connect to www.distributedfolding.org:80 (Unknown) {errno=No such file or directory}

    This is just a small sample from a randomly-chosen machine. I could pick up the same from any of my 15 or so boxen, some at work, some at home, etc...

    You don't see the problem because you're on the other side of the counter! Do you run the client from home?

    We can't say, "Hang on, Howard, and we'll make it happen again." It's completely random, and the above is the result. That error.log is over 16K bytes and that's just from the last couple of days! It gets a bit embarassing, telling the people who have agreed to let me run the client, "Excuse me, I need to get in and delete the error log, again."

  10. #10
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    What he said - times 40+ computers, regardless of OS or client used.

  11. #11
    I do not know what causes that error, i.e. why it fails to connect, assuming your connection is all functional of course. Network problems are always difficult to debug. I'll mention it to the IT and see if they have any ideas.
    Howard Feldman

  12. #12
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    See whatcha get, but I really think it's just collisions in brief periods of super-high activity.

    (Like when FoBoT dumps, for instance.)

    BTW, if these errors were time-stamped, maybe we could see a pattern...

  13. #13
    Ol' retired IT geezer
    Join Date
    Feb 2003
    Location
    Scarborough
    Posts
    92

    Angry Update from Dialup VERY SLOW

    Did you guys forget to build an index for your database again???

    Updating from a dialup location is very very slow. Normally I can get 20 generations in 3 minutes or so, this morning its more than 30 seconds per gen .....

    HELP !!!!

  14. #14
    Member
    Join Date
    Jul 2002
    Location
    Down the road from Mr. Fist :D
    Posts
    76
    Originally posted by Paratima
    I also. I've written about this before, to not much avail. Seems like I spend way too much time going around to my various (borged and otherwise) boxen an removing humongous error.log files.

    Has Canada set a tariff on uploads?

    Howzabout... setting a "minimum time between uploads" in the client?

    Maybe even making it user-selectable?

    Or some other bright idea? But hurry! I'm running out of these damn lightbulbs!

    Easy there boy! You don't cut back on that electricity useage with those lightbulbs, you'll cause another blackout here!

  15. #15
    Member
    Join Date
    Jul 2002
    Location
    Down the road from Mr. Fist :D
    Posts
    76
    Paratima,

    I've got that same error in my logs just filling the sucker up. I had clients exiting all the time during the last protein. It got to the point where I should have been doing a million a day, but with the clients exiting I could only muster about 800k after a couple days of it running ...

  16. #16
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    Yup! Hard to tell if it's something on the DF end, something on my end, or something in between. With time-stamps, we MIGHT be able to correlate it... I dunno. I've got a lot of boxen on -if & I dump them at night.

    Considering the current high fuel costs, I have tried to limit the use of . We're switching over to power!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •