wierd upload problem with latest client

[pfb]

System:

Windows XP Home, SP1
P4 2.4 'B' @ 2.7
512 MB DDR
1Mbit Cable connection

running as a service with the ramuse option, monitored by dfGUI v1.8

Problem:

Some structures do not upload - removing from filelist.txt allows any buffered after that 5000 block to upload but trying to upload that 5000 set always returns a 'ERROR: [000.000] {foldtrajlite.c, line 5672} Error during upload: NO RESPONSE FROM SERVER - WILL TRY AGAIN LATER' message.

I experience the upload problem on 2 different PCs (by transferring the .bz2 and fileslist.txt files) but only by the structures produced on the system above.

All previous runs have been OK - only since the new smaller uploads have I noticed this problem - and it occurs about twice a day - meaning I lose 2 x 5000 structures from my upload process.

Solution:

Unknown :( Any help/suggestions welcome...

[Brian the Fist]

I do not understand your description of the problem. What is it that is not working exactly? You say you delete filellist.txt, which should clear the buffered files, and then what?

No Response From Server usually means just that. If the server happens to be busy for some reason, it may not reply in time. Usually trying again later is the appropriate action to take in this case.

[pfb]

I have a set of buffered structures, but the first 5000 appear to block the upload process - removing the two lines pertaining to them from filelist.txt allows the rest to upload, putting the lines back stops uploads.

I had 16 sets earlier - removing the lines


fold_hp6mn0sy_95867_protein.log.bz2
hp6mn0sy_protein_0099621.val.bz2

allowed the 2nd to 15th set to upload, and after putting that back in any further uploads have stopped. This has happened 4 times since the recent update but never before...copy the files to a second PC - which is running on a leased line rather than a cable modem connection - also experiences this same problem...I only get the not responding error when I try to upload the 5000 strucutres which appear to block an upload .

(hope that's a bit clearer - bit of a wierd problem).

if I leave it to later it still is blocked - although if it happens again I will leave it - either until it uploads or the buffer fills up...

[pfb]

it appears to have happened again - about 7 hours ago - and have 9 sets to upload. I will take your advice and leave it for sometime to see if they do upload before I remove the one that appears to be blocking the upload...

[Brian the Fist]

Now Im still not clear. Is it always the same two files that you list above, which prevent you from uploading? Or are there different ones? If its always the same one, just forget about them. If you are generating new ones that are giving you trouble, what is the EXACT error you get, either in your error.log or on your screen, when it tries to upload those files and doesn't succeed?

[pfb]

it's not the same files - I removed the 2 listed and uploads worked until earlier today when another upload appeared to get stuck - I removed those as well earlier to see if my uploads would start again, and it did. Putting them back in for upload causes the 'Server not responding' message and 'ERROR: [000.000] {foldtrajlite.c, line 5672} Error during upload: NO RESPONSE FROM SERVER - WILL TRY AGAIN LATER' in the error log.

What happens is something like:

Upload set 1 (5000 structures) gets generated but doesn't upload
Sets 2 to 10 get generated after and they are not uploaded
Removing the entry for set 1 from filelist.txt allows uploads to happen, putting set 1 back into filelist stops uploads happening (always get the server not responding message' even though 2 to 10 have uploaded)

As I mentioned it's happend a few times since the new update - and if I remove them from the filelist uploads happen, putting them back in they don't...

It is a bit of a wierd problem as I can't see anything different with the sets that don't upload against those that do - I have copies of the last 2 sets that haven't upload (they cause the 'server not responding' message') and can't upload them...

Hope it's a bit clearer...or not :p

[bwkaz]

Is there a firewall between you and the DF servers maybe, that might be filtering connections based on contents? Maybe the contents of these data sets look harmful to it, so it drops the packets?

It's very, very strange that you'd get a "server not responding" with certain sets of data all the time, unless the data itself was causing connection failures somehow...

Unless that error gets printed in other cases than just "no server response"? Not very likely, but possible I guess...

[Brian the Fist]

If that is the case, please e-mail me the suspicious files (to trades@mshri.on.ca) and I will see if I can determine what is wrong with them. Maybe I will try uploading them and see if I get the same error as you too. Then we'll know for sure if its the files, or just you ;)

[pfb]

There is a firewall (XP's own) but it is allowing them out as using AnalogX NetStat Live to monitor the flow in and out shows a 21K spike followed by 2 smaller sub 1K spikes...so it appears it is going out but either not getting there or not be accepeted for some reason...normal uploads just show the ~21k spike when uploading...

As I said earlier, it's only started with this latest update - so not sure where the problem exists or how it could be fixed...

Anyway - sent the email (titled 'Units that appear to be causing upload problems') with 5 sets on it - 4 are 5000 structures processed and one is ~ 1300...

[pfb]

I've emailed you again - but the acknowledgement problem still exists even after I tried to disable the transparent proxy...

oh well, the manual solution I have is OK (for the moment)...:)

edit - scratch that...found a transparent proxy where it seems to work (well, the ones that wouldn't acknowledge now do) - it seems I had to pick one which is slightly further than my nomral and use a different port...

could it be my uploads were going too quick and the proxy wasn't passing through the info properly? now I've gone to one slightly further it seems ok...will let you know if the problem re-occurs...

thanks for the help :cheers: