Update Proxy

[PinHead]

What is the trick to getting the proxy to work?

I always get the following error in the logs:

========================[ Jan 28, 2003 6:48 PM ]========================
ERROR: [000.000] {foldtrajlite.c, line 1497} Unable to retrieve update file list

The updates have been downloaded and there is a file sitting in the directory called filelength.txt .


What is the name of the file that the error message refers to: "update file list".

This is the 3rd protein change that I have tried using the update proxy. I am 0 for 2 and soon to be 0 for 3.

Anyone have this working?

[Kileran]

hmm, i have a similar question.

if a friend has successfully updated, or you wanted to manually update from 1 machine, which files should you copy? could you just copy the entire installed directory, or are there certain files that should be unique to each installation?

[Scoofy12]

as long as you don't have a foldtrajlite.lock file (when the client isn't actually running) you can copy the installation. However, if you want to use a different handle than your friend, you want to keep the handle.txt file. Also you might want to keep autoupdate.cfg and foldit.bat if you want to keep your settings.

[PinHead]

After much of this and then a little of this

I have come to the realization that the client never sends the http request for the update file.

Doesn't seem to matter if its a linux or windows client!
Doesn't seem to matter if its a linux or windows web server!

Doesn't seem to matter if foldtraj_fileserver.cfg contains:
http://www.myserver.com/dfp/ or
http://www.myserver.com/dfp or
http://192.168.1.17/dfp/ or -------------- Ignore the links
www.myserver.com/dfp/ or --------------- too tired to lookup the code tags
www.myserver.com/dfp or
192.168.1.17/dfp/ or
//hostname/dfp/ or
//hostname/dfp


Bottom line - the web server logs never indicate that a request was received. Go to the same client computer open a web browser and type in the address. Boom! The files are there and the log indicates the request.

[Darkness Productions]

I came to the same conclusion some time ago, and posted it all to this thread...

http://www.free-dc.org/forum/showthr...&threadid=2129

Howard never could figure it out.

[PinHead]

The Apache error logs yields a little more info than the IIS log.

----
request failed: erroneous characters after protocol string: GET dfp/ /filelength.txt HTTP/1.0
----

So it looks like the Apache server receives the request and errors out. The IIS server probably receives the request and ignores it because of the extra characters.

dfp is the directory where the update files are placed by foldtrajupdateproxy.

[Brian the Fist]

Hi, I'd like to fix this if its not working. The error you indicated means the clinet cannot get filelength.txt from YOUR proxy server. That 'erroneous characters after protocol string' message sounds very suspicious though - any clue what the erroneous chars. are?? Can you run a port sniffer at the same time to see the exact request foldtrajlite is sending to your proxy server? That might clarify what the problem with the request is, if it is malformed for you for some reason.

[PinHead]

When a Windows client attempts an update to an IIS server:

The initial GET packet seems to be received by the server ok.
I get an ACK packet from 206.248.62.8
I get an ACK packet from my server
Then there is a continuation of the initial get. Here is the packet:

Frame 27 (85 on wire, 85 captured)
Arrival Time: Jan 29, 2003 12:04:36.689332000
Time delta from previous packet: 0.000089000 seconds
Time relative to first packet: 8.864732000 seconds
Frame Number: 27
Packet Length: 85 bytes
Capture Length: 85 bytes
Ethernet II
Destination: 00:c0:f0:20:c9:48 (ntserver40)
Source: 00:40:33:9f:b1:c0 (charlese)
Type: IP (0x0800)
Internet Protocol, Src Addr: charlese (192.168.1.30), Dst Addr: www.classifieds.int (192.168.1.197)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 71
Identification: 0x5e2b
Flags: 0x04
.1.. = Don't fragment: Set
..0. = More fragments: Not set
Fragment offset: 0
Time to live: 128
Protocol: TCP (0x06)
Header checksum: 0x1852 (correct)
Source: charlese (192.168.1.30)
Destination: www.classifieds.int (192.168.1.197)
Transmission Control Protocol, Src Port: 2585 (2585), Dst Port: http (80), Seq: 1995565818, Ack: 155745, Len: 31
Source port: 2585 (2585)
Destination port: http (80)
Sequence number: 1995565818
Next sequence number: 1995565849
Acknowledgement number: 155745
Header length: 20 bytes
Flags: 0x0018 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 32767
Checksum: 0x072b (correct)
Hypertext Transfer Protocol
Data (31 bytes)

0000 00 c0 f0 20 c9 48 00 40 33 9f b1 c0 08 00 45 00 ... .H.@3.....E.
0010 00 47 5e 2b 40 00 80 06 18 52 c0 a8 01 1e c0 a8 .G^+@....R......
0020 01 c5 0a 19 00 50 76 f1 ea fa 00 02 60 61 50 18 .....Pv.....`aP.
0030 7f ff 07 2b 00 00 64 66 70 2f 66 69 6c 65 6c 65 ...+..dfp/filele
0040 6e 67 74 68 2e 74 78 74 20 48 54 54 50 2f 31 2e ngth.txt HTTP/1.
0050 30 0d 0a 0d 0a 0....

------------------------

The packet behind it is from the server and is a 400 error Bad request. "The parameter r is incorrect" is part of the message.

I will try this with the apache server also.

[PinHead]

Here is the packet sent by a windows client to an apache server:

--------------------------------------------------------------------------------
Frame 21 (85 on wire, 85 captured)
Arrival Time: Jan 29, 2003 12:47:18.033222000
Time delta from previous packet: 0.000031000 seconds
Time relative to first packet: 5.147389000 seconds
Frame Number: 21
Packet Length: 85 bytes
Capture Length: 85 bytes
Ethernet II
Destination: 00:00:b4:74:02:72 (RAIC01)
Source: 00:40:33:9f:b1:c0 (charlese)
Type: IP (0x0800)
Internet Protocol, Src Addr: charlese (192.168.1.30), Dst Addr: RAIC01 (192.168.1.210)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 71
Identification: 0x60c1
Flags: 0x04
.1.. = Don't fragment: Set
..0. = More fragments: Not set
Fragment offset: 0
Time to live: 128
Protocol: TCP (0x06)
Header checksum: 0x15af (correct)
Source: charlese (192.168.1.30)
Destination: RAIC01 (192.168.1.210)
Transmission Control Protocol, Src Port: 2611 (2611), Dst Port: http (80), Seq: 2634909555, Ack: 512395426, Len: 31
Source port: 2611 (2611)
Destination port: http (80)
Sequence number: 2634909555
Next sequence number: 2634909586
Acknowledgement number: 512395426
Header length: 20 bytes
Flags: 0x0018 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 32767
Checksum: 0xfda5 (correct)
Hypertext Transfer Protocol
Data (31 bytes)

0000 00 00 b4 74 02 72 00 40 33 9f b1 c0 08 00 45 00 ...t.r.@3.....E.
0010 00 47 60 c1 40 00 80 06 15 af c0 a8 01 1e c0 a8 .G`.@...........
0020 01 d2 0a 33 00 50 9d 0d 87 73 1e 8a 88 a2 50 18 ...3.P...s....P.
0030 7f ff fd a5 00 00 64 66 70 2f 66 69 6c 65 6c 65 ......dfp/filele
0040 6e 67 74 68 2e 74 78 74 20 48 54 54 50 2f 31 2e ngth.txt HTTP/1.
0050 30 0d 0a 0d 0a 0....
--------------------------------------------------------

After the server sends an ACK, the following error packet is sent. Haven't seen this error message, it must get sent to the client and never displayed.

---------------------------------------------------------
Frame 23 (629 on wire, 629 captured)
Arrival Time: Jan 29, 2003 12:47:18.034551000
Time delta from previous packet: 0.001035000 seconds
Time relative to first packet: 5.148718000 seconds
Frame Number: 23
Packet Length: 629 bytes
Capture Length: 629 bytes
Ethernet II
Destination: 00:40:33:9f:b1:c0 (charlese)
Source: 00:00:b4:74:02:72 (RAIC01)
Type: IP (0x0800)
Internet Protocol, Src Addr: RAIC01 (192.168.1.210), Dst Addr: charlese (192.168.1.30)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 615
Identification: 0xb0ab
Flags: 0x04
.1.. = Don't fragment: Set
..0. = More fragments: Not set
Fragment offset: 0
Time to live: 64
Protocol: TCP (0x06)
Header checksum: 0x03a5 (correct)
Source: RAIC01 (192.168.1.210)
Destination: charlese (192.168.1.30)
Transmission Control Protocol, Src Port: http (80), Dst Port: 2611 (2611), Seq: 512395426, Ack: 2634909586, Len: 575
Source port: http (80)
Destination port: 2611 (2611)
Sequence number: 512395426
Next sequence number: 512396001
Acknowledgement number: 2634909586
Header length: 20 bytes
Flags: 0x0018 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 5840
Checksum: 0x2ea4 (correct)
Hypertext Transfer Protocol
HTTP/1.1 400 Bad Request\r\n
Date: Wed, 29 Jan 2003 17:47:26 GMT\r\n
Server: Apache-AdvancedExtranetServer/1.3.23 (Mandrake Linux/4.2mdk) PHP/4.1.2\r\n
Connection: close\r\n
Content-Type: text/html; charset=iso-8859-1\r\n
\r\n
Data (366 bytes)

0000 00 40 33 9f b1 c0 00 00 b4 74 02 72 08 00 45 00 .@3......t.r..E.
0010 02 67 b0 ab 40 00 40 06 03 a5 c0 a8 01 d2 c0 a8 .g..@.@.........
0020 01 1e 00 50 0a 33 1e 8a 88 a2 9d 0d 87 92 50 18 ...P.3........P.
0030 16 d0 2e a4 00 00 48 54 54 50 2f 31 2e 31 20 34 ......HTTP/1.1 4
0040 30 30 20 42 61 64 20 52 65 71 75 65 73 74 0d 0a 00 Bad Request..
0050 44 61 74 65 3a 20 57 65 64 2c 20 32 39 20 4a 61 Date: Wed, 29 Ja
0060 6e 20 32 30 30 33 20 31 37 3a 34 37 3a 32 36 20 n 2003 17:47:26
0070 47 4d 54 0d 0a 53 65 72 76 65 72 3a 20 41 70 61 GMT..Server: Apa
0080 63 68 65 2d 41 64 76 61 6e 63 65 64 45 78 74 72 che-AdvancedExtr
0090 61 6e 65 74 53 65 72 76 65 72 2f 31 2e 33 2e 32 anetServer/1.3.2
00a0 33 20 28 4d 61 6e 64 72 61 6b 65 20 4c 69 6e 75 3 (Mandrake Linu
00b0 78 2f 34 2e 32 6d 64 6b 29 20 50 48 50 2f 34 2e x/4.2mdk) PHP/4.
00c0 31 2e 32 0d 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 1.2..Connection:
00d0 20 63 6c 6f 73 65 0d 0a 43 6f 6e 74 65 6e 74 2d close..Content-
00e0 54 79 70 65 3a 20 74 65 78 74 2f 68 74 6d 6c 3b Type: text/html;
00f0 20 63 68 61 72 73 65 74 3d 69 73 6f 2d 38 38 35 charset=iso-885
0100 39 2d 31 0d 0a 0d 0a 3c 21 44 4f 43 54 59 50 45 9-1....<!DOCTYPE
0110 20 48 54 4d 4c 20 50 55 42 4c 49 43 20 22 2d 2f HTML PUBLIC "-/
0120 2f 49 45 54 46 2f 2f 44 54 44 20 48 54 4d 4c 20 /IETF//DTD HTML
0130 32 2e 30 2f 2f 45 4e 22 3e 0a 3c 48 54 4d 4c 3e 2.0//EN">.<HTML>
0140 3c 48 45 41 44 3e 0a 3c 54 49 54 4c 45 3e 34 30 <HEAD>.<TITLE>40
0150 30 20 42 61 64 20 52 65 71 75 65 73 74 3c 2f 54 0 Bad Request</T
0160 49 54 4c 45 3e 0a 3c 2f 48 45 41 44 3e 3c 42 4f ITLE>.</HEAD><BO
0170 44 59 3e 0a 3c 48 31 3e 42 61 64 20 52 65 71 75 DY>.<H1>Bad Requ
0180 65 73 74 3c 2f 48 31 3e 0a 59 6f 75 72 20 62 72 est</H1>.Your br
0190 6f 77 73 65 72 20 73 65 6e 74 20 61 20 72 65 71 owser sent a req
01a0 75 65 73 74 20 74 68 61 74 20 74 68 69 73 20 73 uest that this s
01b0 65 72 76 65 72 20 63 6f 75 6c 64 20 6e 6f 74 20 erver could not
01c0 75 6e 64 65 72 73 74 61 6e 64 2e 3c 50 3e 0a 49 understand.<P>.I
01d0 6e 76 61 6c 69 64 20 55 52 49 20 69 6e 20 72 65 nvalid URI in re
01e0 71 75 65 73 74 20 47 45 54 20 64 66 70 2f 66 69 quest GET dfp/fi
01f0 6c 65 6c 65 6e 67 74 68 2e 74 78 74 20 48 54 54 lelength.txt HTT
0200 50 2f 31 2e 30 3c 50 3e 0a 3c 48 52 3e 0a 3c 41 P/1.0<P>.<HR>.<A
0210 44 44 52 45 53 53 3e 41 70 61 63 68 65 2d 41 64 DDRESS>Apache-Ad
0220 76 61 6e 63 65 64 45 78 74 72 61 6e 65 74 53 65 vancedExtranetSe
0230 72 76 65 72 2f 31 2e 33 2e 32 33 20 53 65 72 76 rver/1.3.23 Serv
0240 65 72 20 61 74 20 31 39 32 2e 31 36 38 2e 31 2e er at 192.168.1.
0250 32 31 30 20 50 6f 72 74 20 38 30 3c 2f 41 44 44 210 Port 80</ADD
0260 52 45 53 53 3e 0a 3c 2f 42 4f 44 59 3e 3c 2f 48 RESS>.</BODY></H
0270 54 4d 4c 3e 0a TML>.

-------------------------------------------------------------------

Hope some of this helps!

[PinHead]

This isn't quite how I understood the updateproxy to work, but here is how I was able to get it to work on NT4 IIS4.

website (intranet) = www.classified.int
location = C:\InetPub\wwwroot\Classifieds
foldtraj_updateproxy dir = C:\InetPub\wwwroot\Classifieds\dfp

Now setup an new ftp site with the same ip as www.classified.int
ftp Home Dir = C:\InetPub\wwwroot\Classifieds\dfp

be sure and start the site

Set the foldtraj_fileserver.cfg to:
www.classified.int
or even
http://www.classified.int

Now the updates work!

I'm am using a copy of an old client, so it downloads 3 updates one right after another until it can't find a newer file.

Now to try something similar on the Apache server!

I guess I made an error in assuming that the get request was an http get request, it appears to be an ftp get request. Doh!!

[Brian the Fist]

Hmm, no that can't be right. It does not make use of the FTP protocol anywhere. Is there anything else you changed? It sound like you got it working by something you changed.. if you can figure out what let me know and maybe I can make the docs more clear.

[PinHead]

Surprised me that the setup worked! I was beginning to believe that there was a problem with name resolution. The clients were using the hosts file.

To make things more complicated, I just found 2 Win2K servers on the same switch as the Apache Server and they updated just fine.

Anyway, here is a packet that was captured during an internet update. The only thing I notice is that the internet packect has a leading / , where the previous posted packets do not have the leading / .

..........................

Frame 23 (113 on wire, 113 captured)
Arrival Time: Jan 29, 2003 14:51:11.994761000
Time delta from previous packet: 0.000050000 seconds
Time relative to first packet: 0.392282000 seconds
Frame Number: 23
Packet Length: 113 bytes
Capture Length: 113 bytes
Ethernet II
Destination: 00:06:25:5e:80:67 (The_5e:80:67)
Source: 00:40:33:9f:b1:c0 (ADDTRON_9f:b1:c0)
Type: IP (0x0800)
Internet Protocol, Src Addr: charlese (192.168.1.30), Dst Addr: ftp.mshri.on.ca (192.197.250.74)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00)
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 99
Identification: 0x6f82
Flags: 0x04
.1.. = Don't fragment: Set
..0. = More fragments: Not set
Fragment offset: 0
Time to live: 128
Protocol: TCP (0x06)
Header checksum: 0x0e3c (correct)
Source: charlese (192.168.1.30)
Destination: ftp.mshri.on.ca (192.197.250.74)
Transmission Control Protocol, Src Port: 2801 (2801), Dst Port: http (80), Seq: 200441277, Ack: 776224088, Len: 59
Source port: 2801 (2801)
Destination port: http (80)
Sequence number: 200441277
Next sequence number: 200441336
Acknowledgement number: 776224088
Header length: 20 bytes
Flags: 0x0018 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgment: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 32767
Checksum: 0x930d (correct)
Hypertext Transfer Protocol
Data (59 bytes)

0000 00 06 25 5e 80 67 00 40 33 9f b1 c0 08 00 45 00 ..%^.g.@3.....E.
0010 00 63 6f 82 40 00 80 06 0e 3c c0 a8 01 1e c0 c5 .co.@....<......
0020 fa 4a 0a f1 00 50 0b f2 7d bd 2e 44 3d 58 50 18 .J...P..}..D=XP.
0030 7f ff 93 0d 00 00 2f 70 75 62 2f 64 69 73 74 72 ....../pub/distr
0040 69 62 66 6f 6c 64 2f 64 6f 77 6e 6c 6f 61 64 2f ibfold/download/
0050 70 61 74 63 68 2f 66 69 6c 65 6c 65 6e 67 74 68 patch/filelength
0060 2e 74 78 74 20 48 54 54 50 2f 31 2e 30 0d 0a 0d .txt HTTP/1.0...
0070 0a .
....................................................

Does any of this make sense to you?

[PinHead]

Remember the 2 Win2K machines that I said did do the update ok.
Well I mirrored their network settings on the boxes that would not update, but that didn't work.

Remember the "/" I was questioning?

Well, if you change foldtraj_fileserver.cfg
from:
http://www.classified.int/dfp/

to:
http://www.classified.int//dfp/

the update works against an nt server.

Was about to test it on the linux server, distributedfolding.org is unreachable now.

I am not sure where the compatability issue is coming from. It could be a browser version, a winsock version or one of those wonderful ms security patches.

Does anyone out there use the foldtraj_updateproxy??

[Brian the Fist]

I hope someone uses it besides you Ok, you might be onto something there. Are you suggeting that, at least with some web servers,

GET dfp/filelength.txt

is incorrect and

GET /dfp/filelength.txt

is the proper form of the request?
That appears to be true, yes?
Because I'm using the former for the request I believe. I'll fix that and send you an update to try it with if you want (just write me at trades@mshri.on.ca)

[PinHead]

Here are the results of the testing:

The / in the address server/directory is getting eaten somehow.

If the foldtraj_fileserver.cfg reads:
http://server/directory

It is unable to retrieve the filelength.txt file.

If the foldtraj_fileserver.cfg reads
http://server//directory

everything works

Verified these settings on
Windows clients to Windows web server
Windows clients to Apache web server
Linux clients to Windows web server
Linux clients to Apache web server

Verified all of the above on the 129 protein df client
and the 108 df client

Maybe Darkness Productions could give this a try and verify the results on a different network.

[Darkness Productions]

If I had something to update to, I probably could. I'll email Howard and talk with him.

[PinHead]

Hopefully you will get the same results.

If I can help with the old client, just let me know.

Just sent Howard an email so hopefully we can get this working for both of us.

[Brian the Fist]

The 'eating' of the first '/' was intentional - just call me dumb but I didn't know you needed to have a leading / in the HTTP request. Anyways fixed packages will be posted as soon as I get all the version re-compiled, for those who make use of this feature. It will also be incorporated into future auto-updated versions of course.

[PinHead]

Who knew?

Now those pesky little "/" have moved up a notch behind those
annoying ";" and just above those untraceable "}"

Anywho!

Thanks for your hard work and dedication Howard


hmmm!
Now how can I trick that program into working without a web server?
hmmmm!

[bwkaz]

Originally posted by PinHead
Now how can I trick that program into working without a web server?
hmmmm!

Not too hard, all you'd have to do is bind() to port 80 and listen() for a connection, then read() the request (which will look like "GET /path/to/file"), then open that file, and write() it to the socket (probably with a "Content-type: text/html" header though).

Not too hard, at least not in a Unix-like system. I have no idea how to bind to a port with Winsock...

[Paratima]

There is a Winsock "bind" function. Much like *nix, actually.

[Darkness Productions]

Howard -

Did you ever get a chance to fix this problem?

[Brian the Fist]

Hmm good question since that was 5 months ago. I think so. Try it with the new daemon (posted yesterday) and let me know what happens on the next update - if its still not fixed Ill make sure to put it on the list and get it this time...

[Darkness Productions]

I just tried it a bit ago, and it seems I still need the double //.


Thanks again for all your hard work!

[Scotttheking]

I hope it works.
I had the same problems as DP did a while back, and it would be nice to have it working again.
I wonder if running the clients thru my squid proxy would yield the same result...?

[Brian the Fist]

the phase Two client has the correct code in it, so the extraneous '/' should not be needed. Please try it with the next update, and let us know if it works for you or doesn't (and if it doesn't, let us know the HTTP request the client appears to make if you can)