Upload problems with *nix's

[TheOtherPhil]

I, and a few other members here seem to be having problems with the DF client not sending results to the server. Every other DC client that I run has no problems sending and receiving work. Upon closer examination, it seems that I have a problem with my ISP's transparent proxy cache that works on port 80...basically, everything that uses port 80 *must* go through this transparent cache. Is there any chance Howard, of allowing the client to use an alternative port to connect to the servers and allowing the user to select the alternative port in case of difficulties? For example, leave port 10000 open for connections on your server. To use this alternate port, the user just modifies a text file called "port". The file contains the following variable as default: Port80="1". Just change the one to a zero to use port 10000.

This doesn't just affect one distro as I have the same problem on Red Hat (7, 7.1, 7.2, 8), Mandrake (8, 8.1, 9), Gentoo, SUSE8, FreeBSD etc. All other web functions work fine....just not the DF client. At the moment, I am having to ftp into my *nix boxen and copy the work to my Windows machines for upload...this becomes very tedious and time consuming.

Thanks,
Phil.

[Brian the Fist]

This is a known problem although not something I can fix. I believe the easiest workaround is to tell the client to use a different proxy (see readme on proxy.cfg). It will then use the proxy you specify which I believe will bypass the transparent proxy. (Because as you say, the traffic is no longer on port 80, its probably on 8080). There are lists of FREE http proxies available on the web, or if you wish, set up your own squid proxy machine, its pretty simple. Let me know if this works for you.

[TheOtherPhil]

Is it not possible to allow connections on an alternate port? Free http proxies are not always reliable and I would still have to go through my ISP's transparent proxy cache when using a local squid proxy

[TheOtherPhil]

OK, just an update...I have been able to get my *nix boxen to upload by using a proxy in Poland. IP 195.205.230.210:8080. I suppose this is better than how I was doing it even though it is a litlle slow. Allowing connections to your server on an alternate port would be the best idea though

Thanks,
Phil.

[Brian the Fist]

I am not convinced that changing our server configuration is the solution. What I would really like to do is find out what these friggin transparent proxies have against my packets in the first place... I am sure it is something simple which I could fix if I only knew how.
Anyone out there consider themselves a TCP/IP expert?

[bwkaz]

What are the proxies doing to the packets? I'm not an expert by any stretch of the imagination, but I do know a bit about the protocols.

Are you seeing the packets at all at your end? Or is the proxy dropping them?

[Brian the Fist]

It appears that some transparent proxies 'eat' the reply packets from our server, so the data gets uploaded fine but the client does not receive acknowledgement (absolutely no reply as confirmed by packet sniffers) of this and so assumes the upload failed. And keeps the data to try again next time, and so on, and so on..

[bwkaz]

Hmm... is your server acting like it's a web server to the outside world? If not, that might have something to do with it, maybe. The proxies may look at the packet payload and decide that "this isn't from a web server" or something, and block it because they think it's malicious.

I assume the return packets aren't being sent to any specific port, right? You're not doing like FTP and trying to create a separate connection? Well, no, you aren't, because my iptables firewall is letting everything work normally. All it does on return packets is match on --state ESTABLISHED,RELATED (est. matches stuff on the same connection, related matches stuff like FTP that's on a new connection but is related to the original) and jumps to accept, and it works.

Of course, without seeing how these proxies are set up (if they're iptables scripts, or if they're proprietary solutions, or what), I'm not quite sure what to look at, but these are the first few things that come to mind.

Something on the proxy is deciding either that the packets are malicious, or that they're not related to the outgoing connection, so it's killing them, at least that's my interpretation...

[DB7654321]

One thing I have noticed with my Squid transparent proxy, Howard;

When the client sends the HTTP request, it does not appear to send the hostname in the GET or POST request..... An example of the consquences of this:

[16:26:48][david@david:~]$ telnet distributedfolding.org 80
Trying 206.248.62.8...
Connected to distributedfolding.org.
Escape character is '^]'.
GET / HTTP/1.0

HTTP/1.0 200 OK
Date: Sun, 30 Mar 2003 22:27:08 GMT
Server: Apache/1.3.26 (Unix) Debian GNU/Linux mod_gzip/1.3.19.1a mod_ssl/2.8.9 OpenSSL/0.9.6g PHP/4.1.2
X-Powered-By: PHP/4.1.2


...This request ends up at the webserver on my squid box. ...Squid doesn't know where to send the packet. Compare to this:

[16:27:08][david@david:~]$ telnet distributedfolding.org 80
Trying 206.248.62.8...
Connected to distributedfolding.org.
Escape character is '^]'.
GET http://www.distributedfolding.org/ HTTP/1.0

HTTP/1.0 200 OK
Date: Sat, 01 Mar 2003 21:28:35 GMT
Server: Apache/1.3.23 (Unix) (Red-Hat/Linux) DAV/1.0.3 PHP/4.1.2 mod_perl/1.26

...Which, is your webserver. Perhaps this has nothing to do with proxies and DF, but, I suspect it does. It should also be easy to fix.

[Brian the Fist]

Can you elaborate on that a bit - I'm not sure I get while the full hostname and domain is needed in the GET request... You may be on to something though. thanks.

Furthermore, when you telnet to port 80, would that still be going through your squid proxy??? How would that work?

[DB7654321]

Originally posted by Brian the Fist
Can you elaborate on that a bit - I'm not sure I get while the full hostname and domain is needed in the GET request... You may be on to something though. thanks.

Furthermore, when you telnet to port 80, would that still be going through your squid proxy??? How would that work?

Sure. I believe the domain is necessary in the HTTP request to tell the proxy which domain to request... Perhaps some (or most) proxies cannot, when being used transparently, gain as much information on the connection to be made as happens when the client communicates directly with the proxy. It's kinda tricky to explain. :-\

Yes -- I'm using a transparent proxy on my home LAN...All traffic heading to the internet interface on port 80 is routed through my Squid proxy.

If you'd like to test this firsthand, email me and I'll get you a shell on this box.