Interesting memory usage.

[Digital Parasite]

I have 2 DF clients running right now (on my Dual CPU system) that have been going for over a day and their memory usage (as reported by Windows Task Manager) is quite high in XP Pro.

When I first started the two client, they were taking up 88908K and 88956K. They are now taking 156423K and 175234K each, about double the original amount of ram.

DF is running as a service with:
service=1
useram=1
progress=1

When I stop the service and restart the service again, both clients go abck to taking 88900K again. I will see if this happens again after letting the clients run over night.

Jeff.

[pointwood]

My client has gone of from ~90Mb -> 105MB.

[Digital Parasite]

This morning my 2 clients have gone to 93104K and 96604K.

I came back about 60 minutes later and they are now at 94630L and 97105K. It seems to go up about 700-1000K after each generation is complete.

Jeff.

[Brian the Fist]

Could be a memory leak although I think we checked for these already. We'll see what we can find...

[Digital Parasite]

Ok thanks. I just checked all my machines on XP and Win2k and they are all doing this, increasing their memory every generation. I happen to be running all of them as a service.

Jeff.

[IronBits]

Here is the before I stop the client - Windows XP

[IronBits]

Here is after I stopped it, then restarted. I waited approximately one minute before taking this snapshot.
[edit] checked 7 other w2k boxen, one was over 200meg usage, the rest were much higher than, the less than 90meg, it should be using.

[Brian the Fist]

Keep in mind that it keeps certain things (eg log files) in memory while folding and dumps them out to disk when you quit, so this explains part of the extra memory usage. Nevertheless there could indeed be a leak.

[IronBits]

Just got back from stopping and re-starting the clients on 17 remote boxen. One was over 300meg.
All w2k, none running as a service, all using the -qt -rt flags only.

[Welnic]

I just restarted a client running as a service on XP for about a day which was up to 124MBs.

4 linux boxen running for 2 days were from 140-166MBs.

1 linux box running for 3 days was at 184MB.

[bwkaz]

Hmm... my systems were doing something superficially similar to this too, back in one of the later betas. Of course, it took MUCH longer than one night to become noticeable, though, so I have a feeling it's not quite the same thing.

However, my firewall/router (the only one of the two that's on all the time anymore) is now up to 179MB resident, and 194MB virtual (so 25MB swapped out). It's been running for about 3 days.

During the beta, my resident size would stay at around 100MB, not rise with the other (the only rising value was virtual size). Unfortunately, I can't run pmap on the process ID on that box, due to the grsecurity options I have enabled -- and to disable them, I'd have to change an init script and reboot, which would kill the process anyway. ;)

But, I can run top, like you were asking for last time, Howard. Results (I only copied the foldtrajlite process; no point in getting anything else):


PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
21992 avatar 20 19 194M 179M 4236 R N 96.4 71.4 4011m foldtrajlite Switches are -qt -it -rt.

[theBRAINbelly]

Here is the current output from a Win 2000 Pro without a reboot since the phase change.

http://www.adflix.com/wb/Images/win2k-pro-2-process-1.jpg

I'll take another snapshot in 12 hours...

[Brian the Fist]

Hmm.. whilst we try to figure this out, could someone try running withOUT the -rt option, and let us know if you still appear to me getting a leak of the same magnitude (you should, if it really is a leak).? Thanks.

[IronBits]

I'm all over that request starting right now Howard.
I removed all switches. On one box of course. :)

[IronBits]

I had only removed the -qt switch, (sorry) and didn't see any increase in ram usage.
Now running with the -qt switch and no -rt.

[IronBits]

It's still early to tell, but it appears to be ok with just the -rt.
I'm thinking it may have something to do with running two switches at the same time.

Can a few of you run with only just ONE of the switches please?
-qt only or
-rt only
I'd like to see what results you get and post here any observations.

[Welnic]

I found a couple of my borged computers that are RAM challenged and so run without the -rt switch.

Both run the client as a service under XP.

90 hours cpu time and 47MBs memory.
86 hours cpu time and 56MBs memory.

[theBRAINbelly]

Here is an XP Pro withOUT the -rt switch applied.

http://www.adflix.com/wb/Images/XP-Pro-process-2-no-rt.jpg

Large enough to drive a Mini C through...

[rstarr]

I was getting out of memory errors on my win98SE systems, with 512 meg of PC2100 ram. Once I stopped the GUI and only ran the client Dos Text box verison, things were ok.

BTW...what are the -rt and -qt??? What do they stand for?

[rstarr]

Never mind the -rt and -qt questions. I found the answer. :)

But I thought the -rt, extra ram, was good?

[tpdooley]

If you have 256megs of Ram, (depending on protein, and operating system, the value may be lower..) the -rt (use extra ram) switch roughly doubles your performance.

The people here are trying to see what's causing a buildup in memory usage; which is why Howard asked them to test out the client without the -rt switch.

[CyberDuck]

Memory usage from some machines run without the -rt swich:




CPU time Mem usage Generation

115:19 31.4 Mb 29
116:48 47.8 Mb 22
116:53 47.8 Mb 29
116:27 55.4 Mb 43
116:16 57.7 Mb 60


All similar machines, client not restarted since gen 1. Os is windows 2000 and clients are all run as service.

[Terminator]

Dual P4-2.8 Xeon server - 2 instances installed as a service with the extra RAM option in the service.cfg, direct internet connection - running since Friday night by this morning (Monday) one instance was using 285MB and the other was using 260MB, other servers around the 185MB mark, stopping and restarting brings this back down to the 90-100MB - I'll monitor and see if they increase again :(

[PY 222]

I have just downloaded a fresh copy from the DF site onto a P4 1.3GHz, Win2k machine and used dfGUI to manage the client.

Use the Extra RAM flag and let it rip through the weekend.

Just came in to the office today and tried opening up IE. The whole machine almost had a heart attack.

Looked at the Task Manager, noticed that foldtrajlite has taken over 200MB worth of memory.

Something fishy? I should say so! :crazy:

To make it even stranger, this "memory leak" is not found on my other machines! I don't get it. :cry:

[Angus]

This requires no further explanation :scared:

[pfb]

just noticed mine has jumped to :

http://wibble.bounceme.net/DD/DF/hi_mem.png

Gen 168, no laxness, Windows XP, running as a service, progress update as 1, use extra RAM...

just hope it goes down :crazy:

[Welnic]

Originally posted by pfb
just noticed mine has jumped to :

http://wibble.bounceme.net/DD/DF/hi_mem.png

Gen 168, no laxness, Windows XP, running as a service, progress update as 1, use extra RAM...

just hope it goes down :crazy:

I don't think that hoping it goes down is as effective as restarting. :)

[Darkness Productions]

Can anybody compare laxness levels with higher memory usage? AKA, if the structure is more lax, does the memory usage go up?

If we can't figure out exactly what it is that's causing this, maybe we can narrow it dow for Howard ;)

[PinHead]

Based on what I have seen so far:

Win 98 does not release the memory even after shutting down the client. A reboot is necessary to reclaim the memory.

Win NT4 and Win2K seem to reclaim the memory after shutting down the client.

It's the linux boxes that I find most interesting. Several show that the client is only using 80M but the total is in the 120M range. It show the remaining in the swap file even though the swapfile shows no usage.

This leads me to believe that either the file handle wasn't destroyed or the object wasn't destroyed. But it is not necessarily related to the -rt switch or the -g switch.

It does seem to be related to the amount of stickyness and the completed gens. Almost as if the temp files cause memory to be retained.

Just my thoughts and observations!
They are probably incorrect, but thats my gut feeling anyway.

[Angus]

Originally posted by pfb
just noticed mine has jumped to :

http://wibble.bounceme.net/DD/DF/hi_mem.png

Gen 168, no laxness, Windows XP, running as a service, progress update as 1, use extra RAM...

just hope it goes down :crazy:

It never goes down by itself. Restart the client to start over at the <90MB memory level.

[Darkness Productions]

I just noticed on this windows box that C:\WINNT\Temp had about 103MB of files related solely to DF in it. Not sure if it's related, but it's possible.


Originally posted by PinHead
It does seem to be related to the amount of stickyness and the completed gens. Almost as if the temp files cause memory to be retained.

Just my thoughts and observations!
They are probably incorrect, but thats my gut feeling anyway.

[Grumpy]

Thanks for the tip on shutting down 98 SE..it explains a lot of missing ram :bs: Looked at my Gigabyte 98SE machine with 256 MB, and thank goodness I did. 17K Physical Ram free .

:bonk:

[pointwood]

I just restarted the client on my Duron 1.2 with 512MB mem and WinXP - client installed as a service.

It was using ~187MB mem - after the restart it's now down to <90MB.

[AMDPHREAK]

Has anyone determined a max for the client in terms of RAM? i.e. would it help to load one of my machines with a gig of RAM and see if there is an upper limit to how much it will use? The biggest I have seen anyone report so far is in the area of 335MB.

And what about those "memory clearing" progs that are everywhere? They always seemed useless before now except when working with gigantic (2gb+) TIFF files and the like, but would they perhaps force the disk writes that Howard mentioned?

Just my curious .02

[Angus]

I've had W2K Adv Srvr machines using well over 500MB per instance of DF. I doubt that it will stop acquiring memory until there is no more.

[erk]

I have several P4 FreeBSD servers, and they are also all showing an increase in memory usage over time. I run the -rt and -q flags.