Results 1 to 36 of 36

Thread: Interesting memory usage.

  1. #1

    Interesting memory usage.

    I have 2 DF clients running right now (on my Dual CPU system) that have been going for over a day and their memory usage (as reported by Windows Task Manager) is quite high in XP Pro.

    When I first started the two client, they were taking up 88908K and 88956K. They are now taking 156423K and 175234K each, about double the original amount of ram.

    DF is running as a service with:
    service=1
    useram=1
    progress=1

    When I stop the service and restart the service again, both clients go abck to taking 88900K again. I will see if this happens again after letting the clients run over night.

    Jeff.

  2. #2
    25/25Mbit is nearly enough :p pointwood's Avatar
    Join Date
    Dec 2001
    Location
    Denmark
    Posts
    831
    My client has gone of from ~90Mb -> 105MB.
    Pointwood
    Jabber ID: pointwood@jabber.shd.dk
    irc.arstechnica.com, #distributed

  3. #3
    This morning my 2 clients have gone to 93104K and 96604K.

    I came back about 60 minutes later and they are now at 94630L and 97105K. It seems to go up about 700-1000K after each generation is complete.

    Jeff.
    Last edited by Digital Parasite; 06-20-2003 at 08:17 AM.

  4. #4
    Could be a memory leak although I think we checked for these already. We'll see what we can find...
    Howard Feldman

  5. #5
    Ok thanks. I just checked all my machines on XP and Win2k and they are all doing this, increasing their memory every generation. I happen to be running all of them as a service.

    Jeff.

  6. #6
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    Here is the before I stop the client - Windows XP
    Attached Images Attached Images

  7. #7
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    Here is after I stopped it, then restarted. I waited approximately one minute before taking this snapshot.
    [edit] checked 7 other w2k boxen, one was over 200meg usage, the rest were much higher than, the less than 90meg, it should be using.
    Attached Images Attached Images

  8. #8
    Keep in mind that it keeps certain things (eg log files) in memory while folding and dumps them out to disk when you quit, so this explains part of the extra memory usage. Nevertheless there could indeed be a leak.
    Howard Feldman

  9. #9
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    Just got back from stopping and re-starting the clients on 17 remote boxen. One was over 300meg.
    All w2k, none running as a service, all using the -qt -rt flags only.

  10. #10
    Senior Member
    Join Date
    Apr 2002
    Location
    Santa Barbara CA
    Posts
    355
    I just restarted a client running as a service on XP for about a day which was up to 124MBs.

    4 linux boxen running for 2 days were from 140-166MBs.

    1 linux box running for 3 days was at 184MB.

  11. #11
    Senior Member
    Join Date
    Mar 2002
    Location
    MI, U.S.
    Posts
    697
    Hmm... my systems were doing something superficially similar to this too, back in one of the later betas. Of course, it took MUCH longer than one night to become noticeable, though, so I have a feeling it's not quite the same thing.

    However, my firewall/router (the only one of the two that's on all the time anymore) is now up to 179MB resident, and 194MB virtual (so 25MB swapped out). It's been running for about 3 days.

    During the beta, my resident size would stay at around 100MB, not rise with the other (the only rising value was virtual size). Unfortunately, I can't run pmap on the process ID on that box, due to the grsecurity options I have enabled -- and to disable them, I'd have to change an init script and reboot, which would kill the process anyway.

    But, I can run top, like you were asking for last time, Howard. Results (I only copied the foldtrajlite process; no point in getting anything else):

    Code:
      PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME COMMAND
    21992 avatar    20  19  194M 179M  4236 R N  96.4 71.4  4011m foldtrajlite
    Switches are -qt -it -rt.
    Last edited by bwkaz; 06-20-2003 at 07:41 PM.

  12. #12
    Junior Member theBRAINbelly's Avatar
    Join Date
    Jun 2003
    Location
    Everywhere
    Posts
    24
    Here is the current output from a Win 2000 Pro without a reboot since the phase change.



    I'll take another snapshot in 12 hours...
    Every passing minute is a chance to turn it all around...

  13. #13
    Hmm.. whilst we try to figure this out, could someone try running withOUT the -rt option, and let us know if you still appear to me getting a leak of the same magnitude (you should, if it really is a leak).? Thanks.
    Howard Feldman

  14. #14
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    I'm all over that request starting right now Howard.
    I removed all switches. On one box of course.

  15. #15
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    I had only removed the -qt switch, (sorry) and didn't see any increase in ram usage.
    Now running with the -qt switch and no -rt.

  16. #16
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    It's still early to tell, but it appears to be ok with just the -rt.
    I'm thinking it may have something to do with running two switches at the same time.

    Can a few of you run with only just ONE of the switches please?
    -qt only or
    -rt only
    I'd like to see what results you get and post here any observations.

  17. #17
    Senior Member
    Join Date
    Apr 2002
    Location
    Santa Barbara CA
    Posts
    355
    I found a couple of my borged computers that are RAM challenged and so run without the -rt switch.

    Both run the client as a service under XP.

    90 hours cpu time and 47MBs memory.
    86 hours cpu time and 56MBs memory.

  18. #18
    Junior Member theBRAINbelly's Avatar
    Join Date
    Jun 2003
    Location
    Everywhere
    Posts
    24
    Here is an XP Pro withOUT the -rt switch applied.



    Large enough to drive a Mini C through...
    Every passing minute is a chance to turn it all around...

  19. #19
    HCW DF Team Leader rstarr's Avatar
    Join Date
    Dec 2001
    Location
    Fort Knox, KY
    Posts
    82
    I was getting out of memory errors on my win98SE systems, with 512 meg of PC2100 ram. Once I stopped the GUI and only ran the client Dos Text box verison, things were ok.

    BTW...what are the -rt and -qt??? What do they stand for?
    Folding 24/7 for a cause
    www.hardCOREware.net
    HCW DF Team!

  20. #20
    HCW DF Team Leader rstarr's Avatar
    Join Date
    Dec 2001
    Location
    Fort Knox, KY
    Posts
    82
    Never mind the -rt and -qt questions. I found the answer.

    But I thought the -rt, extra ram, was good?
    Folding 24/7 for a cause
    www.hardCOREware.net
    HCW DF Team!

  21. #21
    Senior Member
    Join Date
    Jul 2002
    Location
    Kodiak, Alaska
    Posts
    432
    If you have 256megs of Ram, (depending on protein, and operating system, the value may be lower..) the -rt (use extra ram) switch roughly doubles your performance.

    The people here are trying to see what's causing a buildup in memory usage; which is why Howard asked them to test out the client without the -rt switch.

  22. #22
    Memory usage from some machines run without the -rt swich:


    Code:
    CPU time Mem usage	Generation
    
    115:19	 31.4 Mb	29
    116:48	 47.8 Mb	22
    116:53	 47.8 Mb	29
    116:27	 55.4 Mb	43
    116:16	 57.7 Mb	60
    All similar machines, client not restarted since gen 1. Os is windows 2000 and clients are all run as service.

  23. #23
    Dual P4-2.8 Xeon server - 2 instances installed as a service with the extra RAM option in the service.cfg, direct internet connection - running since Friday night by this morning (Monday) one instance was using 285MB and the other was using 260MB, other servers around the 185MB mark, stopping and restarting brings this back down to the 90-100MB - I'll monitor and see if they increase again

  24. #24
    Keeper of the Fridge PY 222's Avatar
    Join Date
    Jul 2002
    Location
    San Jose, CA
    Posts
    2,706
    I have just downloaded a fresh copy from the DF site onto a P4 1.3GHz, Win2k machine and used dfGUI to manage the client.

    Use the Extra RAM flag and let it rip through the weekend.

    Just came in to the office today and tried opening up IE. The whole machine almost had a heart attack.

    Looked at the Task Manager, noticed that foldtrajlite has taken over 200MB worth of memory.

    Something fishy? I should say so!

    To make it even stranger, this "memory leak" is not found on my other machines! I don't get it.

  25. #25
    Ancient Haggis Hound Angus's Avatar
    Join Date
    Jan 2002
    Location
    Seattle/Norfolk Island
    Posts
    828
    This requires no further explanation
    Attached Images Attached Images

  26. #26
    Boinc'ing away
    Join Date
    Aug 2002
    Location
    London, UK
    Posts
    982
    just noticed mine has jumped to :



    Gen 168, no laxness, Windows XP, running as a service, progress update as 1, use extra RAM...

    just hope it goes down

  27. #27
    Senior Member
    Join Date
    Apr 2002
    Location
    Santa Barbara CA
    Posts
    355
    Originally posted by pfb
    just noticed mine has jumped to :



    Gen 168, no laxness, Windows XP, running as a service, progress update as 1, use extra RAM...

    just hope it goes down
    I don't think that hoping it goes down is as effective as restarting.

  28. #28
    Stats God in Training Darkness Productions's Avatar
    Join Date
    Dec 2001
    Location
    The land of dp!
    Posts
    4,164
    Can anybody compare laxness levels with higher memory usage? AKA, if the structure is more lax, does the memory usage go up?

    If we can't figure out exactly what it is that's causing this, maybe we can narrow it dow for Howard

  29. #29
    Based on what I have seen so far:

    Win 98 does not release the memory even after shutting down the client. A reboot is necessary to reclaim the memory.

    Win NT4 and Win2K seem to reclaim the memory after shutting down the client.

    It's the linux boxes that I find most interesting. Several show that the client is only using 80M but the total is in the 120M range. It show the remaining in the swap file even though the swapfile shows no usage.

    This leads me to believe that either the file handle wasn't destroyed or the object wasn't destroyed. But it is not necessarily related to the -rt switch or the -g switch.

    It does seem to be related to the amount of stickyness and the completed gens. Almost as if the temp files cause memory to be retained.

    Just my thoughts and observations!
    They are probably incorrect, but thats my gut feeling anyway.

  30. #30
    Ancient Haggis Hound Angus's Avatar
    Join Date
    Jan 2002
    Location
    Seattle/Norfolk Island
    Posts
    828
    Originally posted by pfb
    just noticed mine has jumped to :



    Gen 168, no laxness, Windows XP, running as a service, progress update as 1, use extra RAM...

    just hope it goes down
    It never goes down by itself. Restart the client to start over at the <90MB memory level.

  31. #31
    Stats God in Training Darkness Productions's Avatar
    Join Date
    Dec 2001
    Location
    The land of dp!
    Posts
    4,164
    I just noticed on this windows box that C:\WINNT\Temp had about 103MB of files related solely to DF in it. Not sure if it's related, but it's possible.

    Originally posted by PinHead
    It does seem to be related to the amount of stickyness and the completed gens. Almost as if the temp files cause memory to be retained.

    Just my thoughts and observations!
    They are probably incorrect, but thats my gut feeling anyway.

  32. #32
    OCworkbench Stats Ho
    Join Date
    Jan 2003
    Posts
    519
    Thanks for the tip on shutting down 98 SE..it explains a lot of missing ram :bs: Looked at my Gigabyte 98SE machine with 256 MB, and thank goodness I did. 17K Physical Ram free .

    I am not a Stats Ho, it is just more satisfying to see that my numbers are better than yours.

  33. #33
    25/25Mbit is nearly enough :p pointwood's Avatar
    Join Date
    Dec 2001
    Location
    Denmark
    Posts
    831
    I just restarted the client on my Duron 1.2 with 512MB mem and WinXP - client installed as a service.

    It was using ~187MB mem - after the restart it's now down to <90MB.
    Pointwood
    Jabber ID: pointwood@jabber.shd.dk
    irc.arstechnica.com, #distributed

  34. #34
    Has anyone determined a max for the client in terms of RAM? i.e. would it help to load one of my machines with a gig of RAM and see if there is an upper limit to how much it will use? The biggest I have seen anyone report so far is in the area of 335MB.

    And what about those "memory clearing" progs that are everywhere? They always seemed useless before now except when working with gigantic (2gb+) TIFF files and the like, but would they perhaps force the disk writes that Howard mentioned?

    Just my curious .02
    -Self Denying Stat Ho-

  35. #35
    Ancient Haggis Hound Angus's Avatar
    Join Date
    Jan 2002
    Location
    Seattle/Norfolk Island
    Posts
    828
    I've had W2K Adv Srvr machines using well over 500MB per instance of DF. I doubt that it will stop acquiring memory until there is no more.

  36. #36
    I have several P4 FreeBSD servers, and they are also all showing an increase in memory usage over time. I run the -rt and -q flags.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •