New Client memory Leak

[Grumpy]

It appears the old memory leak may have returned. Both my XP and Win 2000 Systems are slowly chewing up ram for Foldtrajlite. Starts at around 88 MB, most now between 115-145 MB...::Pokes:

[deranged128[OCAU]]

Originally posted by Grumpy
It appears the old memory leak may have returned. Both my XP and Win 2000 Systems are slowly chewing up ram for Foldtrajlite. Starts at around 88 MB, most now between 115-145 MB...::Pokes:

I'm seeing it also Grumpy, Win2K system with memory usage now at 147MB, from 91MB at start and a WinXP system now runnign at 149MB from 92MB.

Haven't checked the others (they are at home) but is this going to affect performance? :help:

[erk]

Originally posted by deranged128[OCAU]
I'm seeing it also Grumpy, Win2K system with memory usage now at 147MB, from 91MB at start and a WinXP system now runnign at 149MB from 92MB.

Haven't checked the others (they are at home) but is this going to affect performance? :help: Hmm this might explain why my diskless farm fell over, as they run off a RAM disk image.

[Ned]

Looks like Grumpy is right... Task manager reported that trajfoldlite.exe was using 136,844K. When I stopped and restarted it, it used 95,800K. A LEAK is present with this client... :(

Ned

[rsbriggs]

Ditto here. All my diskless pharm boxen are falling over...

[Nofinger]

It seems not all systems are affected because I cant see any leak of memory at my systems

3x win xp
2x amd 2400+, 1x amd 1333

[the-mk]

Originally posted by Nofinger
It seems not all systems are affected because I cant see any leak of memory at my systems

3x win xp
2x amd 2400+, 1x amd 1333

And, what is your memory-usage by the client?

[bwkaz]

I didn't check what the mem usage was when the client updated, but it's been running for the past three days on my firewall box, and the total VSZ (that is, including code, shared libraries, data, and everything) is about 94MB. Doesn't seem to be leaking for me...

Linux (From Scratch), using the ICC client, on a P3-800 (though the processor shouldn't matter, in theory at least...).

[Nofinger]

Originally posted by the-mk
And, what is your memory-usage by the client?

97mb, 98mb and 107mb
these usages are about the same as when I yesterday looked.

[Grumpy]

So Penguin Boxen are free of holes :spank:

[tpdooley]

.. winXp Pro Athlon 2600+ 1gig ram.
at 54 hours - 132,152k
at 57 hours - 135,205k
at 65 hours - 78,504k

looks like it recovers itself..

although all the win2k and winxp (home or pro - intel or AMD) machines at work were also in the 120M+ range

[Stardragon]

Keep in mind that the client stores quite a few values in memory as it is crunching through the generations - since this protein requires more detailed calculations to obtain the fitness score, it is possible that it is simply using more memory than the previous protein.

But do keep an eye out, even the DF team isn't safe from potential memory leaks :jester:

[GHOST]

MEMORY USED 205,540. WINXP, just crunching, left alone for couple days.:trash:

[tpdooley]

keep track of the memory usage every hour and you should see it eventually plummet.. and then climb; like my spot checking showed.

[Digital Parasite]

The old memory leak turned out to be that the client did not release Handles properly. If you go into your Windows Task Manager and add the column "Handles" to your processes list, you can watch to see if the # of handles keeps increasing for your foldtrajlite.exe process and never goes down. The more often your "progress.txt" file gets updated by the DF client, the more handles got eaten up.

If that is not happening, then its a different problem.

Jeff.

[Brian the Fist]

Since the 'memory leak' does not occur on every machine (not even every windows box) I do not think it could be a true memory leak. The usage reported by Windows cannot be taken as 100% reliable. Can anyone verify with a 3rd party application that it is truly leaking memory - and remember its only a leak if the memory is not eventually released.

[deranged128[OCAU]]

One of our team members is using diskless clients with a *nix variant and FreeBSD. He also experienced this problem to the point that the clients fell over with all work lost.

I'll get him to post in our forums the series of events and transcribe it here if he's not registered for these forums.

All that said, it's not just a windows problem. More info later to back that up.

An interesting obervation so far though is that of my clients, the faster the client, the quicker the memory usage increases. I'll check the handle scenario also as suggested by Jeff.

[JetBlack69]

I appear to be having a memory issue as well. My windows XP box has been running it as a service for 97 hours now and it's using about 174,664K.

Also, my linux box (gentoo) has been running for 228 hours and is using 38.4% of 768MB of memory (according to 'top') That's 294MB of memory :scared: :shocked: Both of these are auto updates.

[bwkaz]

I'm going to have to reverse what I said earlier in this thread. The client has now been running for about 11 days, and it's VSZ is up to 243,776K, with 172,584K of that being RSS (according to ps):


USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
<me> 5992 90.6 67.1 243776 172584 ? RN Oct28 14711:20 ./foldtrajlite -f protein -n native

pmap doesn't show anything useful (due to restrictions on the contents of /proc/maps from grsecurity; not knowing the size or load address of libc.so makes it a lot harder for anybody to perform a return-to-libc type attack), except for about 30 writable-private maps where a client that's only been running for 10 hours only has about 10 of them. But since I can't tell what's mapped there, I can't be any more help...

It may still not be a memory leak, though. It might just be that glibc doesn't always actually un-brk() memory that you free() -- it keeps it so that if you malloc() it again later, it just uses it instead of having to re-call brk(). Not that I actually believe that's happening, but it is a possibility.

[Brian the Fist]

Since Linux is more trustworthy at reporting memory usage, I'll try the latest client build here and see if I observe any leaks... It'll probably take a while from the sounds of things though so please be patient :D

[Dyyryath]

I'm definately seeing it under Linux. I just went through all of my Linux boxes and switched them to -rf from -rt. Not one of them appeared to be working properly. Here's an example of what I was seeing:

http://www.zerothelement.com/offsite-images/df-memory.png

Not one of my machines had a client using less than 300mb of RAM. Some (as above) were using considerably more than 300mb.

[erk]

I downloaded the new Linux client a few hours ago, after running for 2 hours with the -rt and -q flags the memory usage has already crept from 91MB to 102MB, that's not a good sign :(

I made sure there were no files from the previous client by deleteing the directory and starting fresh.

[Brian the Fist]

I dont have any great leak detection software for Linux but Ill try what I've got and see what happens...

[bwkaz]

May I suggest one of:

mmalloc (Mark Malloc; it may only work when the client exits)
memprof (this looks like a good one!)
libyama (don't know much about it, but it turned up on Freshmeat...)

You of course wouldn't want to keep any GPL'ed leak detector linked into the client when you distribute it, or the client would have to be GPL'ed. But libyama can be LD_PRELOAD'ed instead of explicitly linked, and memprof doesn't even require linking into your program (it looks like a standalone program). I think mmalloc is BSD-licensed, but you'd want to double check that.

[iggy]

I found nothing strange on WinXP Pro. Client takes anything from 15 to 125 Mb, depending of computation phase, uses more memory when it gets stuck with difficult formation, but the memory is released or acquired as necessary for its work.

Settings: -qt -rt -i f -g 1, with priority ranges from 0 to 10.

I can't really say this is memory leak...

[Grumpy]

When it reaches 314 MB, I call it a leak :gone:

[bwkaz]

From the News page:


11/20/2003 Memory leak fixed

A small memory leak has been fixed - new executables are available for download. The changes to the client will be included in the next protein auto-update Guess you didn't need to use any of those leak detectors after all... ;)

:notworthy :thumbs:

[Welnic]

Looks like the same behavior as before to me.


PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME COMMAND
14430 clint 20 19 93992 91M 2056 R N 99.6 18.1 5:58 foldtrajlite
12465 clint 19 19 158M 158M 2224 R N 99.2 31.3 3755m foldtrajlite
14435 clint 2 0 940 940 748 R 0.9 0.1 0:00 top