Memory leak in Windows text client?

[Welnic]

I have one Windows box that has enough RAM and is enough mine that I run with use ram enabled. I have looked at it a couple of times after it has been running over two days on this last protein (69AA) and the memory it uses goes from around 65MBs when first started to around 88MBs. I don't know if it would get any higher than that if it runs longer. If stopped and restarted it goes back to 65MBs.

I am used to this in the OSX version so I have it stop and restart every 12 hours. This is the first I have seen of this on the Windows side. I looked at the boxen that I have running without the extra RAM and they are all right around 35MBs as expected.

Clint

[Halon50]

Huh. You're right. I've never noticed this myself until you mentioned it. On one of my machines, I've calculated the leak to be 82KB per set of 5000 proteins submitted (27000/327, sample size of roughly 3 days of CPU time).

A simple stop/restart of the client releases the memory back to Windows though, and it seems small enough that the weekly protein updates would force the client to release accumulated memory before it causes problems on the host.

I'll have to take a closer look a couple of my Win9x machines and see if it's duplicated there.

[Halon50]

The problem exists on my Win9x machines as well, but these get rebooted every 2 to 3 days, so there's no problem for me there.
;)

[Brian the Fist]

Must be a leak in the OS as it doesn't seem to happen on XP or 2000 using the SAME executable...

[Welnic]

The box that has the leak is XP. I guess I forgot that part.

[IronBits]

w2k Server and w2k Workstation. Using Task Manager
Checked both, memory was at 82mb, restarted, down to 69mb
the other (workstation) was at 77mb.
Win98 was 87mb now, after restart 75mb (using TaskInfo)
Current running clients...
w2k workstation, DUAL up to 84.5 and 85.2
Mandrake 8.2, Dual, 83mb each process
Win98 112.6 mb (running client since new protien) re-started now 78mb
Who's cooking fish? ;)

[Paratima]

TaskInfo on Win2K workstation reported 7MB loss overnight. :(
Only the DF client was running.

[markhl]

The amount of memory used by the Windows text client on my Win2000 box went from 69 MB to 95 MB over the weekend (65 hours). I used Task Manager to measure it. Whenever I restart, it goes back to 69 MB.

I notice that the memory allocation is changing all the time; could there a small leak associated with this allocation / deallocation?

I haven't checked to see whether memory usage jumps after an upload.

Mark

[Paratima]

After reading this thread last night, I rebooted a Win2K box, started the client, and measured the RAM with TaskInfo.

Free physical RAM 115,972 KB at 2315 Sunday.

No other programs were run on this box. I just measured it again:

Free physical RAM 90,468 KB at 2145 Monday.

Stopped and restarted the client.

Free physical RAM 112,492.

So over 22 1/2 hours, it lost 25 MB, with a discrepancy of 3 MB.

Whether that's something Howard wants to investigate further, only Howard knows. There are tools that can measure memory leakage precisely...this was just a very crude test.

Could it be the OS? Right now, with a really HOT protein to work on, I'm unwilling to have an 1800+ box sit idle to test the theory.

[markhl]

PS; I should have said that my measurements were done on a text client running as a service with the high-RAM -rt option.

Mark

[Paratima]

Oops! Mine too.

[IronBits]

text clients using -df -rt on all of em.
Up on the screen where I can see em running to. ;)

[tquade]

The problerm seems to exist on Win95, 98, NT4.0 and W2K.

Ted

[Brian the Fist]

Ok, I haven't looked at it that closely with the -rt option, you didnt mention that before. I will look around and see if I spot any leaks.

[Halon50]

New protein memory leak specs:

188 sets of 5000 proteins submitted over 24 hours.

Memory usage went from 58,344K to 67,152K.

8808K/188 sets ~= 47K lost per set.

EDIT: I mistakenly took the last memory usage sample at the end of a protein set. Subtracting ~167K from 8808K makes average memory lost ~=46K per set instead.

I've noticed the client takes up an additional 4K for every approximately 30 proteins complete, which seems about right if it's storing data on each protein. I have no statistics for what happens right when a data set completes (i.e. if ~47K is lost every data set, or if it's a more randomized number).

[McGoff]

here is a screen dump of a stop/start of one of my Bits that has been running since the protein change the ram was at 146meg used and has dropped to 133meg on the restart

http://users.bigpond.net.au/duenorth/pics2/memloss.jpg

McGoff

[runestar]

Hmm... I haven't noticed it on my W2K box... you do realize that the amount of RAM will vary from structure to structure... as it processes the structure, more RAM is used, and when it finishes it drops back down to around 60MB on the current protein. Also to be taking into consideration (based on what Howard has said earlier) is that the -rt is storing most of the files in memory. So as the file sizes get bigger, it would follow that the memory usage would increase.

If you're using Windows 95 and you are rebooting every few days... you probably already realize that its 95 that is the memory culprit and not necessarily the client itself. Win98SE is probably better off coming about 4 years or so after the original Windoze 95. However, it too still suffers from this chronic dirty memory syndrome too to some extent. NT and 2000 and XP (Pro) were designed to be run continously so they are better at memory cleanup than the 9x platforms including Win ME (Mostly Errors?).

I'll have to track it myself and see if note anything...

Best,

RS½
PBB

[markhl]

Echoing Halon50's results, memory leak with the new protein:

In 24 hours my Win2000 box submitted 178832 structures with the -rt switch and the new protein, about 36 uploads.

Task Manager: memory usage went from 60 to 70 MB. If the memory jumps after each upload, then that's about 280 KB/upload.

[markhl]

Memory leak with the new protein in low-RAM mode:

In 8.3 hours my Win2000 box submitted 36017 structures without the -rt switch, about 8 uploads.

Task Manager: memory usage went from 25 to 27 MB. If the memory jumps after each upload, then that's about 250 KB/upload.

That is about the same rate / upload as the high-RAM -rt mode, just less noticeable because the low-RAM mode runs more slowly, and because it uses less RAM to start with.

[Angus]

I've been quietly watching this for quite a while now. It seems to be worst on Win98 boxes.

I have one 98SE box with 256MB ram that does nothing but DF, and after running the text client for a day or so, the locked physical memory has increased dramatically, and the PC GUI noticeably slows down - the mouse pointer motion gets very jerky, and attempting to open other apps is extremely slow. However, DF itself does not appear to slow down - doing a bench with DFGui shows the same output level after a number of days.

Re-booting will cure the OS responsiveness for another day or so.
Without DF running, the OS remains stable and responsive for weeks. (I know, hard to believe for 98SE).

[IronBits]

Tyan Dual XP1900 running Manrake 8.2.
Both clients have been running the whole time since last switch over(IIRC).
One process was just shy of 130mb, the other was tad over 131mb.
Stopped and started both clients.
I checked a little while later, the memory used was down to 59mb
for each client.
I'm using the Intel compiled client, options -df -rt -g 0 -p -19.
Both windows always up where you can see them working.
Nothing else running (dedicated cruncher)...

[TheOtherPhil]

Similar findings here on my duallies using Linux or Windows. My XP box has both clients at ~100Mb after 77hours run time.

[ulv]

Same thing as IronBits on my dual machines using W2K

[Brian the Fist]

I tried running the code in Rational Purify over the long weekend, if made about 2500 structures with an upload interval of 1000. Results: no memory leaks. Sorry folks. Whatever it is, its not a leak, it is legit memory usage apparently. I'll take a look to see and figure out what it is. Is anyone noticing similar problems without the -rt switch? Note it is normal for the RAM usage to fluctuate +/- 10MB or so while running.

[KWSN_Millennium2001Guy]

Using the last protein, on Windows 2000 running the client as a service and the USERAM=1 switch, after 6 days the client is using 134 megs of RAM. When I start the service again it stabilizes originally around 69 megs. It slowly (over days) built up to the higher amount. Sounds like the classic definition of a memory leak to me. :)

Ni!

[markhl]

I, and several others, saw a leak after tens of uploads and a few hundred thousand structures.

Please run for that long to test it. I realize it runs slower in the Purify mode, so just run a regular client in -rt mode overnight or over a few days.

I appreciate that the RAM usage fluctuates while running, though I'd say nearer 1% i.e. 0.5 MB than 10 MB.

[runestar]

Hey Howard,

What about running with a higher upload interval? Might be possible the others are running higher intervals especially with this fast protein and the problem may lie within there.

I've been trying to watch it myself, and although sometimes I see a higher memory usage, I haven't seen anything as dramatic as 106MB with the current crop of proteins. Today I've seen it go up to 59MB although right its around 49MB. I HAVE noticed for example, that when it is at 59MB and I shut down and restart it, that when it settles down from its climb that it is lower, although occassionally I've seen it go right back to where it was before.

RS½

[Brian the Fist]

No memory leak means no memory leak. It doesnt matter if its one interval or one hundred. Zero bytes are leaked.
I have witnessed the increase in memory usage you describe, but it is not a leak as everything is correctly freed when the program terminates (according to Purify). Nevertheless, like I said, Ill take a look and see if I can establish what is using the extra memory to provide you with an explanation.

[Paratima]

Originally posted by Brian the Fist
but it is not a leak as everything is correctly freed when the program terminates (according to Purify). When the program terminates? :confused:

That's not my definition of a memory leak! We're talking about when the program keeps running and keeps grabbing more and more RAM for itself. That's a memory leak.

Every time it finishes a cycle, whether it sends results in to the server or just writes out to a result file, memory usage should be balanced at that time.

[runestar]

Hey Howard,

I was watching the memory usage last night seeing if I saw anything related "leak," and I don't if its related, but I noticed something rather odd with the client on Windows 2000.

When I shut down the client, and then restart it, the amount of free memory actually for a brief moment increases. So for example if 123MB of memory is supposedly in use, for maybe 2-3 seconds it will actually drop down to 120MB before it ramps back up.

Now I've watched a lot of things use memory before, way back from DOS... and I've seen things use MORE memory at first and then shrink back down, but I've never seen anything act like DF did on the memory...

Does anybody else see anything like this? Note, I tried this a few times and it did the same thing each time.

RS½

[wirthi]

HI,

I witness quite strange behaviour (Win XP Pro, newest Protein, using -rt swtich):

Usually the client uses about 80 MB of memory. Only just after minimizing the client-windows the memory usage drops down to a very low level (between 5 and 20 MB, but I have also seen 2 MB). After that the memory level rises again, within one or two minutes it reaches the normal level again. The folding itself continues without any interruption.

The value of the memory used I take from the "process" list of the task manager.

The behaviour is reproducable even on my other computer (win2k server, not using -rt)

Is it a bug or a feature :confused:

[runestar]

Originally posted by wirthi

Usually the client uses about 80 MB of memory. Only just after minimizing the client-windows the memory usage drops down to a very low level (between 5 and 20 MB, but I have also seen 2 MB). After that the memory level rises again, within one or two minutes it reaches the normal level again. The folding itself continues without any interruption.

I hadn't noticed it on minimizing to my recollection (feels like Oliver North, "I have no recollection of that, sir."). I did notice it on switching between Window and Full-Screen. I noticed the 80MB too though. It seems to peak at a little over 82,000KB ... but most of the time it sits around roughly 79-80MB.

For some very odd reason, Explorer memory usage actually jumped up a little. It actually went down when I opened a File Explorer window... Go figure Microsoft. The interesting thing though was any impact if any was negligble on the DF executable memory usage.

I noticed in the past with the large CASP5 protein that DF was pretty good at using the available pysical memory. If you had some applications open it would fall back in its memory usage, and when you closed down the applications it would climb back up.

It SEEMS to recover memory faster if its the foreground application, but I won't testify to that. ;) DF does seem to have a lot of little idiosyncratic quarks like that. Whether they are indeed quarks or bugs or things with Mickey$oft is hard to tell at times.

Best,

RS½

[pfb]

Originally posted by wirthi
HI,

I witness quite strange behaviour (Win XP Pro, newest Protein, using -rt swtich):

Usually the client uses about 80 MB of memory. Only just after minimizing the client-windows the memory usage drops down to a very low level (between 5 and 20 MB, but I have also seen 2 MB). After that the memory level rises again, within one or two minutes it reaches the normal level again. The folding itself continues without any interruption.

The value of the memory used I take from the "process" list of the task manager.

The behaviour is reproducable even on my other computer (win2k server, not using -rt)

Is it a bug or a feature :confused:

this is a feature of the NT (NT, 2K and XP) kernel - not DF...upon minimising a programme the kernel flushes out any memory areas it can and therefore the memory usage decrease for a minimised programme...

example - textpad under xp uses 5MB maximised, and only 900k minimised (try it with any other programme and most non-MS progs will exhibit this behaviour)...

[alpha]

Originally posted by pfb


this is a feature of the NT (NT, 2K and XP) kernel - not DF...upon minimising a programme the kernel flushes out any memory areas it can and therefore the memory usage decrease for a minimised programme...

example - textpad under xp uses 5MB maximised, and only 900k minimised (try it with any other programme and most non-MS progs will exhibit this behaviour)...

(sorry if this is a little offtopic)
Ah-ha. Ever since I started using XP Pro my machine stutters for a short while after I minimize an application. The HD light flashes in the rhythm of a ticking clock, causing the cursor to jump across the screen (amongst other things). Do you (or anyone) know how to turn this off, or make it more friendly, etc?

[pfb]

Originally posted by alpha


(sorry if this is a little offtopic)
Ah-ha. Ever since I started using XP Pro my machine stutters for a short while after I minimize an application. The HD light flashes in the rhythm of a ticking clock, causing the cursor to jump across the screen (amongst other things). Do you (or anyone) know how to turn this off, or make it more friendly, etc?

best way is to ensure that you have 512MB RAM - and ensure that the swapfile is 2.5x the RAM available and both minimum and maximum are set to this (ensures a fixed swap file)

[alpha]

Originally posted by pfb


best way is to ensure that you have 512MB RAM - and ensure that the swapfile is 2.5x the RAM available and both minimum and maximum are set to this (ensures a fixed swap file)

Bah. The machine in question has only 192MB but runs smooth as a bowling alley until I minimize something. The choppy behaviour subsides after a few seconds but it causes audio to stutter, which is annoying. I guess I'll have to just put up with it, I'm not forking out for more RAM.

[Halon50]

Is this fixed now? I went around and installed WinXP and IE6 service packs on Monday and Tuesday, so I don't have any tracking available earlier than yesterday. For the previous 24-hour period, though, it appears as if the latest executable doesn't grab more memory for every protein set that's uploaded. I'll keep an eye out and see if this holds true, but if it's fixed...

Thanks Howard !!! :thumbs:

[Welnic]

I have been watching the memory usage on the one XP machine that I have that uses -rt and I have not seen any constant increases in memory the way that I did with that one protein. I am not sure but I do not think that it was an issue with the one last week either.

On the whole this is a very smooth running project. Thanks, Howard. Good luck with the CASP predictions.

Clint

[Brian the Fist]

I can't take the credit this time :rotfl: as I didn't change anything.. I still say there never was any leak. Thats my story and Im sticking to it :crazy: I like it when problems fix themselves.

[Halon50]

I have an idea as to what may have caused the problem. In this current protein, we're using the old (pre-CASP) scoring system. I may be delusional, but I believe you (Howard) once posted something to the effect of the scoring being done on the client side?

Was it the experimental 60-40 system eating up memory while it ran?

[tpdooley]

Intermittent problems drive me nuts at work. (especially the heat related ones, since the basement I work in is cooler than our client's businesses.. ;)
They always come back until it repeats enough that we can pinpoint the cause..
Hopefully you'll have better luck tracking down whatever is holding onto more memory than it should when this starts happening again, Howard. :)

[Brian the Fist]

When weird things like this happen, personally, I like to blame them on Bill Gates. So lets do that and move on. :smoking:

[Paratima]

Be gentle with Bill G.

With the market slump (not, I repeat NOT a recession), I read where he's down about $11 billion.

Ya gotta wonder how he makes ends meet. ;)

[runestar]

Bill or MS? Between the two of them they got enough CASH reserves to outright buy just about anything on the market...

Mr Gates could on a whim goto down to the local grocery chain store and write them a check for their chain if he really wanted to. Then of course MS products would be used for the grocery chain operation unless the current system actually ran better than his... ;)

RS½

[markhl]

This issue seems to have been fixed. In fact, with the latest 109-residue protein, memory usage goes down with time!

Starts as usual with 1 MB / residue, 109 MB.
After a few hours or days, it goes down to 98-103 MB!

Howard, congratulations on the first reverse memory leak!
I think you need to move to the state of Washington and work with Microsoft products :-).

Mark