Client pausing on calculating energy

[Bananeweizen]

Hi all !

I only wanted to report that the new client is pausing (i.e. running with about 0 to 2 percent CPU usage) for some seconds whenever it displays "calculating enery". My guess is that this may happen because of some file access at that moment. I run the client via a network drive (using options df rt if g0) and that would explain this delay.
Other members of my team experienced this behavior too.

Ciao, Michael.

[Brian the Fist]

This is an intentional pause.
(Kinda like This page intentionally left blank)

[Norman]

why ? why Howard ? I dont know why ?

that is a not optimized cpu usage and I think the lost cpu-power is most unnecessary !

:bang:

[Roman]

I have the same problem, but wanted to add that there's definitely NO disk access during these pauses(I run it from the local disk).

Here's the CPU usage graph:
http://distributed.org.ru/pics/df/calc-energy.png

I also would like to ask, WHY this "intentional pause" was added?

[FoBoT]

just run another lower priority DC client and it will smooth that graph right out

[Paratima]

The pause is most likely for the display.

...WTF are you doing running to the display, anyhow? :gone:

You want fast, run the text client with -qt. No display, no pause, smooooooth graph! :p

[Bananeweizen]

Originally posted by Paratima
You want fast, run the text client with -qt. No display, no pause, smooooooth graph! :p

Nope. Next time please test your assumptions before giving such an advice.
I do have those pauses with the no display configuration.

Ciao, Michael.,

[Paratima]

Sir, that assumption is tested daily on about 15 boxen (it varies), with graphing at the highest available refresh rate! Makes smoooooth graphs.

Also let me point out the original post.
I only wanted to report that the new client is pausing (i.e. running with about 0 to 2 percent CPU usage) for some seconds whenever it displays "calculating enery".This indicates not "-qt", yes?

Bananeweizen, what CPU / OS gives these lumpy graphs?

[Roman]

Though I'm not Bananeweizen, I got such graph on Duron 1100, 256 DDR, running Windows XP.
I see it both when I run the client with and without -qt.

I don't think the issue is specific to some CPU/OS combination: Howard already confirmed that there are intentional pauses programmed in the client, now we just should wait and he will (hopefully)tell why they are needed.

[Brian the Fist]

First of all, the pause is 5 seconds per generation, so don't tell me valuable resources are being wasted.

Now if you MUST know, the pause is necessary to ensure proper inter-thread communications, especially on faster machines. If the pause was not there, there would be a danger of entering an endless loop - would you prefer that? :bonk: :haddock:

[Bananeweizen]

Originally posted by Paratima
Also let me point out the original post. This indicates not "-qt", yes?

Yes. But where did I say that the computers mentioned in those posts are the same? :)


Bananeweizen, what CPU / OS gives these lumpy graphs? [/B] W2K (German) on a P4@1650. Don't think there is anything special in that configuration.

Ciao, Michael.

[Paratima]

Originally posted by Brian the Fist
First of all, the pause is 5 seconds per generation, so don't tell me valuable resources are being wasted. Aha! So I was looking at the graph WAY too fast! Makes sense now. 5 seconds per gen? Not a problem. :sleepy:

[Roman]

First of all, the pause is 5 seconds per generation, so don't tell me valuable resources are being wasted.
This doesn't seem to be correct in my case.
http://distributed.org.ru/pics/df/calc-energy2.png
This graph was taken using maximum possible refresh-rate, where the CPU load values are recorded every 0.5 seconds, and vertical lines are drawn every ~3 seconds. If I counted them right, there's 24 vertical lines, so it represents about 72 seconds. :)
As you can see, there has been four pauses of about 4 seconds each, so the system spent 16 seconds of 72 in idle state - more than 20% of the time!


Now if you MUST know, the pause is necessary to ensure proper inter-thread communications, especially on faster machines. If the pause was not there, there would be a danger of entering an endless loop - would you prefer that?
Why there's a need to do inter-thread communications while the client runs only one thread?
Why not use other, more common and less wasteful means to do it - e.g. OS objects(semaphores, mutexes, atoms..), lock-files or global variables?
Why there hasn't been any pauses in an earlier release of the client which I used a couple of months ago?

[Brian the Fist]

Originally posted by Roman
This doesn't seem to be correct in my case.
Why there's a need to do inter-thread communications while the client runs only one thread?
Why not use other, more common and less wasteful means to do it - e.g. OS objects(semaphores, mutexes, atoms..), lock-files or global variables?

I do not know where you are getting this graph from, and as I scientist, I have to be highly skeptical of any graph with no legend, units, or axis labels. The client uses more than one thread at times (obviously, or I wquldnt have said so). Again, 5 seconds/generation is not considered wasteful by me though perhaps a mutex could be used to accomplish the same goal here, however like I said it is there for a godo reason. If you are seeing pauses at times other than at the end of energy minimization (at the end of each generation) then they are caused by something else. Even if the CPU is not 100% active it does not mean nothing is happening, perhaps the disk is being accessed or something. Basically what you are seeing could be caused by just about anything and you should not worry about it.

[bwkaz]

Notice how, on that graph, the kernel time (I assume the red line is the kernel time, though I have no evidence whatsoever to support this...) drops to zero right about the same time that the userspace time drops to zero?

I bet that's related. What kind of box is this, desktop, laptop, something else? Is ACPI on? If so, could the OS possibly be turning the CPU usage down to cool it off maybe?

I've never seen anything like this on Linux (short of that 1-second kernel-time CPU usage increase I was talking about a while ago). I therefore wonder if it's OS related.

[rsbriggs]

Perhaps the first thing to find out is, what program is producing those graphs?

[bwkaz]

Looks vaguely like a Russian (or some other Cyrillic-alphabet) version of Windows' Task Manager's third tab, maybe...

[rsbriggs]

My version of task manager doesn't produce graphs like that...

[AMD_is_logical]

Perhaps there's something on that system which is using CPU time, but which is cloaking that usage so that it doesn't show up on the graph.

[FoBoT]

Originally posted by rsbriggs
My version of task manager doesn't produce graphs like that...

its the performance graph/tab from winXP , there is an option to include the kernal , view-show kernal times, that is the red part

i have only seen this type of behavior when first starting it up, and perhaps only during gen 0 , i would have to look at some boxen

[Roman]

I do not know where you are getting this graph from
This graph is from the third tab of Windows XP Task Manager(taskmgr.exe). This program allows to select from three different refresh rates, and at the highest rate "...the CPU load values are recorded every 0.5 seconds, and vertical lines are drawn every ~3 seconds."(yes, Microsoft provided no legend or units, so I measured this with a stopwatch).


If you are seeing pauses at times other than at the end of energy minimization (at the end of each generation) then they are caused by something else.
Client pauses only when the yellow label "CALCULATING ENERGY" appears in its window. At the time I was taking the graph, this label appeared 4 times over 72 seconds. There's no disk or network access during the pauses, the machine is a desktop, ACPI is off. Other distributed computing clients(Folding@home, Seventeen or Bust, ECC2-109, SETI@home...) always use 100% of CPU time.
Finally, as FoBoT said, the pauses seem to happen so often only when calculating "gen. 0". After that, pauses happen only when a generation finishes, so this is not such a significant problem.

[Brian the Fist]

Ok, you forgot to mention the gen. 0 part :D The pauses I was referring to occur at the END of each generation. Now that I know what you are talking about Ill take a look and tell you what these pauses are shortly.

[Brian the Fist]

That pause was indeed unnecessary as there as not multiple threads at this time. I fixed it so the pause only occurs at the end of each generation when minimizing energy and not during gen. 0. It will be fixed in the next release. Thanks for pointing this out. :thumbs:

[FoBoT]

yay!! communication with the project dude works!!! :elephant:

DF ROCKS ! ! :drums:

[HaloJones]

That kind of response is why I do DF and not other DC projects.

Thanks Howard. I look forward to the next release. At present, the first 10000 structures take an inordinate amount of time and you have just helped reduce that time.

:notworthy

[Paratima]

Yes! Not mentioning the Gen-0 part was rather misleading.

"Oh and by the way, it only happens in Gen 0, which is a small percentage of the total processing time..."

Yay team, anyhow, I guess. :looney: