Quickie Benchmark Script for Linux Users

[Dyyryath]

I was curious how some of my systems compared at crunching DF so I wrote a really quick Perl script to give me some performance estimates.

It's based on real-world time not processor time, so it is affected by a system's non-DF load/usage. This is, however, what I was looking for when I wrote it. I want to know how much my machines are *really* doing, not what they are theoretically capable of.

With that in mind, my script checks the time the client was started (via it's lock file creation date/time), and then it's progress.txt file for how many structures it's completed since then. Given these two pieces of information, it calculates how many structures the system is averaging per second, minute, hour, and day.

I also had it pull some basic info about the system's kernel and processor.

The output looks like this:



------------------------------------------------------------
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 100984 structures over 145627 seconds.

Structures Per Second: 0.69
Structures Per Minute: 41.61
Structures Per Hour: 2496.39
Structures Per Day: 59913.36

Linux OS - Running Kernel Version 2.4.18-6mdk
AMD Athlon(tm) processor @ 1400mhz (256 KB cache)
------------------------------------------------------------


If you are interested in using this script, you can get it right here (http://www.free-dc.org/downloads/tools/df-linux-bench.pl).

To use it, just copy it to your Distributed Folding directory, and set it's permissions to 755 (or something similar) with 'chmod 755 df-linux-bench.pl'. Then just execute it in place.

[Dyyryath]

For comparison purposes, here's a P4 running at a similar speed to the Athlon above:



------------------------------------------------------------
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 451890 structures over 680377 seconds.

Structures Per Second: 0.66
Structures Per Minute: 39.85
Structures Per Hour: 2391.03
Structures Per Day: 57384.72

Linux OS - Running Kernel Version 2.4.8-26mdk
Intel(R) Pentium(R) 4 CPU 1500MHz @ 1483mhz (256 KB cache)
------------------------------------------------------------

[Dyyryath]

How about a dual P3-1000?

Processor 1:


------------------------------------------------------------
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 309002 structures over 679913 seconds.

Structures Per Second: 0.45
Structures Per Minute: 27.27
Structures Per Hour: 1636.10
Structures Per Day: 39266.40

Linux OS - Running Kernel Version 2.4.8-26mdksmp
Pentium III (Coppermine) @ 993mhz (256 KB cache)
------------------------------------------------------------

Processor 2:


------------------------------------------------------------
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 313240 structures over 679915 seconds.

Structures Per Second: 0.46
Structures Per Minute: 27.64
Structures Per Hour: 1658.54
Structures Per Day: 39804.96

Linux OS - Running Kernel Version 2.4.8-26mdksmp
Pentium III (Coppermine) @ 993mhz (256 KB cache)
------------------------------------------------------------

[Dyyryath]

My quick hack is working out better than expected, so I'm making a Windows version right now. As soon as it's ready, I'll have some scores for P4-2000s and XP2000s....

[Digital Parasite]

Originally posted by Dyyryath
My quick hack is working out better than expected, so I'm making a Windows version right now. As soon as it's ready, I'll have some scores for P4-2000s and XP2000s....

If you haven't finished already, I just uploaded a Windows version if you want to use that:
http://www.free-dc.org/forum/showthread.php3?s=&threadid=856

[Richard Clyne]

Okay if I chuck in some results I have been gathering over the last couple of weeks.

[Paratima]

Neat proggie, Dyy. See what a difference the bigger cache makes on an Athlon Classic.
------------------------------------------------------------
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 2400 structures over 5051 seconds.

Structures Per Second: 0.48
Structures Per Minute: 28.51
Structures Per Hour: 1710.55
Structures Per Day: 41053.20

Linux OS - Running Kernel Version 2.4.10-4GB
AMD Athlon(tm) Processor @ 848mhz (512 KB cache)
------------------------------------------------------------

[pointwood]

Very cool little program!

[Scoofy12]

You mentioned the cache?
------------------------------------------------------------
Distributed Folding Windows Benchmark Script V1.0

Sample Size: 43180 structures over 70293 seconds.

Structures Per Second: 0.61
Structures Per Minute: 36.86
Structures Per Hour: 2211.43
Structures Per Day: 53074.32
------------------------------------------------------------
That's a Duron @ 938MHz (64K cache) and Win XP

[Scoofy12]

Originally posted by Richard Clyne
Okay if I chuck in some results I have been gathering over the last couple of weeks.

You don't REALLY mean a SIMM, do you? (ie 72-pin single inline memory module, circa 486/p1-era?) do you mean just SDR or SDRAM?

[Richard Clyne]

DO, your right scoofy12:o

Must have lapsed in to a time warp:D

[Dyyryath]

Just a couple more benchmarks. This one is from a Windows XP box running an XP1800+ on an Iwill KK266 with PC133 SDRAM:



------------------------------------------------------------
Distributed Folding Windows Benchmark.NET App V1.0

Sample Size: 3710 structures over 4951 seconds.

Structures Per Second 0.75
Structures Per Minute 44.96
Structures Per Hour 2697.64
Structures Per Day 64743.28

Operating System is MS Windows XP
AMD Athlon(tm) XP
------------------------------------------------------------

[Dyyryath]

Here's another. This one is a newly put together XP1900+ running on an Abit AT7 (I love this board so far) with PC2700 DDR RAM:



------------------------------------------------------------
Distributed Folding Windows Benchmark.NET App V1.0

Sample Size: 1085 structures over 1226 seconds.

Structures Per Second 0.88
Structures Per Minute 53.10
Structures Per Hour 3185.97
Structures Per Day 76463.30

Operating System is MS Windows XP
AMD Athlon(tm) XP 1900+
------------------------------------------------------------


This one has a smaller sample size than I usually run since I just put it together and started it crunching.

Compared to the one above, there is only a small difference in actual processor speed (about 66mhz), yet there is a huge difference in production. This is almost certainly due to the different memory types. This board uses the new KT333 chipset and Corsair's fastest DDR RAM. It appears to be worth the extra money. ;)

[Dyyryath]

BTW, as you might have noticed from the output, these numbers were generated by a different version of my little app that runs under Windows instead of Linux.

This one is NOT written in Perl, but rather in C#. As such, it requires that you have the .NET Framework (which is a bit like a JVM) installed on your machine to run it. You can install the .NET Framework from this location (http://www.windowsupdate.com).

If anybody wants to experiment with my Windows version, let me know and I'll post it for download. It's only about an 8k single executable.

[Terminator]

I'd be interested in the Windows version to check my P4 with RDRAM :)

Very cool little apps :thumbs:

[pointwood]

Here is a little benchmark from me - this is currently all I got that is running the DF client - impressive isn't it :p :D


------------------------------------------------------------
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 49915 structures over 181136 seconds.

Structures Per Second: 0.28
Structures Per Minute: 16.53
Structures Per Hour: 992.04
Structures Per Day: 23808.96

Linux OS - Running Kernel Version 2.4.19-r1
Celeron (Mendocino) @ 416mhz (128 KB cache)
------------------------------------------------------------

[TheOtherPhil]

Ok, here's my farms results:


MENACE:
Dual XP1800's, 512Mb PC2100, Win2K Pro



CPU0
Sample Size: 57600 structures over 74284 seconds.

Structures Per Second: 0.78
Structures Per Minute: 46.52
Structures Per Hour: 2791.45
Structures Per Day: 66994.80

CPU1
Sample Size: 57700 structures over 74455 seconds.

Structures Per Second: 0.77
Structures Per Minute: 46.50
Structures Per Hour: 2789.87
Structures Per Day: 66956.88

TIGER:
Dual XP1800's, 1024Mb PC2100, Win2K Server


CPU0
Sample Size: 59235 structures over 78320 seconds.

Structures Per Second: 0.76
Structures Per Minute: 45.38
Structures Per Hour: 2722.75
Structures Per Day: 65346.00

CPU1
Sample Size: 58385 structures over 78414 seconds.

Structures Per Second: 0.74
Structures Per Minute: 44.67
Structures Per Hour: 2680.47
Structures Per Day: 64331.28

TIGER2:
Dual MP1200's, 256Mb PC2100, Gentoo Linux


CPU0

Cannot connect to network at this time, will bench later

CPU1

Cannot connect to network at this time, will bench later

TIGER3:
Dual XP1800's, 512Mb PC2100, Win2K Server


CPU0
Sample Size: 55450 structures over 73348 seconds.

Structures Per Second: 0.76
Structures Per Minute: 45.36
Structures Per Hour: 2721.55
Structures Per Day: 65317.20

CPU1
Sample Size: 55450 structures over 73348 seconds.

Structures Per Second: 0.76
Structures Per Minute: 45.36
Structures Per Hour: 2721.55
Structures Per Day: 65317.20


Going on the above results, I would expect my daily output to be approx 495K.

[Dyyryath]

Damn, Phil! That's a pretty sweet set of boxes you've got going there! :thumbs: :thumbs:

[TheOtherPhil]

Originally posted by Dyyryath
Damn, Phil! That's a pretty sweet set of boxes you've got going there! :thumbs: :thumbs:


Thanks Dyyryath :D I had genomers disease pretty bad, but Stanford cured that recently :rolleyes:

[Dyyryath]

Thanks Dyyryath I had genomers disease pretty bad, but Stanford cured that recently

I can imagine. :(

Welcome to Distributed Folding, though! :thumbs:

[Dyyryath]

Here's an update on my XP1900+ -w- PC2700 DDR RAM now that it's been running awhile. The earlier bench with a small sample size was actually pretty close:



------------------------------------------------------------
Distributed Folding Windows Benchmark.NET App V1.0

Sample Size: 27270 structures over 29827 seconds.

Structures Per Second 0.91
Structures Per Minute 54.86
Structures Per Hour 3291.38
Structures Per Day 78993.13

Operating System is MS Windows XP
AMD Athlon(tm) XP 1900+
------------------------------------------------------------


I'll be ordering another of these, only with an XP2100+ first thing Monday. :D

[Dyyryath]

OK, here's a copy of the Windows version of my script. Remember, you must have the .NET Framework installed to use it. You can get the .NET Framework from the Windows Update site.

Additionally, this little app is about 10 minutes worth of programming, so don't expect any exception handling or anything. If the client isn't running, or you've got it in the wrong directory or something, the benchmark won't complain gracefully, it'll just die. ;)

I've been toying with the idea of doing something a little more complex with a GUI front end, but I'm not sure I'll bother. In the meantime, here's the current version for Windows:

[Paratima]

I'll be ordering another of these, only with an XP2100+ first thing Monday.
:scared: No need to get carried away!

[Dyyryath]

:scared: No need to get carried away!
LOL! :D :rotfl: :D

[Darkness Productions]

Ok, just so you guys can feel speshul....

This is my work box, p3 500 Katmai (512K L2) 128MB RAM and Win2k Pro....

------------------------------------------------------------
Distributed Folding Windows Benchmark Script V1.0

Sample Size: 40 structures over 346 seconds.

Structures Per Second: 0.12
Structures Per Minute: 6.94
Structures Per Hour: 416.18
Structures Per Day: 9988.32
------------------------------------------------------------

That's a bit early into it, so I'm letting the client run and get some stuff done so that I can get a better benchmark. Unfortunately, I'm not gonna be coming back to DF anytime soon, only because TLC still needs the help pretty badly.

Here's the next update from the script:

------------------------------------------------------------
Distributed Folding Windows Benchmark Script V1.0

Sample Size: 75 structures over 540 seconds.

Structures Per Second: 0.14
Structures Per Minute: 8.33
Structures Per Hour: 500.00
Structures Per Day: 12000.00
------------------------------------------------------------

Hmm, when it get's to 150 structures, I'll paste that in here too.....

------------------------------------------------------------
Distributed Folding Windows Benchmark Script V1.0

Sample Size: 150 structures over 836 seconds.

Structures Per Second: 0.18
Structures Per Minute: 10.77
Structures Per Hour: 645.93
Structures Per Day: 15502.32
------------------------------------------------------------
See, those numbers make me wonder how well it would do with the output turned off. The first few structures are here. I will continue as above with the output off.


------------------------------------------------------------
Distributed Folding Windows Benchmark Script V1.0

Sample Size: 40 structures over 147 seconds.

Structures Per Second: 0.27
Structures Per Minute: 16.33
Structures Per Hour: 979.59
Structures Per Day: 23510.16
------------------------------------------------------------

Wow..... that's a pretty good jump. Now, while I don't think that it will matter as much with the XP1800+ style boxen, those of you with the slower boxen, take note.

------------------------------------------------------------
Distributed Folding Windows Benchmark Script V1.0

Sample Size: 70 structures over 246 seconds.

Structures Per Second: 0.28
Structures Per Minute: 17.07
Structures Per Hour: 1024.39
Structures Per Day: 24585.36
------------------------------------------------------------

Hmm.... looks almost 50% faster just by loosing the pretty print. I wonder how much of a difference (likely negligable(?)) it would make on a higher-end processor....

------------------------------------------------------------
Distributed Folding Windows Benchmark Script V1.0

Sample Size: 150 structures over 533 seconds.

Structures Per Second: 0.28
Structures Per Minute: 16.89
Structures Per Hour: 1013.13
Structures Per Day: 24315.12
------------------------------------------------------------


Chew on that one for a while......

[MAD-ness]

Grabbed the .NET framework from WindowsUpdate on this WindowsXP system last night. Just ran the benchmark script and it worked fine.

However, I unzipped it to the wrong directory first (not my DF directory) and before I moved it I decided to try it anyways and got an unhandled error, yadda, yadda.

Pretty to be expected for something with no error handling, etc. I assume you are expecting that to cause problems so I won't bother including any info. It isn't a PROBLEM for me, just giving feedback, though like I said, I doubt that the result was unexpected.

Good work Dyyryath.

------------------------------------------------------------
Distributed Folding Windows Benchmark.NET App V1.0

Sample Size: 3100 structures over 5396 seconds.

Structures Per Second 0.57
Structures Per Minute 34.47
Structures Per Hour 2068.20
Structures Per Day 49636.77

Operating System is MS Windows XP
AMD Athlon(tm) Processor
------------------------------------------------------------

This is an AMD Thunderbird 1.2ghz (100mhz fsb) CPU.

[Dyyryath]

Yeah, I wrote it in a hurry as I was leaving the office one afternoon so I didn't add any bells & whistles or exception handling.

I've got to grab the source form the office tomorrow so I can give Digital Parasite a copy. When I do, I'll probably take a few minutes and add some handlers for the exceptions.

I'm also going to tweak the info gathering routine just a smidge so that it always shows the processor speed in MHz as well.

That's the piece Jeff wants to see anyway...

[MAD-ness]

Yeah, I wasn't complaining, just noting.

If Jeff integrated the info gathering stuff that would rock. I know I was the one bugging him to do so.

You both rock. /me wants to grow up and be like Dyy and Jeff.

BTW, mind if I ask what you are using to do stats? I talked with Dr. Nno who does the TVM stats and he uses a combination of mySQL and PHP. I have been looking into this a lot the last few weeks, it seems like this would be the route to go. You could do it with ASP and you could probably mix and match databases, but starting from scratch using PHP and SQL looks like the best route that I can find.

Any general pointers or info you feel comfortable sharing?

Also, when you update the stats do you render them statically and then only update them every hour? Seems like this might take longer at each update but would be a lot faster in between? Not sure if I am asking this properly.

[Chinasaur]

My Tyan S2460's - 2X2 - XP1800's

Struc per Second: 0.76
Struc per min: 45.69
Struc Per Hour: 2741.48
Struc per day: 65795.52

Linux OS Kernel 2.4.9-31smp
AMD Athlon(tm) Processor @1533mhz (256 KB cache)

RedHat 7.2 and SuSE 7.3


Thanks for the script Dyyryath. :thumbs:


:cheers:

[Digital Parasite]

------------------------------------------------------------
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 17835 structures over 39501 seconds.

Structures Per Second: 0.45
Structures Per Minute: 27.09
Structures Per Hour: 1625.43
Structures Per Day: 39010.32

Linux OS - Running Kernel Version 2.4.17
Celeron (Coppermine) @ 908mhz (128 KB cache)
------------------------------------------------------------

Which is pretty sweet since my PIII 800MHz in WinXP only does 34595 structures per day.

[MAD-ness]

Do you think this is the Linux performance boost, or does the Celeron coppermine just perform better than expected in relation to a P3? The few results that I have seen indicate the opposite. :(

@908 = what fsb?

[DuffMan|TSF|]

Ok, i was talking IRC with Pointwood, and he said the stats were really good and to post them.

------------------------------------------------------------
Distributed Folding Windows Benchmark Script V1.0
Sample Size: 1220 structures over 1220 seconds.
Structures Per Second: 1.00
Structures Per Minute: 60.00
Structures Per Hour: 3600.00
Structures Per Day: 86400.00
------------------------------------------------------------

dfGUI gives the same numbers

Machine stats:
Cpu: AthlonXP 1.47@1.6
Mem: 256meg corsair at 146MHz, cas 2
mobo: Soyo dragon+

When ECCp finishes I will back thses numbers up. I was just reserving my name because i wanted to run distribfolding after ECCp ended.

[Dyyryath]

He's right, those are GREAT numbers. It really goes to show what a high FSB and good memory bandwidth do for the DF client! :thumbs:

[pointwood]

Doesn't the latest P4's have a faster FSB than the previous ones?

Maybe those will be pretty good a DF then?