Linux Benchmark script

[TheOtherPhil]

Hi there, I remember that Dyyryath posted a benchmark script a while ago for Linux. I have looked in that thread but the attachment isn't there anymore. I'd appreciate it if somebody who has a copy could post it. I'd like to see just how RH8 compares to WinXP on identical hardware (dual XP1800's).

Thanks,
Phil.

[Paratima]

#!/usr/bin/perl
#
# This PERL script should be run in the same directory as your Distributed
# Folding files. The script can only be run while the DF client is running.
#
use strict;
system("cls");

if (!(-e "foldtrajlite.lock"))
{
print "ERROR: foldtrajlite.lock file not found! Client must be running...\n";
exit(-1);
}

my $seconds = (time() - (stat("foldtrajlite.lock"))[9]);

open(IN,"<progress.txt");
<IN>=~/(\d+)/;
my $progress = $1;
close(IN);

my $workPerSecond = sprintf("%.2f",($progress/$seconds));
my $workPerMinute = sprintf("%.2f",($progress/($seconds/60)));
my $workPerHour = sprintf("%.2f",($progress/($seconds/3600)));
my $workPerDay = sprintf("%.2f",($workPerHour*24));

print "-" x 60 . "\n";
print "Distributed Folding Windows Benchmark Script V1.0\n\n";

print "Sample Size: $progress structures over $seconds seconds.\n\n";

print "Structures Per Second:\t\t\t$workPerSecond\n";
print "Structures Per Minute:\t\t\t$workPerMinute\n";
print "Structures Per Hour:\t\t\t$workPerHour\n";
print "Structures Per Day:\t\t\t$workPerDay\n";

print "-" x 60 . "\n";

[TheOtherPhil]

Thanks, it works great. I am seeing a good difference in speed by using Linux :shocked:

CPU0
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 4356 structures over 1148 seconds.

Structures Per Second:                  3.79
Structures Per Minute:                  227.67
Structures Per Hour:                    13659.93
Structures Per Day:                     327838.32

CPU1
Distributed Folding Linux Benchmark Script V1.0

Sample Size: 4158 structures over 1097 seconds.

Structures Per Second:                  3.79
Structures Per Minute:                  227.42
Structures Per Hour:                    13645.21
Structures Per Day:                     327485.04

Dual XP1800's, Tyan Tiger S2460, 768Mb....Win2K results to follow.

[TheOtherPhil]

OK, using the exact same hardware as the Linux box but with Win2K Pro I get:

CPU0
Distributed Folding Windows dfGUI v1.7 Benchmark

Sample Size : 5200 structures over 1537 seconds.
Protein Size: 108AA

Structures Per Second: 3.38
Structures Per Minute: 203.0
Structures Per Hour  : 12180
Structures Per Day   : 292310

OS : Microsoft Windows 2000    MHz: 1533
CPU: AMD Athlon(tm) MP 1800+

CPU1
Distributed Folding Windows dfGUI v1.7 Benchmark

Sample Size : 4950 structures over 1534 seconds.
Protein Size: 108AA

Structures Per Second: 3.23
Structures Per Minute: 193.6
Structures Per Hour  : 11617
Structures Per Day   : 278801

OS : Microsoft Windows 2000    MHz: 1533
CPU: AMD Athlon(tm) MP 1800+

Over both CPU's, that is an approximate 84,000 structure per day advantage just by running Linux on that one box.

/me looks at my 5 dual AMD boxen and ponders whether to swap them all to Linux

[Paratima]

There's a question here?

[dnar]

Phil, did you ever doubt this?

[TheOtherPhil]

Originally posted by dnar
Phil, did you ever doubt this?

LOL, no but I didn't expect such a difference

[IronBits]

Does this script still work correctly for Phase II ?

[magnav0x]

I sure hope so. I want to test out this gentoo performance! Spent all weekend getting it set up the way I want.

[ECL]

I've been looking into the difference between the perl script and the built-in benchmarking tool (.\foldtrajlite.exe -bench). The built-in feature should be more objective, since it'll theoretically be more representative of actual folding performance after generation 0, and it doesn't depend on flags or other tweaks.

The output gives User and System timings in seconds. "Foldtraj" is supposedly more important than "Maketraj" but the "Maketraj" numbers are large enough to be non-trivial.

Here are sample timings from two of my systems:

Usr time Sys time
-------- --------
Maketraj 10.545 1.152
Foldtraj 38.495 22.032 (P4 2.4 / WinNT4)
Maketraj 8.750 1.359
Foldtraj 48.609 9.984 (XP 2400 / WinXP)

According to this, I'm likely to get approximately 1 structure every 70 seconds out of either box. According to Howard's readme, that's a baseline figure and could be comparable across machines.

In the real world, I'm using dfGui and have "Use extra RAM" and "Quiet Mode" checked, and my actual numbers are somewhat different. Using the time graph, a calculator and an ancient technology known as a "pencil" I have figured that my real average time per generation on the XP2400 is ~11 minutes, or 13.2 seconds per structure. Using the benchmark.txt output by dfGui, I find that my average time on the current gen is 22.96 seconds per structure (25 structures over 574 seconds).

I'd like to see some other Foldtraj/Maketraj timings from others on the team. There might turn out to be interesting CPU vs OS patterns in the data. Or not.

[Dyyryath]

Originally posted by IronBits
Does this script still work correctly for Phase II ?

Probably not. With the different generations, it'd be easy to come up with extremely skewed results. I'd use the built in one now days.

magnav0x: Welcome to the light, brother! Can I get an 'emerge!'?

[Paratima]

With a bit of fussing around (which I'm not interested in doing just now), you could get that script to give you good numbers during Generation 0 only. The problem is that you don't get a very large sample that way.

When we were using this script for a benchmark, we encouraged users to run it for at least a couple of hours, and preferably a day or longer. That way you would smooth over the occasional peaks and valleys of the algorithm and actually get representative performance numbers. Today, Gen 0 doesn't last long enough on the faster machines most of us use and the results would be skewed.

Sorry! dfGUI forever!!

ECL: There's a thread for this over here.

[magnav0x]

magnav0x: Welcome to the light, brother! Can I get an 'emerge!'?

OMG, I couldn't believe this emerge thing. It's friggin awesome. All my other linux distros are now coasters! I've been messing with the framebuffering and bootsplash all day today. Seems to be a problem with the 2.4.20-gentoo-r5 gentoo sources, but I think I've got it all worked out after some kernel patching. I didn't want to switch to the vanilla or ck sources.

[Kosh]

Originally posted by magnav0x
OMG, I couldn't believe this emerge thing. It's friggin awesome. All my other linux distros are now coasters! I've been messing with the framebuffering and bootsplash all day today. Seems to be a problem with the 2.4.20-gentoo-r5 gentoo sources, but I think I've got it all worked out after some kernel patching. I didn't want to switch to the vanilla or ck sources.

Good to hear ... if you are feeling really adventurous try development-sources or mm-sources. And welcome to Gentoo

[magnav0x]

I just may. I'm still working on getting everything in that I want to use. Gentoo is blowing my mind!