Call for Benchmarks

[ddn]

Requesting benchmarks:

Please submit them in this thread in this format:
Manufacturer,Processor Type,CPU Speed in Mhz,OS,Maketrj usr, Maketrj sys,Foldtr\
aj usr, Foldtraj sys

ie:
Intel,Xeon,2600,RedHat Linux 9.0,2.710,0.820,24.580,11.740

[gistech1978]

AMD, XP 2100+, 2000, Windows XP Pro. SP1, 7.609, 0.359, 36.625, 6.891

[hallmar]

AMD
xp2500+ Barton
2367mhz
XP Pro sp1
6.125
.344
31.516
6.000

[BuddhaMan]

Remember this is an "apples to oranges" comparison as the bench is done on the current generation of the client.

Suggestion: Someone post a zip file for Windows and a Linux tar.gz of the client install directory in it's entirety so we all bench using the same data. Include a .bat file for Windows and bash script for Linux to run the client quiet, with large mem and no net access.

[wirthi]

Originally posted by BuddhaMan
Remember this is an "apples to oranges" comparison as the bench is done on the current generation of the client.

Suggestion: Someone post a zip file for Windows and a Linux tar.gz of the client install directory in it's entirety so we all bench using the same data. Include a .bat file for Windows and bash script for Linux to run the client quiet, with large mem and no net access.

It should be enough to delete your filelist.txt for that. So, copy your client to another directory, delete its filelist, run the benchmark, delete this benchmark directory.

Doesn't the built in benchmark ignore witch generation you are at?

[Brian the Fist]

The benchmark is always the same test and is independent of the protein being folded or client version. The only thing it depends on is hardware and the compiler and compiler flags we used to build it. (And occasionally, very low-level changes to the folding algorithm could affectthe timing slightly).

[BuddhaMan]

Ok, I sit corrected then. I thought I had read somewhere what I wrote above.

With that being said, here's my results:


AMD, Athlon Thunderbird,1400,Win2K-SP4,12.127,1.052,54.208,19.198

Intel, Pentium 3,900,WinXP-SP1,18.547,0.911,118.100,22.763

Intel,Celeron,550,Win2K-SP4,26.422,2.031,121.422,47.344

[derek]

Just an fyi to the OSX client maintainer : bench crashes on the OSX client under 10.2.6 (both server and the normal jag distribution). It crashes on an iBook 800MHz, dual Xserves (1.3GHz), and a 12" power book.

- derek

[dano]

Intel,P4,1800,windows2000,14.578,0.875,60.750,23.031

Intel,P4,1800,Mandrake 9.1,4.860,1.240,49.550,20.330

Linux seems to be quite a bit faster on the same hardware.

AMD,xp 2600+,2130,mandrake 9.0,4.180,0.840,41.010,14.210

[Brian the Fist]

Originally posted by derek
Just an fyi to the OSX client maintainer : bench crashes on the OSX client under 10.2.6 (both server and the normal jag distribution). It crashes on an iBook 800MHz, dual Xserves (1.3GHz), and a 12" power book.

- derek

hmm, I get a message in the error.log 'foldtraj returned an error 3' - is that what you are talking about?

[derek]

Originally posted by Brian the Fist
hmm, I get a message in the error.log 'foldtraj returned an error 3' - is that what you are talking about?

Yup it sure is, I should have posted the error.log. Sorry.

- derek

[ddn]

Sun,UltraSparcIIi 500,500,Solaris 8,15.840,1.550,225.600,5.070
Intel,Celeron 550,550,RedHat Linux 9.0,15.800,3.490,140.450,66.520
Intel,Celeron 550,550,Windows 2000 SP4,26.422,2.031,121.422,47.344
Sun,UltraSparc III 900,900,Solaris 9, 7.530,1.070,121.990,3.270
Intel,Pentium 3,900,Windows XP SP1,18.547,0.911,118.100,22.763
AMD,Athlon 1050,1050,RedHat Linux 9.0,8.530,0.860,93.400,16.570
AMD,Athlon XP 1200,1200,RedHat Linux 9.0,7.620,0.700,86.870,15.010
AMD,Athlon Thunderbird 1400,1400,Windows 2000 SP4,12.127,1.052,54.208,19.198
AMD,Athlon 1400,1400,RedHat Linux 9.0,7.140,0.680,78.900,12.220
Intel,Celeron 1700,1700,RedHat Linux 9.0,6.860,1.220,48.710,21.280
Intel,Pentium 4 1800,1800,Windows 2000,14.578,0.875,60.750,23.031
Intel,Pentium 4 1800,1800,Mandrake 9.1,4.860,1.240,49.550,20.330
AMD,Athlon XP 2000,2000,Windows XP Pro. SP1,7.609,0.359,36.625,6.891
AMD,Athlon XP 2130,2130,Mandrake 9.0,4.180,0.840,41.010,14.210
AMD,Athlon XP 2367,2367,Windows XP Pro SP1,6.125,.344,31.516,6.000
Intel,Pentium 4 2400,2400,RedHat Linux 9.0,3.210,0.800,29.290,11.920
Intel,Xeon 2600,2600,RedHat Linux 9.0,2.710,0.820,24.580,11.740



Linux is much faster for the same hardware.

I get the same error on the OSX client.

[Grumpy]

That P4 2400 score seems a bit too good to be right

Barton 2233 Mhz Win2000 6.297, 0.172, 33.016, 6.609

[ddn]

[root@mail distribfold]# cat /proc/cpuinfo | grep "model name" && ./foldtrajlite -bench
model name : Intel(R) Pentium(R) 4 CPU 2.40GHz
One moment, opening rotamer library...
Predicting secondary structure and generating trajectory distribution...
Folding protein...
Benchmark complete.

Summary
-------
Usr time Sys time
-------- --------
Maketrj 3.180 0.780
Foldtraj 29.710 11.640

[Grumpy]

Is that a 800 FSB 2.4 ? Red Hat 9.0 Rox..Whatever you have done to that P4, leave it be

[erk]

Originally posted by derek
Just an fyi to the OSX client maintainer : bench crashes on the OSX client under 10.2.6 (both server and the normal jag distribution). It crashes on an iBook 800MHz, dual Xserves (1.3GHz), and a 12" power book.

- derek

I get the same -bench problem under 10.2.6.

ERROR: [001.001] {foldtrajlite2.c, line 5053} Foldtraj returned an error 3

[HaloJones]

One moment, opening rotamer library...
Predicting secondary structure and generating trajectory distribution...
Folding protein...
Benchmark complete.

Summary
-------
Usr time Sys time
-------- --------
Maketrj 6.203 0.313
Foldtraj 29.891 6.031

T-Bred @ 2400MHz, WinXP

[pointwood]

P4 2ghz

Summary
-------
          Usr time  Sys time
          --------  --------
Maketrj     13.547     2.141
Foldtraj    58.766    19.516

[Morphy375]

XP2700+ (non oc)

Summary
-------
Usr time Sys time
-------- --------
Maketrj 7.984 0.516
Foldtraj 42.344 6.594

[rsbriggs]

The P4 2.4 numbers above seem a little odd to me, since they seem to indicate that the P4 outperforms both my P4 2.8 Ghz and 3.2 Ghz systems... Here are the numbers for my 2.8 system

P4 2.8 Ghz, Windowx XP:

7.719 0.844
27.547 10.500

same under RedHat 9.0

5.480 0.750
27.390 10.580


Umm. According to the docs, you probably don't want to graph the first number/first row - that deals with making trajectories between generations. You want the first number of the SECOND row, which pertains to actual folding time - the program spends the vast majority of its time in this state.

[Morphy375]

One of my P4's:

P4 2,4GHz, 512MB,W2K Server


Summary
-------
Usr time Sys time
-------- --------
Maketrj 9.474 0.200
Foldtraj 43.533 8.773

Thought the XP2700+ should be much better.....

[TheOtherPhil]

Intel, P4, 3500MHz, WinXP Pro SP1:

Summary
-------
Usr time Sys time
-------- --------
Maketrj 6.219 0.547
Foldtraj 22.000 7.734

[TheOtherPhil]

Intel P4, 3640MHz, WinXP Pro SP1:

Summary
-------
Usr time Sys time
-------- --------
Maketrj 5.813 0.688
Foldtraj 21.719 6.969

[AMD_is_logical]

Originally posted by rsbriggs
Umm. According to the docs, you probably don't want to graph the first number/first row - that deals with making trajectories between generations. You want the first number of the SECOND row, which pertains to actual folding time - the program spends the vast majority of its time in this state.

The official word from Howard (Brian the Fist) on what best represents folding speed is:

If you want just one number, add the usr+sys time for foldtraj together and ignore the maketrj one.

(The above quote is from: http://www.free-dc.org/forum/showthr...&threadid=3646 )

And I find that if I bench several times, this sum is much more consistant than either the usr or sys time alone.

[ddn]

You'd save yourselves a lot of time if you'd quit whining about the P4 2400 time. The machine is an IBM xSeries, and it screams. Does it ever occur to you that some machines, at the same Mhz, might be faster than your little ricer-style consumer grade home-built hardware.

[TheOtherPhil]

Originally posted by ddn
Does it ever occur to you that some machines, at the same Mhz, might be faster than your little ricer-style consumer grade home-built hardware.

Er, no. My homebuilt, consumer grade, ricer style P4 rocks

[rsbriggs]

You'd save yourselves a lot of time if you'd quit whining about the P4 2400 time. The machine is an IBM xSeries, and it screams. Does it ever occur to you that some machines, at the same Mhz, might be faster than your little ricer-style consumer grade home-built hardware.

Well, if nothing else you just managed to insult EVERYONE here.

And, if you follow the instructions given by Howard to ignore the maketraj, your machine is nothing special for the range it is in - it is faster than a P2200, and slower than a P2800.

And, I'd be more than happy to compare the price tag of my sub $400 folding boxes to the price tag of your IBMx series box....

Heck - I'd be more than happy to compare the price tag of my entire 7 computer "farm" to the price tag of the IBMx.

[ddn]

And, if you follow the instructions given by Howard to ignore the maketraj, your machine is nothing special for the range it is in - it is faster than a P2200, and slower than a P2800.

Why don't you go back and actually READ the posts. If you did, you would notice that the maketraj data is there, but is not being used for anything. You are correct that that machine is nothing special, you should tell that to the people complaining that the numbers are "odd".

And, I'd be more than happy to compare the price tag of my sub $400 folding boxes to the price tag of your IBMx series box....

The problem with this comment is that it shows that you are a wanker. If your folding boxes cost anywhere near $400, you are doing it wrong. I could build a folding box for $200, and it would have the highest possible performance.

Heck - I'd be more than happy to compare the price tag of my entire 7 computer "farm" to the price tag of the IBMx.

I'm sorry that your "farm" only consists of 7 boxen, however, if you would check IBM.com, you would notice than an IBM xSeries 305 with a P4 2.4, only costs $1400. But I'll give you a hint, the 305 isn't a folding box. So 7x400 = $2800. You could even buy 2 305's.

Sorry guys, but you gotta play with the little kids before you can play with the big boys.

[rsbriggs]

Well, everyone here would be interested in hardware suggestions from one of the leading folders - feel free to recommend your favorite hardware configuration. I for one would listen.

Er, umm, where exactly did you say that you were in the overall stats again?

[Grumpy]

Boy, talk about touchy . I was curious as the P4 speeds are definately improved over Phase I speeds compared to the AMD Cpus. With Phase I , they could not match the XPs, now the situation seems reversed. Has there been some serious Intel Optimizations or am I even more dumb ass than I thought

[Welnic]

Originally posted by ddn
Why don't you go back and actually READ the posts. If you did, you would notice that the maketraj data is there, but is not being used for anything. You are correct that that machine is nothing special, you should tell that to the people complaining that the numbers are "odd".

The maketraj data is there, and it is being used to plot the data in your graph. A lot of people plot graphs to show useful information, it seems some people were sucked into paying attention to yours.

[ddn]

rsbriggs: Why don't you show me where you are in the overall stats. What's your username?

The maketraj data is there, and it is being used to plot the data in your graph. A lot of people plot graphs to show useful information, it seems some people were sucked into paying attention to yours.

How do you figure it is being used to plot the data in the graph. I did not use all those fields to plot that graph. Besides, it's a very crude graph.

[Grumpy]

Oh hush you silly person, I was curious for the reasons stated in my above post, I was not meaning to offend you Do not embarass yourself any further, lets just make peace and get on with the results, and maybe someone can cast light on my P4 Optimization question.

[derek]

Actually, I am curious where you guys are in the stats race? Last time I checked, I was 9th in daily production:
http://stats.zerothelement.com/cgi-b...s=50&color=YES

BTW, my name username is derek.

Oh course, I have only been doing it wed/thur.

I would contribute several bench marks to ddn's graph if the client wouldn't give a strange error under OSX (apparently it gives the same error under PPC Linux).

- derek

[Welnic]

Sorry, I thought I saw a correlation between the maketraj numbers and the graph. Looking at it again I guess that it is pretty much just random heights.

[Grumpy]

Humph, I have gone over all the records of Benchmarks from Phase I. It would appear Phase II Client must have been compiled/written to take advantage of Netburst & Or SSE 2, becuase the change in performance for P4s from Phase I to Phase II is quite large. Maybe the Compiler Howard is using is just AMD unfriendly...I think the more experience among us may shed some light on this, maybe Howard can shed some light on the subject

[rsbriggs]

Originally posted by ddn
rsbriggs: Why don't you show me where you are in the overall stats. What's your username?

I would be "CodeMonkey", 11th place on the FreeDC team, and somewhere around sixty-fifth place overall last time I looked (which was quite a while ago.) In a tight race for 9/10/11th places on the FreeDC team. I'd be more than happy to hear suggestions for high performance $200 folding boxen - you are more than welcome to PM me with the info, or to get my email address

I wouldn't mind putting together another 4 or 5 boxen at that price, that might be enough to move me up to team 6th or 7th place

[tpdooley]

Originally posted by ddn
You'd save yourselves a lot of time if you'd quit whining about the P4 2400 time. The machine is an IBM xSeries, and it screams. Does it ever occur to you that some machines, at the same Mhz, might be faster than your little ricer-style consumer grade home-built hardware.

You could have said "The p4 2400 time is from an IBM xSeries, and it screams."

I'm on the chart as BennyRop and bouncing around in the 30's range. I've got 16-20 Windows machines running DF (depending on how many of my helpers decided to quit because of the various Phase II problems). Each of us do what we can for the project, regardless of the nature of the machines we're running. The vast majority of of the 1600? current active folders are in the single or double folding machine category (90%?).

[TheOtherPhil]

Originally posted by Grumpy
Humph, I have gone over all the records of Benchmarks from Phase I. It would appear Phase II Client must have been compiled/written to take advantage of Netburst & Or SSE 2, becuase the change in performance for P4s from Phase I to Phase II is quite large. Maybe the Compiler Howard is using is just AMD unfriendly...I think the more experience among us may shed some light on this, maybe Howard can shed some light on the subject

I think it is more to do with the 800fsb chips on the i865/i875 chipset. SSE/SSE2 doesn't provide any speedup for DF if I remember correctly.

Here's a nice article that shows just how far behind AMD have been pushed with the release of the i865/i875 chipset/ P4C combo.

[ddn]

Maybe the Compiler Howard is using is just AMD unfriendly...I think the more experience among us may shed some light on this, maybe Howard can shed some light on the subject

At the prospect of offending anyone I haven't offended yet: Howard's code sucks. His vast experience has led to PowerPC routines that are broken (try -bench) which indicates to me even the whole PPC client is broken. The code is obviously faster on Intel hardware, because either it is optimized for that, or isn't optimized at all and just runs faster on Intel than AMD.

Look at SETI@Home for a serious computing project. The clients are uber-optimized for each hardware platform by the best people on each respective platform. Do you really think that Howard can write better routines on Sparc than a compiler geek from Sun. I already spoke of the horrid PPC code. If you all, especially Howard, would give up your unrelentingly pompous attitudes, some people with SERIOUS experience (dtj) would be willing to optimize your code.

I'd be more than happy to hear suggestions for high performance $200 folding boxen - you are more than welcome to PM me with the info, or to get my email address

I'm sure you would. If you were on Ars, I'd even tell you.

Like I said, you gotta step out of the nursery before you can play with the big boys.