Chessbrain stats update

**Chinasaur** · 08-12-2003, 06:38 PM

ROFL

I have Two Top Ten slots...Linux and Mac OS X

http://www.chessbrain.net/stats-top-machines-graph.html

Hard to believe that that few people are running Linux on this project. Amazing.

Too bad teams don't show..be good advertising

**IronBits** · 08-12-2003, 08:03 PM

I've pulled the client from all boxen.
I made it to the top 8, close to Condor, now I need to get focused for the gauntlet.
It's been fun, hope I helped out in some small way

**Chinasaur** · 08-12-2003, 08:36 PM

IB,

Look at your production go up and tell us what the Chessbrain performance hit was. It's been about 10-20K for me I think over 5 cpu's.

**Beyond** · 08-12-2003, 08:50 PM

It's 15%-20% hit on Dnet.

**IronBits** · 08-12-2003, 10:39 PM

Originally posted by Chinasaur
IB,
Look at your production go up and tell us what the Chessbrain performance hit was. It's been about 10-20K for me I think over 5 cpu's.

That's gonna be a tough call with DF ouput, the way it cycles fast/slow. Just keep an eye on my stats.
I was breaking into 1.8 mil per day, now it's down to 1.5 mil ...
If I remember right, you can watch how much CPU time it takes away from the DF client with task manager... it's about 1-2 hours a day x how many boxen.
It's not as passive as I thought it was gonna be... If you only have a few boxen, probably not a big deal, but if you have alot of boxen and are getting ready for a gauntlet, it all adds up quickly

**magnav0x** · 08-12-2003, 11:46 PM

Good news:
*Rewrote the merge script and all appears to be ok. If you have other accounts that you need merged just give me the names and I'll merge them in with whatever you want.

*Fixed the issue where some Free-DC members where being assigned incorrect tID identifiers.

Bad news:
*I fixed some other things but I don't recall what they are

*There's no more good news

Hopefully after tonights stats update I won't need to make any more changes

**magnav0x** · 08-13-2003, 12:23 AM

ezracing.net domain will be down for a bit, "routine er....yeah you know the drill" time

**magnav0x** · 08-13-2003, 07:33 AM

All problems seem to be rectifyed. Site is back up.

**Chinasaur** · 08-13-2003, 08:04 AM

**cygnussphere** · 08-13-2003, 09:20 AM

Originally posted by IronBits
I've pulled the client from all boxen.
I made it to the top 8, close to Condor, now I need to get focused for the gauntlet.
It's been fun, hope I helped out in some small way

We would expect no less.... But after you and PCZ Fold your way to victory...

**cygnussphere** · 08-13-2003, 09:22 AM

Originally posted by magnav0x
All problems seem to be rectifyed. Site is back up.

thanks for all the hard work magnav0x

**cygnussphere** · 08-19-2003, 05:50 PM

/bump

never though I would see a CB thread with 4 pages.

oh and while I have your attention please review THIS

**Beyond** · 08-21-2003, 06:38 PM

ripped from the CB forum;

I found a problem in the stats generation code that was improperly totaling stats for some individuals. I’ve fixed the problem and now some users will see an increase in their stats. This will likely cause a headache for individuals who generate their own stats reports. If you’re such a person (Mack, and magnav0x are at least two!) then contact me and I’ll help answer questions and provide you with any assistance you need to adjust stats at your end.

The relative ranking of individuals and teams hasn’t changed much because the problem seems to have been rare.

- CJ

So it looks like a longggg climb to regain the #4 position

**magnav0x** · 08-21-2003, 08:24 PM

I'm glad you pointed this out ot me Beyond. I was beginning to think there was a coding problem with my stats again when I saw todays numbers. At least the problem is fixed now and we don't have to worry about it again (hopefully). I'm going to email CJ tonight and see what the problem was.

**cygnussphere** · 08-21-2003, 08:33 PM

paranoid post removed.

Faith in CB stats restored

./me scurries off to reciently dug hole to take my meds and hide.

sorry.

**magnav0x** · 08-21-2003, 08:36 PM

Cyg i'll try to get a straight answer from CJ tonight.

**Beyond** · 08-21-2003, 09:58 PM

The really odd thing is that we moved (gained) very little if anything from this "adjustment", when everyone one else in the top 20 or so seems to have had a significant jump in thier stats.

I don't think we were singled out

but it is odd that we did not benefit to the degree the others.

**magnav0x** · 08-21-2003, 10:07 PM

Still no response from CJ

**cygnussphere** · 08-21-2003, 11:00 PM

paranoid post removed.

Faith in CB stats restored

./me scurries off to reciently dug hole to take my meds and hide.

sorry.

**Beyond** · 08-21-2003, 11:11 PM

"shades of Genome@Home come to mind"

**ECL** · 08-21-2003, 11:26 PM

"Brain" is not a word I'd associate with this project.

I've deleted this piece of garbage from my machine (of course the uninstaller doesn't work) and moved on. Life is too short to waste any more time on this turkey. There are worthwhile projects out there; this isn't one of them.

**magnav0x** · 08-22-2003, 12:04 AM

From CJ:

Yesterday evening I was trying to find the cause of a problem that was
reported by animal on behalf of gistech1978.
http://www.chessbrain.net/bbs/viewtopic.php?t=541

Gistech1978 was climbing steadily in the rankings until the 19th when
his stats slipped from 273 to 2. I've seen this problem before on
rare occasions with other members.

Before continuing let me first explain how stats are logged. While
the SuperNode server is running it records all PeerNode client
transactions in a memory hashmap. Every five minutes the contents of
the hashmap is flushed to a disk file. This approach is considerably
faster than a database access and has the advantage that the resulting
disk files can be easily scanned and ultimately imported into a
database. This was important when the SuperNode was running on our
old PIII-1GHz machine with only 256MB. Now that we're using a P4
3GHz sever with 768MB its clearly less of an issue! However we're
still in the process of making the transition to a new stats system.

With our current approach, stats data is recorded based on game
sessions. This allows us to see exactly what happens during a single
game. In fact this information is stored in a game database.

A problem arises when the hashmap is being flushed to disk and the
server crashes for some other reason. The SuperNode server is a
heavily multithreaded application and so any number of things may be
happening while stats are being flushed. So it's not the flushing
of stats that causes a crash - its any other thing that may happen
on the server. Keep in mind that this is all relatively rare. The
SuperNode is quite stable at this point.

During the past six months, the times when the server has crashed, the
stats file has been incompletely written. This is a problem because
the same file is used by the server when it restarts. When the
SuperNode restarts it loads an incomplete user list into memory. This
causes users who connect and are not in the hashmap to receive new
records.

IMPORTANT: This is a reset in their stats for that session only!

When the stat session file is later imported into the database there
is a discrepancy for the users that were affected (that is the users
data which was not written into the incomplete stats files).

This happened often during the start of stage two, and less often
over time. What I did was rebuild the database by scanning all prior
stat data files and compensating for when the server crashed by adding
the total stats for that session to the users last recorded data.

The data is real and represents actual recorded transactions - the
problem was entirely in the program that scans the stats files and
imports into the database! I fixed the program so that it correctly
tallies user stats and so some users

The important thing to keep in mind is that the stats generated for
each user is stored in files so we can recreate a database in any form
and track problems over the cause of a project phase.

So I realized that if I "fixed" the import problem all users
will get their fair share of corrections and so this was the honest
thing to do. However, there was no way to avoid the disappointment of
contributors who thought that they were a bit further along and
slipped when another competitor moved up the ranks because of the
correction. It's a difficult decision to make however; I felt that
it's best to choose honesty - no matter what the consequences.

I suspect that other (larger) DC projects would probably hide this
sort of thing for fear of losing contributors. So I wonder how
contributors really know when their stats results are real..?
Perhaps the answer is open source stats J certainly open source in
general. That was part of my reason for openly releasing stats2.txt,
stats3.txt and the new rtstats.txt - so that problems could be
quickly identified and the results open to public scrutiny.

I don't know if this is a good enough answer for you guys, but he has layed everything out in front of me and these stat changes look very legit to me. I know it doesn't seem "fair", but things like this can happen.

**rshepard** · 08-22-2003, 06:57 AM

Things are back to the way they were- This from CJ:

I found a problem in the code that auto-generates the stats related web pages. The stats display should be back to normal at this point. During the past two days the stats have visually fluctuated but the problem was only due to the display and not the logging (tracking) of actual stat data. Its what I get for making late night changes

Please report any discrepancies to me and I'll investigate the database if necessary.

- CJ

**rsbriggs** · 08-22-2003, 08:36 AM

Code:

1 TEAMANANDTECH     10624.962450 
2 SWISSTEAM.NET     8408.483223 
3 EXTREMEDC         6353.575556 
4 FREE-DC     5477.563132 
5 ZUSES_HIRN        5412.212998

**IronBits** · 05-09-2004, 10:01 PM

Are there any 3rd party stats out there for Free-DC.

**PCZ** · 05-09-2004, 10:35 PM

IB

What's wrong with these

**IronBits** · 05-09-2004, 11:49 PM

Nothing, I just want more

**danzigrules** · 05-25-2004, 09:50 PM

Hello all

was wondering if anyone could tell me why magnavox's site only shows me with 1.66 total nodes and the main stat site has me at 22,497,171,452. I also have 4 pc's running and not the 3 that show up.

edit: guess I should say my nick too

ziggy

and if Bok sees this 30 minutes ago I was on your stats, now I am gone

if it helps any I am on Team-TNT

Thanks for all the hard work you all put into these stats

**Bok** · 05-25-2004, 10:11 PM

I think it's probably because the .txt files from the chessbrain site that magnav0x uses to generate his stats are not quite accurate.

I'm using the same ones and get big discrepancies as you can see here

I'm sure mag hits the same obstacles.

I'm presuming cjus has a DB which his stats are generated from which is accurate.

Bok

**magnav0x** · 05-25-2004, 10:12 PM

Bok is right. For some odd reason they can't keep the text files updated correctly. I talked to CJ a few weeks back and he is having problems with the company he is with right now. Real life problems come first over stats

**danzigrules** · 05-25-2004, 10:16 PM

ah ok,

Thanks for the super fast replies

**IronBits** · 03-03-2005, 09:22 AM

Originally posted by magnav0x
Bok is right. For some odd reason they can't keep the text files updated correctly. I talked to CJ a few weeks back and he is having problems with the company he is with right now. Real life problems come first over stats

Do they need a new hosting site? I could possibly host it like chessbrain.dbestern.com if they need a new home!

Thread: Chessbrain stats update

Thread Tools

Rate This Thread

Display

Another Top Ten LOL

Posting Permissions