Thinking of writing some stats...

[jonnyw]

...for DF.

Before anyone says it, I know there are loads available already, but I'm doing it more educate myself than anything else.

I want to do it running MySQL and Perl I think.

Was just wondering what the process was.

I'd figured:

1) get latest update from distributed folding server every two hours.

2) go through the update, putting all data into a MySQL db, in the schema I'd chosen

3) write perl scripts to allow the data to be displayed from the database.

Does this sound about right?

Does step 1, require getting and parsing the data from each team page on the site, or is there a single file that contains all these details?

Would both steps 1 and 2 be done with perl scripts (strategically timed to run, to do their job)?

Oh and what size (in storage) will the database be? (i.e how much initially, and how much would it increase for each update after that)?

Thanks for any help guys :D

[pfb]

I use the tarball from http://anteater.blueprint.org/teampages.tar.gz - that contains both Phase 2 stats (team*.txt) and Phase 1+2 (teamtotal*.txt) where * = Team ID...

For my stats, the total DB size is currently 86MB - 82MB of which is the conquest data...really depends on how and what you're storing (my method is designed around as compact database as possible - 4MB for the bulk of the stats seems to satisfy that requirement :))

http://wibble.bounceme.net/Sneakers/dfstats_size.png


As for the process, that seems fine...mine is bit different as it goes:

1) get stats tarball

2) extract stats files

3) parse team*.txt files and update stats

4) produce user conquests based on hourly production (if needed)

5) produce team conquests based on hourly and daily production (if needed)

6) produce user conquests based on daily production (if needed)

All the webpages are PHP (and so dynamic) - as are any graphs produced...

If you want any pointers/info you know where to find me ;) :crazy:

[jonnyw]

EDIT: dion't see the rest of your reply <ignore>

[jonnyw]

oh, and forgot to metion, thanks for the quick reply.

I'm just trying to do a little research (as you've probably noticed with the increase in my posting over at LH) in all areas first.

Will prob get a server up and running first, and then concentrate more on this.

Just wanted to check that my initial ideas where ok? :D

[pfb]

seem to be - the biggest gray area is the database schema, and what you're planning to offer stats wise...

If you want some examples of how I store it in my setup PM me - but the most basic would be to store each update as a transaction (very basic level would a UID, a TID, name, protein and total points)...

Only issue with the DF stats are the Teamless users, if the user name is None then UID is 0...this makes it fairly hard to use as-is as you end up with a lot of Users with the same UID and TID but different points...doing transactional processing with them is a tad impossible...

[jonnyw]

not sure what i'm planning on offering (still have to think this up before I design the database).

Probably simple stats, with previous updates (not planning on conquests, cos there's not much point when I can get that info for myself from your stats :D)