Older stats issue

[AMDave]

I just fielded a question or two about the apparent 'double' scores on 30-07-2010.

To help answer the same question for anyone else, the dates on the "Last 28 Days" table are calculated labels on the table in the web page (eg today, today -1, -2 etc).

Because the non-BOINC process froze and did not fail, the nightly rollover was prevented from shifting the data across in the daily data table, which in theory should have created a column full of zeros because the refresh did not happen on 29th.

Consequently the table appears to be out by a day prior to 30-7-2010.
The date label saying 29-7-2010 contains the volumes from 28-7-2010.
The date label saying 30-7-2010 contains the volumes from 29-7-2010 and 30-7-2010.

The volumes are correct.
No data is missing.
No data has been duplicated.
The 'apparent' anomaly is self correcting and will shift itself out of the "Last 28 Days" table day by day.
This does not affect the weekly or monthly stats.

Trust us. It is Auto-magic

[alpha]

Some people are reporting particular milestones on Whatpulse not being announced, like 10 million keystrokes. How are the milestones flagged up these days? Is it still a hard-coded list of thresholds?

[gopher_yarrowzoo]

I do believe it is still hard coded so should be working fine it all depends on what data we get from what pulse itself.. you got any particular user(s) in mind so we can look at this closer.

Thanks

[Bok]

No, it's dynamic, based on entries in a table which can be tailored per project if needed. I'd need to know a particular example as Gopher mentioned in order to check it out.

Crude milestone page shows some 10M and the last log certainly showed that it tested for 10M

Bok

[alpha]

Sellyme says his 10 mil didn't show. Indeed, it isn't listed in the recent milestones.
VG says his 2.5 mil didn't show. Also not listed in recent milestones.

[Bok]

I see, actually they've found a bug! And it's been like that forever, perhaps it's just that the exact rounded type numbers from whatpulse have made it more pronounced.

In VG's case on August 1st he dropped 150,000 points to reach EXACTLY 2,500,000 then on the 3rd he dropped another 50,000 to be a 2,550,000

However my sql was checking for new value > 2,500,000 and old value < 2,500,000 which did not match under either condition. It should actually be

new value >= milestone and old value < milestone

WOW, that code has been in place for about 8 years now.. fixed and I manually added those two milestones.

[alpha]

Nice bug!

Thanks for addressing this so quickly.

[Sellyme]

Yay! I feel useful

[gopher_yarrowzoo]

Hmm appears this needs a major kick again... I'll look at it when I get a chance.

[alpha]

I have another query regarding the Whatpulse stats. According to our team page, we lost over 4 million keystrokes on 17th August. At a glance, I can't see this as a result of someone leaving the team, and I can't find another team with a stat reduction for that day. Any clue what happened here?

edit: Additionally, it looks like there might be something wrong with user ranking. The project stats page shows that the current top two users overall ([db]Deluxy, qwertycomputer) each have less keystrokes than the guy in third place (Orznorz). Also, clicking on each of the names of these users shows that they all have "project rank" of 1.

[Bok]

Not sure what happened on the 17th. The tracking of user movement is not perfect for some of the non boinc projects though.

User ranking is a simple one though. I don't 'hard' delete users which are gone (often name changes), I simply mark them for deletion. They are not used as part of the project ranking at all and in some places I don't actually show them, but by no means all places. I've deleted them now, so next time it runs they won't be visible.

[alpha]

Thanks Bok. The project rankings look in order for the top 5 now, and their scores look sensible. I might contact the Whatpulse guys about the 4+ million drop on 17th because that's around a tenth of our entire team total! Hopefully they can shed some light on what happened.

[Paratima]

Hey, Bok. Quantum Fire Alpha seems to be stalled. The project site shows me over 1 million, while our stats aren't updating. TIA!

[Bok]

All the stats are a little backed up today, a monthyl job went a little haywire. They are running now. On my parallel (faster) server though, according to the latest exported xml you are still only at 999,782 So you might have to wait a few more hours yet..

[Paratima]

THANKS! The champagne is in the cooler!

[kingcarcas]

Can someone help me out here, for years i simply used the alternative sig images, but now i joined a DIMES team and they don't display with the others, just free-dc it seems....

I could never get my sig to display properly, the other scores aren't showing up


My old folding stats don't have a cross project identifier and the team name is spelled differently so i fear i won't be able to attach those to me
Same with DIMES, can you fix this?

[Bok]

I can indeed. The tag.php as you are using should only be used when trying to display projects which are nonBOINC as it provides a way to link ANY projects togetehr by allowing you to input your project name. Therefore it is not automatic at all and does not use any form of cross project identifier.

Consequently for you these are your names :

Folding@Home : kingcarcas36362 (I have to concatenate with the teamname to prevent duplicates in Folding.. )
GpuGrid : 47002
WCG (Boinc) : 522633

The last two are your id's in those two projects which you can see either at the project or going to your userpage here

I've updated it for you and purged the cache so it shows up immediately Hope that helps!

Bok

[kingcarcas]

Thank you!!!!!!!!!!!!!!!! Here is a cupcake

[alpha]

Do we need to request specific teams to be added to the Whatpulse stats? I can't remember what the deal is with that (and the forum search is returning nothing relevant). If so, please could you add "Kongregate Elite".

[Bok]

yes, I have to scrape each one as they don't provide a full file of stats in any form

Added that team.

[alpha]

Many thanks.

[vaughan]

Could you check the NPLB individual user stats page please? It hasn't updated correctly in the last day.

[enderak]

Seventeen or Bust stats have been screwy the past few days as well with zero's across the board for daily work, total work, etc.

[gopher_yarrowzoo]

Could you check the NPLB individual user stats page please? It hasn't updated correctly in the last day.

Hmm Just looked at your page now and it seems fine where are you looking and what do think is wrong?
http://stats.free-dc.org/stats.php?p...b&name=vaughan

Seventeen or Bust stats have been screwy the past few days as well with zero's across the board for daily work, total work, etc.

Hmm looked at this I do see 0.00's but I don't see major issues with your personal stats page maybe a glitch although I do notice there is no last update time on the user pages now... hmm odd sure there should be.

[Bok]

There was a crashed table on the nonboinc side of things. I set off a repair yesterday evening and then the normal automated scripts take over and correct it all. I'm getting this very occasionally but haven't found any reason for it as of yet. It never affects the stats tables themselves just the replicated version that the website uses so no harm is ever done.

I haven't found a way to automate the repair (or notify me) yet so am relying on anyone noticing it to keep letting me know here

Oh and yes there is a glitch in the TZ times I noticed it too yesterday.

[gopher_yarrowzoo]

There was a crashed table on the nonboinc side of things. I set off a repair yesterday evening and then the normal automated scripts take over and correct it all. I'm getting this very occasionally but haven't found any reason for it as of yet. It never affects the stats tables themselves just the replicated version that the website uses so no harm is ever done.

I haven't found a way to automate the repair (or notify me) yet so am relying on anyone noticing it to keep letting me know here

Oh and yes there is a glitch in the TZ times I noticed it too yesterday.

Gotta love those TZ glitches...
Hmm I've had an idea but i'll need to look at the table(s) in question to see if it would work.

[LAURENU2]

I thought of a good add on
In our CPID page http://stats.free-dc.org/stats.php?p...d79febe1b85a39
In the Project Rank column of each project If the Rank # could be a link to bring you to that project rank page.
What do you Think

[Bok]

Done..

The link on the combined row is a little off due to split cpid's messing up the offset earlier in the list. Not too much I can do about that at the moment.

[LAURENU2]

Thats Way Cool Bok Your the MAN
__

That should save some page loads

[Paratima]

I don't know if this has been reported yet, but for the last week or so, the HOSTS report on individual stats pages is not working. I have done nothing to set the "Hide My Hosts" switch, but all I see on my personal stats page on any project is the HOSTS header line, followed by the column headers for the Hosts report, then it immediately displays the Historic Team Movements header. No biggie, but sometimes I like to see how my far-flung hosts are performing. TIA

[Bok]

No, didn't know about it. Corrupt host table on the webside. Should be fixed soon...

Tx

Bok

[LAURENU2]

OK I have a new one In the "Combined Users" page

http://stats.free-dc.org/stats.php?page=boincusers

You Can Not sort in the "Today" " Last Update" " Yesterday"
Like you can in the project user's page I Like Sorting He He
It was nice with the old stats that sorted All columns except one

[gopher_yarrowzoo]

OK I have a new one In the "Combined Users" page

http://stats.free-dc.org/stats.php?page=boincusers

You Can Not sort in the "Today" " Last Update" " Yesterday"
Like you can in the project user's page I Like Sorting He He
It was nice with the old stats that sorted All columns except one

Now looking at the code Today / Yesterday are calculated fields i.e they are current score - day1 ; yesterday is day1 - day2 so really not that easy to do, last update yes since that is already a DB field. The reason we can do it on users&proj=....&sort=today is there are the DB fields in place now i've just checked how many DB rows are in the table used
there are 2,085,143 that is a lot of rows to work on to add 2 extra fields and then adding it into the update scripts so it works... the old stats were also not very efficient.

[Bok]

actually, there is a 'today' field on that table, I coded it quite some time ago, not sure why I never updated this page to use it rather the calculated number, so it can be added easily (I fixed the sql at least now). It's not currently indexed on the table but that's easy.

I'll add a new column for 'yesterday' too and fix up the perl scripts to calculate it during rollover when I get a chance.. I did add them to the regular user table long time ago just for this.

Need a better way to keep track of the 'todo' list as I get lost...

[gopher_yarrowzoo]

Need a better way to keep track of the 'todo' list as I get lost...

Hmm don't we all I do believe there is one of sorts already maybe we should add things like this to it too

[LAURENU2]

Hmm don't we all I do believe there is one of sorts already maybe we should add things like this to it too

Blog It
The To-Do Blog

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

OK I found 1 more Glitch
Our Supercomputer is MIA
http://supercomputer.free-dc.org/

[Bok]

Hmm, very broken. I've fixed up quite a bit of it but still no data showing. Mostly caused by a tightening of the rules for php so the shorthand notation of '<?' which is used extensively is not allowed anymore and I have to change it all to <?php..

[LAURENU2]

Ya that is a problem when you upgrade
I guess that's why I still have my 1st wife
Just to much work getting it all back right

[gopher_yarrowzoo]

Ya that is a problem when you upgrade
I guess that's why I still have my 1st wife
Just to much work getting it all back right

That and you probably can't afford the divorce and keep the pharm.

[LAURENU2]

That and you probably can't afford the divorce :P and keep the pharm.