March 2009 Stats Issues

[rilian]

Hi!
Can you please update FreeHAL@home stats :stretcher: ? There were some bugs on HAL site but since yesterday stats are working (for example on PolarSETI stats site)

thank you! :rock:

[Bok]

Nothing I can do about it I'm afraid. Their stats xml export seems to have been stopped..

No files at all here (http://boinc.freehal.org/projects/freehal_at_home/stats/) :(

[Bok]

They have their files in a non standard place, but have at last updated them so the stats picked that up recently...

Bok

[gopher_yarrowzoo]

I hate that when you have to go into some obscure URL to get stats out..
I mean <proj url>/stats/user.php?id=<userid> or /user.php?id=<userid> etc nice simple not
<proj url>/drill/drill/drill/stats/monkey.php?uid=nq<userid>&team=<teamid> etc..
:evil:

[rilian]

WCG listed twice...

http://stats.free-dc.org/stats.php?page=userbycpid&cpid=460722bea70bf2cee4fdc7b76164b6ed :eek:

[Bok]

Indeed and, bizarrely, correct according to the xml as well..

I didn't think it was possible, but the faulty lies entirely with World Community Grid. Luckily I don't have a unique index on cpid only..

Bok

[rilian]

thanks for looking into it, Bok!

There was a CPID glitch (user added another boinc project on the farm and it somehow temporarily changed his CPID on other projects on some machines). Do not know if it is completely fixed now on all his machines....

[Bok]

WCG are a bit different in how they run things. But seeing as the CPID is generted via an md5 hash of the email address AND email address is supposed to be unique...

I'll post on the boinc project list about it, Not sure if the WCG developers read it or not though.

[coo-coo-clocker]

I always come here for the user-level milestones, but alas it's been broken for a bit. Is it me, or is something else wrong?

[Bok]

Working fine for me. Can you show an example of where it's broken ?

[rilian]

Bok, i digged some info about this WCG CPID collision on Vzhik account.

At first (in 2005) account Vzhik was created with vzhik@sivash*** email. It had some CPID and all was fine cuz user did not crunch WCG, but crunched Rosetta and SETI on this email and nickname but there was another CPID for these 2 projects

In 2009 he returned to WCG crunching and saw that stats were messed up, and tried to create new acct (with nickname = vzhik@sivash*** and same email) to get same CPID like on rosetta/SETI. It did not work and he returned to old Vzhik acct with this email and crunched on it, and CPID was fixed.

Some days ago someone from his farm attached WCG to vzhik@sivash*** and it somehow connected to the second account and crunched one WU. After that this new computer was reattached to Vzhik, but one WU log remained in stats.

It is not WCG fault

[Bok]

so where do you believe the fault lies ?

In the user xml is

<user>
<id>555428</id>
<name>vzhik@******.net</name>
<country>UKRAINE</country>
<create_time>1231305957</create_time>
<total_credit>67.483467</total_credit>
<expavg_credit>13.124080</expavg_credit>
<expavg_time>1236089526.168140</expavg_time>
<cpid>460722bea70bf2cee4fdc7b76164b6ed</cpid>
<teamid>4576</teamid>
</user>

<user>
<id>142007</id>
<name>Vzhik</name>
<country>UKRAINE</country>
<create_time>1132921714</create_time>
<total_credit>487163.346963</total_credit>
<expavg_credit>5801.679299</expavg_credit>
<expavg_time>1236432626.005150</expavg_time>
<cpid>460722bea70bf2cee4fdc7b76164b6ed</cpid>
<teamid>4576</teamid>
</user>

I removed the email address even though it's public anyway..

I don't think WCG should have allowed two accounts to be created with the same email address, but I can quite understand how it might have slipped through the cracks.

But in the end it's there in the xml and I'm parsing it as it is. I don't think the stats sites should try to filter anything like this out personally :) Better to show it and then decide whether it should be brought to the projects's attention.

I'll will put a post to the mailing lists and see what others think...

Bok :cheers:

[coo-coo-clocker]

Working fine for me. Can you show an example of where it's broken ?

Normally, this shows the milestones for everyone on the team...

[Bok]

hmm, ok, I see that. Fixed it...

:cheers:

Bok

p.s. that page needs some work though, it's kind of bland....

[coo-coo-clocker]

thanks!
Sorted descending by date would be very helpful.

[Bok]

I'll have to change the sql a little as I'm doing a group by id and order by date desc but the group by takes preference..

I'll wrap it in a separate select and do the order by there..

Done, but I'll change it tomorrow to reformat the team milestones somewhat as well..

Bok :guntotin:

[zombie67]

http://stats.free-dc.org/stats.php?page=team&proj=qcn&team=13&sort=rank

The order of the users seems completely random. Or am I reading it wrong?

[Bok]

ugh, looks like something seriously amiss there.

Have they recently altered the credit at all ?

I don't (purposefully) delete 'old' users but I do flag them in the DB and do not include them in the ranking calculations. It's rare that a BOINC project has 'old' users but I'm showing a lot for QCN in this case..

Only showing 11 are 'active' for SETI.USA which is reflected in the ranks shown.

I'll delete the old ones tomorrow unless someone can see a reason not to.

The homepage here (http://qcn.stanford.edu/qcnalpha/team_members.php?teamid=13&offset=0&sort_by=expavg_credit) shows only 11 with credit so I'm sure it's right but I suspect QCN have reset their credit perhaps... didn't they used to give 50 per day or so ?

Bok

[zombie67]

Yes, they still award about 50/day. No resets that I know of. And I am kinda reluctant to ask them about it, because they said they aren't hot on stat sites carrying their numbers (even though they export them). Their RAC numbers on their own pages are crap, for sure. IMO, as long as the daily and total numbers are good (and I think they are), we are good to carry them.

[zombie67]

First: Thanks for adding the "most xxx" to our personal stats page!

Just a nit:

The order for the signature image (see mine below) is: Type Many RankT RankP

The order for the personal stat page is: Type Many RankP RankT

Can one of them be reordered so they are in the same order? I like the order of the signature image.

And finally a Q: It used to be that, in a tie, the one with the lower beginning letter in the userid was ranked first. That was fixed for the stat pages, I think, where they show tied for the same rank. Was it ever fixed for the signature image?

[Bok]

First: Thanks for adding the "most xxx" to our personal stats page!

Just a nit:

The order for the signature image (see mine below) is: Type Many RankT RankP

The order for the personal stat page is: Type Many RankP RankT

Can one of them be reordered so they are in the same order? I like the order of the signature image.


I hadn't even noticed that... oops, fixed.




And finally a Q: It used to be that, in a tie, the one with the lower beginning letter in the userid was ranked first. That was fixed for the stat pages, I think, where they show tied for the same rank. Was it ever fixed for the signature image?

Data all comes from the same place, so should be the same on pages/signatures. Do you see an anomoly ?

Bok

[zombie67]

I hadn't even noticed that... oops, fixed.
Thanks!

Data all comes from the same place, so should be the same on pages/signatures. Do you see an anomoly ?

Not at this time, but I don't have anything in a position to check. I noticed it a couple of weeks ago. But maybe it's fixed now along with the stats. Just thought I'd ask since I was posting about them anyway.

[bloodrain]

well i am having that problem now. i am log in and when i go to stat tool. it says not login in. i clean my temp cache

[gopher_yarrowzoo]

If you go to stats.free-dc.org now it will say your logged in, your only logged in IF your registered with the board that allows you to show off your personal tag like the one I have in my signature as you can put all your project id's etc into the stats tool and it will remember you, also lets you see all the projects you've put in id's for in one place your own user page, You don't see this if your not registered.
If that made any sense.

Like when I go to stats.free-dc.org
I see a link on the left hand side with "My STATS Page" this links to http://stats.free-dc.org/stats.php?page=usermain&name=gopher_yarrowzoo
your would be
http://stats.free-dc.org/stats.php?page=usermain&name=bloodrain
There is a FAQ on the stats site that should help you with all of this as well :)

[Bok]

Try the suggestions for Q2 in the FAQ (http://stats.free-dc.org/stats.php?page=faq)

and see if that solves it first. If not post back and we'll see if we can see what is going on.

Bok

[alpha]

Hey, I just noticed that we're only displaying keys and clicks on the Whatpulse stats. Would it be possible to get mouse mileage added in as well? It's already exported in the same XML that we parse for everything else so it should be easy right? :)

[Bok]

easy to parse, storing it would normally be the problem... but I actually hold 4 'metric' columns, so yes, it's easy.

Done the code, I'll 'enable' the 3rd metric on the frontend conf. They'll all show as 0 until the stats next run. And I'll have to set it to have 2 decimal places as well.

Bok

[alpha]

Awesome, that's perfect! Thanks chief. :cool:

[Death]

strange bug with sorting.

http://stats.free-dc.org/stats.php?page=team&proj=sha&team=159



Local Rank Team Rank Project Rank Name Today Last Update Yesterday 2 Days Ago Average Last 7 days Last 28 Days RAC Results
1 1 251 Algon 40 40 240 220 243 1,500 9,090 314 172,590
2 2 1749 1 kid_a 0 0 0 0 0 0 0 0 22,890
3 3 1986 1 UA_ReMMeR 0 0 0 0 0 0 0 0 19,480
4 4 3624 1 LostInSpace 0 0 0 0 0 0 0 0 8,010
5 5 4018 1 Skyman 0 0 0 0 0 0 0 0 6,570
6 6 5439 5 Tamagoch 0 0 0 0 0 0 20 11 3,420
7 7 5734 2 Lacher 0 0 0 0 0 0 0 0 3,000
8 8 7202 1 t0rik 0 0 0 0 0 0 0 0 1,640
9 9 7430 3 uNiUs 0 0 0 0 0 0 0 0 1,490
10 10 9936 5 Venenarius 0 0 0 0 0 0 0 0 630
11 11 10507 3 Skeeter 0 0 0 0 0 0 0 0 470
12 12 11625 5 SAngel 0 0 0 0 0 0 70 1 260
13 13 12864 3 Alesha 0 0 0 0 0 0 0 0 110
14 13 15758 Amorph 0 0 0 0 0 0 0 0 1,000
15 14 13522 2 rilian 0 0 0 0 0 0 60 1 70
16 15 13679 4 Death 0 0 10 0 1 10 10 0 60
17 16 14098 5 Eduard 0 0 0 0 0 0 0 0 40
18 17 14870 4 borman 0 0 0 0 0 0 0 0 10


13-14 places. results should be treated as numbers, not strings )))

[AMDave]

Amorph does not appear to be a member of your team on the project at the moment:
http://boinc.iaik.tugraz.at/team_display.php?teamid=159&sort_by=total_credit&offset=0

It appears that there was/is an error in the XML file from that project.
Bok should be able to confirm from the backup of the file when he has some time available.

I have verified that username and CPID are not in the current user.gz file from that project.

[Bok]

Unusual with boinc project for old users to be in there. It's quite common with the non-boinc projects though.

Basically it's just the same problem we've seen many times. These are 'stale' users and are flagged as such in the database, I just don't automatically delete them for safety sake..

Dave, you can see them by checking the 'active_this_run' field for all users in that team. The old one will have a lower number and can be safely deleted. (Though I'd prefer that I do it, so I'll take responsibility! Perhaps I'll write an admin function to make it safer.)

Anyways, all fixed. It'll propagate to live soon.

Bok

[LAURENU2]

what is this Units Available : 3900 on th top of the stats pages Linky to stat page (http://stats.free-dc.org/stats.php?page=proj&proj=poe)
Do you/we Ask the projects how many WU's that are Available ??
Some say 0 but still are working so I am confused :confused:

[Bok]

It's an attempt to see how many wu's are currently available for that project. Some projects don't provide an accurate value though or even any number at all, but yes I interrogate the servers to try to provide an up to date number...

Bok

[Death]

http://stats.free-dc.org/charts/projscorecumul.php?proj=sob

only zeroes at Y axis. looks confusing. maybe G T or even P?

[Bok]

hmm,
the code is all in to do that actually. Not sure why it isn't and I'm not in a place where I can get to the box easily to have a look right now...

Bok

[AMDave]

Thank you for pointing that out.
It looks like it was my mistake.
Fixed.

* please note that when Death posted above, the Y axis did indeed show only the trailing zeros.
* however, the fix is also reflected in the linked image in his post (above)