PDA

View Full Version : Some db issues



Bok
09-02-2011, 06:18 PM
After the power outage today I have some crashed tables in need of repairing..In fact the combined boinc user table needs to be recovered from last night's backup.

Shouldn't be too long.

AMDave
09-02-2011, 08:19 PM
Thanks for your owl-like attention, Bok.
Impressed by your response and restoration times. A lot of private enterprises would be jealous. :thumbs:
Hope the restore goes smoothly.

uBronan
09-02-2011, 09:41 PM
Today when i try to view my stats nothing shows except my cpid
Everything shows up with 0 for 2 days now

Bok
09-02-2011, 10:34 PM
Still having problems unfortunately..working on it.

AMDave
09-03-2011, 12:39 AM
Looks like that last command did the trick :)
Well done Sir!

(edit - I think I just Knighted you with a capital 'S', haha - edit)

Jeff17
09-03-2011, 01:14 AM
Good work on getting the stats mostly restored. In case you were not already aware of lingering problems. I am seeing some oddness in my stats and other teammates. For me I am showing a -6M which I have tracked to 4 missing projects; DNA, Leiden Classical, WEP, yoyo.

Thanks for the hard work you put in.

JerWA
09-03-2011, 01:17 AM
Last update stripped PrimeGrid off my account. -53 million. Ouch!

CPID matches here and on PG site so I imagine it's something wonky in the DB rather than an ID split.

Cross-project ID: 0f741dcc57d0944ef82a0c15bd4d42cc

Just a heads-up Bok. :)

Edit: Guess I'm not alone, must've started my reply right after Jeff did.

ChertseyAl
09-03-2011, 02:23 AM
Bok, is this just to stop me finally getting my 100 projects milestone? ;)

At least I only lost half a million credits - Looks likely just to be Primegrid that's missing.

Al.

AMDave
09-03-2011, 03:14 AM
Bok had to restore some tables from the backup taken the previous day so some stats 'appeared' to have rolled back.
The tables are now cycling through the progressive and nightly-rollover updates.
Your points WILL re-appear when the update cycle gets through all of the projects.
For some projects this can take up-to 24 hours depending on when they release the next updated stats file.
(I think PG is about every 6 hours / 4 times per day - IMMSMC, but I can't find the reference right now to confirm that)

In advance, may I say "Welcome to the 100 club!"

Bok
09-03-2011, 04:06 AM
There are a few problem rows still in the user and host tables. I have backups and will restore those individual rows in the morning. Looks to have affected about 2000 rows from 3.5Million. It might mean that some users appear as new if there is new xml exports prior to me restoring, which is not a huge deal.

Bok
09-03-2011, 04:08 AM
Oh and congrats on the 100projects Al.

I myself hit a very personal milestone yesterday which will be reflected on the next stats update from the project today.. 100K in Chess@Home - a hugely frustrating project to run, given the limited wu's/platform, short timeframe on wu's and the 1 hour backoff every time. Taken me a LONG time to get there...

uBronan
09-03-2011, 05:39 AM
The stats appear again so i guess in a short while all will be fixed and updated.
Great work Bok you wizard :D and congratz on that project.
I am currently trying to get all my projects at number 1 on the sign up date.
But one project is gonna be a problem since some persons runs on that date which i never can catch up.
they are simply is too rich to ever get near them, they simply buy everytime the newest gpu cards and speeds ahead.

ChertseyAl
09-03-2011, 06:39 AM
Oh and congrats on the 100projects Al.

Cheers :) Just need the latest update from YAFU and the MMs to update and I'll be there.

I'm still hoping to hit 100k in Ye Olde Original LHC - Looks like the relaunch as SIXTRACK might be a sign of some work :)

Al.

gopher_yarrowzoo
09-03-2011, 01:09 PM
Yeah well we are hoping that the work we are doing with the stats that you don't see, will stay that way totally hidden so you don't even spot it ;) but yeah thanks for the heads up to what to snafu'd...

I'm doing some UI changes but well you won't see them until they get released :D

uBronan
09-03-2011, 11:10 PM
A bit off topic maybe but i had to change my logon info on all projects since i have seen some changes which i really did not like
How long before the cpid gets settled so i can put it into the stats tool for quick access my stats and will the old retired projects follow up or does that needs some fixing in the dbase as well?

gopher_yarrowzoo
09-04-2011, 11:42 AM
A bit off topic maybe but i had to change my logon info on all projects since i have seen some changes which i really did not like
How long before the cpid gets settled so i can put it into the stats tool for quick access my stats and will the old retired projects follow up or does that needs some fixing in the dbase as well?
Hmm the cpid should settle once all your boinc boxes have contacted the projects and the new cpid hash has been passed to them, as for old & retired project hmm well we would need to do a fix on that for ya. You may get a split cpid for a bit...
Hmm maybe something we could work on....

Bok
09-04-2011, 01:06 PM
If you only do BOINC projects, no need to use statstool as the userbycpid page will list everything. Statstool is mainly just for nonBOINC..

uBronan
09-04-2011, 02:44 PM
Well not yet, on many attempts to get full view.
Somehow i still have not all projects on one page.
Statstool made show some which i knew the userid on that user page but on cpid i get sometimes weird results including some german guy his stats :D
But when i search on name, untill now i am still missing some projects under the total
And ofcourse the old projects leap behind or fail entirely to follow :D
like bbc cpdn, seasonal, rieselsieve not sure what aqua will do now it has been set to retired :)

On the non boinc projects i totally forgot many info and since most no longer exist its also no longer possible to get them.
Like i have been running some time on folding@home and at certain moment put the folding under (Webbie) evga team but since you can not control it in any way i stopped
I also ran dpad,tsc both, seti@home(classic) and more of these projects but in time all got lost

Bok
09-04-2011, 03:47 PM
Looks like a few CPID's - http://stats.free-dc.org/stats.php?page=search&proj=&team=&name=ubronan&exact=N&cross=Y so you've definitely got a split CPID.

Make sure you change your email address at Climate Prediction, Seti@Home, SetiBeta & Ufluids and make sure they are connected on one of your machines that is also connected to your other projects, then they should sync up on the next xml export from each project - I'll fix Seasonal Attribution for you in the database once you've done that as that is no longer exporting.

As for the nonBOINC projects, you should certainly still be in the database but I have no idea what your userids are without knowing the name(s) you used in them :)

uBronan
09-06-2011, 01:40 PM
After 4 days as far as i can see almost everything lined up
Except the dead projects

As i remember i kept for a long time the same username uBronan

For rieselsieve last position was 2851 with 7,545 credits
For BBC experiment last pos i recorded was 53745 with 4,666 credits
For seasonal user position recorded was 1101 with 14976 credits
Send you the userid's from the ones i still have :D

BBC experiment and ufluids is impossible to enter since it has giving me a db error for many months thats why i reported it dead like aqua
The last time i could login is somewhere in 2010, the funny thing is that there is maybe a new bbc project 2 in the making
To be honest i do not care about ufluids much since i ended very early in the project.

Bok
09-06-2011, 07:56 PM
I'll find and fix tonight for you..

uBronan
09-07-2011, 04:09 AM
That would be awesome :)

I am currently struggling to understand prpnet and the badges from primegrid
Scary enough only 2 projects seem to be gpu ones and the cpu projects really take ages .... i got one of those sob ones and still is busy after roughly 5 days (122 hours and counting at 55%)
It prolly gives crappy credits as well :D

On the psa prpnet side was messing with the cuda dos client which after i think was 20 hours gave me 10k points on the prpnet but lol found out this is almost worth nothing, because they are worth if i read it right only 50 cobblestones or so.
And could not find myself on the stats here as well on the primegrid stats, i guess it only shows up when you run for a year or so xD

Good god i should have stayed just on gpu projects which pay tons of cobblestones very fast :D

Bok
09-07-2011, 01:59 PM
I'm doing it now actually.

I think prpnet from PRimeGrid is completely separate from there BOINC part, though they do offer a badge in the PSA (Project Staging Area) which is run outside of BOINC, I don't think prpnet is part of that. You'll likely get close to a badge with just one SoB wu I think. IIRC they give ~9000 credits each, maybe more now that the exponent is higher, so hang in there with the one you are running. Yes, only two of their subprojects are cuda, they've had more in the past like the AP26 subproject but it was basically completed.

You only show up in a BOINC xml export when you've got any sort of credit at all. What is your primegrid ID ? Perhaps run the ppsieve project on your cuda card, it's still pretty good credit and the wu's are only 10mins or so. They export stats 4 times per day so it wouldn't take long to show up..

Let me fix your older id's now and I'll let you know.

Bok
09-07-2011, 02:12 PM
I've done seasonal attribution and ufluids.

I found a 'Bronan' in BBC Climate, nothing remotely like it in Rieselsieve though. There is only a single user with 7,545 credits - Webbie (http://stats.free-dc.org/stats.php?page=user&proj=rsi&name=9908) ?? I think it must be you as if you click on the CPID at the top there is an entry for BBC Climate Prediction too..

If you confirm I'll move the cpid's to your main one :)

uBronan
09-08-2011, 03:51 AM
hahahaha yes those are mine but could not change them since it went offline :D
Rieselsieve should be Webbie cause thats the name which i used for many years on old projects tsc/dpad and older i once ran with some team also team_elteor_borislavj for a couple of months ;)

STE\/E
09-13-2011, 05:11 AM
Looks like the Combined User Totals are either still messed up or messed up again since yesterday I think ...

Bok
09-13-2011, 06:51 AM
Damn. yes, I see the issue. I'll have to recover that from yesterday, it's caused all combined milestones to be regenerated too. Bah.. it'll get on it later this morning.

STE\/E
09-13-2011, 08:07 AM
Whoa, Looks fixed now, lets see somebody beat that 1 day Total ... ;)

Bok
09-13-2011, 09:20 AM
Not yet, it's basically cycling around. I broke it yesterday when fixing the 'main' country for combined users using a materialized view. Just waiting on the current run to complete, then I'll restore the table from yesterday morning. It's going to mean today's totals are too high and yesterdays are too low, but I can probably fix that manually.

Plomos
09-13-2011, 01:52 PM
In response to the concerns on PRPNet work and such hopefully I can add to what Bok said to help you understand how it works. PRPNet is seperate from Prime Grid Boinc because they consider prpnet to be manual sieving of projects. For prpnet projects you earn prpnet credits that are then converted over to BOINC credits. the conversion comes out to roughly 18 prpnet credits/boinc credit. Since users can gain huge amounts of PRPNet credits in a short amount of time the boinc credits are spread out so that the RAC doesn't get crazy high all at once. Also stats from PRPnet are manually added to boinc stats about once a month and it normally takes longer for new users to see credit.

So for example here i have 163,489 PRPNet credits right now. If i stopped right now then once all the stats are processed i should end up with roughly 9000 Boinc credits for Prime Grid. currently my credits come in for boinc in 100-200 credit chunks each day although I'm not running the really long tests like 5oB or MEGA so that makes sense. It should be that the longer the test the more credit you will get.

So as long as PRPNet settings are configured correctly to match your PG info and then that it shows up right here in the stats the credit should eventually transfer