Results 1 to 32 of 32

Thread: Database problem?

  1. #1
    Member
    Join Date
    May 2018
    Location
    High Point, NC
    Posts
    30

    Database problem?

    Seems none of the stats are showing for me. It just shows a page without any of the stats numbers when I try to view my stats.

  2. #2
    CO2 Emissions Algae Scientist Nflight's Avatar
    Join Date
    May 2006
    Location
    Fredericksburg, Pennsylvania, United States
    Posts
    41
    You are, not the only one seeing nothing on the view screen. Seems the DB administrator is high overhead in an Airplane and will not change or reboot the system while the plane is in flight. It may be till Friday midday till the DB is returned to normal.
    Please stand By


  3. #3
    www.amdusers.com
    Join Date
    Apr 2002
    Location
    Sydney, Australia
    Posts
    266
    I'm suffering from bok's stats withdrawal symptoms.

  4. #4
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    Yes, tried some remote debugging via FaceTime with my wife as I’m in Buffalo but it looks to be hardware related. Still trying.

  5. #5
    CO2 Emissions Algae Scientist Nflight's Avatar
    Join Date
    May 2006
    Location
    Fredericksburg, Pennsylvania, United States
    Posts
    41
    Speaking with Bok this morning here on the East Coast, it seems this is not a simple reboot. It may be a hardware problem, and thus till he returns and gets to look at the problem we are out of Stats.


  6. #6
    Member
    Join Date
    May 2018
    Location
    High Point, NC
    Posts
    30
    I'm going to need to go to the Doctor to get a prescription for my withdrawals.

  7. #7
    I emailed him and he has tried with aid of his bride to repair things but still no soap. Seems the site of late has been having more and more issues. Hope to see it return functional soon as BOK does have a nice site!

  8. #8
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    Finally got some time with my wife to look more into this. Unfortunately the database server has crashed hard and while it appears to boot I can’t connect so it must be sitting at a bios prompt. Not going to try and walk my wife through hooking up keyboard and monitor so it will need to wait till I’m home

  9. #9
    Member
    Join Date
    May 2018
    Location
    High Point, NC
    Posts
    30
    Dang, I could drive over there and look at it. lol

    On a more serious note. Any idea when you'll be able to physically look at it? My stats withdraws are at an all time high right now. LOL

  10. #10
    CO2 Emissions Algae Scientist Nflight's Avatar
    Join Date
    May 2006
    Location
    Fredericksburg, Pennsylvania, United States
    Posts
    41
    I picked up a part-time job to get my mindset off the statistics fandango, It didn't work all through work I thought of where I stood with everyone near me, how far I was away from people I am attempting to catch and then decided to scream as loud as I could at a brick. Since the brick did not listen and did not respond, I put the brick down and walked into the Bar nearby and ordered a double Crown Royal Vanilla on the rocks. After one I ordered a second and forgot why I came in there. On the steps coming down from the bar, I tripped over the brick and took two stitches, plus scraped my knee and elbow. I removed the bandages Saturday night, the wounds will continue to heal and the scrapes will be there for a long time. If my Stats worked this would not have happened. The End


  11. #11
    Member
    Join Date
    May 2018
    Location
    High Point, NC
    Posts
    30
    OMG the stats being down almost killed you too???

  12. #12
    www.amdusers.com
    Join Date
    Apr 2002
    Location
    Sydney, Australia
    Posts
    266
    I have nervous anticipation for when Bok can fix the stats. Are we there yet?

  13. #13
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    OS drive is hosed. Database drives are all ok though. I have to rebuild it somehow which is going to take time.

  14. #14
    www.amdusers.com
    Join Date
    Apr 2002
    Location
    Sydney, Australia
    Posts
    266
    Not found error now when trying to reach the stats page. Bok must still be re-building the site. Do you have an ETA?

  15. #15
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    Was that on stats3? I'd forgotten to alter that one, I usually do a redirect on everything to a single page. I'd prefer to get rid of stats3 altogether but I know some people still use it. Anyway it was still trying to connect into the DB server which was causing some contention while I fix up things.

    I've got about 50% of the perl scripts converted and working, including the main ones that download the data and the ones that push the data into MariaDB and do all the aggregation and such. Times seem pretty similar to before so I think I've got the DB parameters set correctly. I can tweak those as I go along.

    At this point I *think* I'll be able to switch things back on again tomorrow. Of course the drives are hanging loose in the DB server right now, so I'll have to tidy that up probably over the weekend.

  16. #16
    Member
    Join Date
    May 2018
    Location
    High Point, NC
    Posts
    30
    You're doing a great job, Bok.

  17. #17
    www.amdusers.com
    Join Date
    Apr 2002
    Location
    Sydney, Australia
    Posts
    266
    Bok, yes it was stats3 - I mainly use that as it shows more fine details than the new version. eg hover over computer and it shows info, bar highlights of my username in lists etc; much better.

  18. #18
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    All switched on again. There may still be some lingering issues.

    Already had one issue, where it was updating one database, but the webside was still pointing to same database leading to some contention. That is fixed now.

    I wasn't backing up some files like

    1) the custom db_dump.xml files I use for projects that have subproject data - fixed I think.
    2) my globalconfigs file!! aarghh, not much in it though so no harm
    4) my cron entries - there may be some scripts not running due to this, hopefully I got them all (wcg badges and subproject data, wuprop hours data, generic badge updates)
    5) mariadb conf file. I think this is recreated now, 2Gb tmpfs, plus all the sort and index settings I use. db is using 26Gb of Ram of the 32Gb on the server (it's max)

  19. #19
    Member
    Join Date
    May 2018
    Location
    High Point, NC
    Posts
    30
    Are the stats currently "frozen" on the time it went down and will update as they normally would now? I am showing 0 for stats today when most projects would have already updated by now. I'm guessing when those scripts run at that time they'll update the information at that time.

  20. #20
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    Quote Originally Posted by [H]Skillz View Post
    Are the stats currently "frozen" on the time it went down and will update as they normally would now? I am showing 0 for stats today when most projects would have already updated by now. I'm guessing when those scripts run at that time they'll update the information at that time.
    they'll work themselves out.

    The way it works is I updated one database, while the website points to the other, then flip it. But because of one of the bugs above it was only ever updating one of the databases and never flipping. I didn't notice till I switched on the site and got all the contention problems.

    You should start seeing some numbers soon.

  21. #21
    Member
    Join Date
    May 2018
    Location
    High Point, NC
    Posts
    30
    You rock man. Thanks for all your hardwork.

  22. #22
    Senior Member Dirk Broer's Avatar
    Join Date
    Jun 2013
    Location
    Leiden, South Holland, Netherlands
    Posts
    379
    I do not know if this is an issue, but the numbers haven't only started coming, I've seen them going too. I started with 34 projects at 1M+, and WEP M2 was at 999,244 at the next update giving me just 33 projects at 1M+. Looks like the two stats servers (let's call them flip and flop) are out of sync witch each other (and one of the two has a serious gap with the projects as well).
    Last edited by Dirk Broer; 12-06-2018 at 07:35 PM.


  23. #23
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    Quote Originally Posted by Dirk Broer View Post
    I do not know if this is an issue, but the numbers haven't only started coming, I've seen them going too. I started with 34 projects at 1M+, and WEP M2 was at 999,244 at the next update giving me just 33 projects at 1M+. Looks like the two stats servers (let's call them flip and flop) are out of sync witch each other (and one of the two has a serious gap with the projects as well).
    So WEP is missing for you?

  24. #24
    Senior Member Dirk Broer's Avatar
    Join Date
    Jun 2013
    Location
    Leiden, South Holland, Netherlands
    Posts
    379
    Quote Originally Posted by Bok View Post
    So WEP is missing for you?
    Not missing, WEP was flipping to and fro round 1,000,000 credits yesterday -both servers not picking up DHEP's last update, which would give me yet another 1,000,000+ credits project.


  25. #25
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    Well that sounds exactly like the issue I mentioned above where one database was not being updated. Is it ok today?

  26. #26
    Senior Member Dirk Broer's Avatar
    Join Date
    Jun 2013
    Location
    Leiden, South Holland, Netherlands
    Posts
    379
    It is much better, only DHEP (Distr. Hardware Evolution) still refuses to update


  27. #27
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    the db_dump.xml file is giving a 403 error

    -bash-4.2$ wget http://dhep.ga/boinc/stats/db_dump.xml
    --2018-12-07 14:04:34-- http://dhep.ga/boinc/stats/db_dump.xml
    Resolving dhep.ga (dhep.ga)... 139.184.49.229
    Connecting to dhep.ga (dhep.ga)|139.184.49.229|:80... connected.
    HTTP request sent, awaiting response... 301 Moved Permanently
    Location: https://www.dhep.ga/boinc/stats/db_dump.xml [following]
    --2018-12-07 14:04:34-- https://www.dhep.ga/boinc/stats/db_dump.xml
    Resolving www.dhep.ga (www.dhep.ga)... 139.184.49.229
    Connecting to www.dhep.ga (www.dhep.ga)|139.184.49.229|:443... connected.
    HTTP request sent, awaiting response... 403 Forbidden
    2018-12-07 14:04:35 ERROR 403: Forbidden.


    I've asked Michael to take a look. I can work around that if need be as it doesn't really change.

  28. #28
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    the db_dump.xml file is giving a 403 error

    -bash-4.2$ wget http://dhep.ga/boinc/stats/db_dump.xml
    --2018-12-07 14:04:34-- http://dhep.ga/boinc/stats/db_dump.xml
    Resolving dhep.ga (dhep.ga)... 139.184.49.229
    Connecting to dhep.ga (dhep.ga)|139.184.49.229|:80... connected.
    HTTP request sent, awaiting response... 301 Moved Permanently
    Location: https://www.dhep.ga/boinc/stats/db_dump.xml [following]
    --2018-12-07 14:04:34-- https://www.dhep.ga/boinc/stats/db_dump.xml
    Resolving www.dhep.ga (www.dhep.ga)... 139.184.49.229
    Connecting to www.dhep.ga (www.dhep.ga)|139.184.49.229|:443... connected.
    HTTP request sent, awaiting response... 403 Forbidden
    2018-12-07 14:04:35 ERROR 403: Forbidden.


    I've asked Michael to take a look. I can work around that if need be as it doesn't really change.

  29. #29
    Quote Originally Posted by Dirk Broer View Post
    Not missing, WEP was flipping to and fro round 1,000,000 credits yesterday -both servers not picking up DHEP's last update, which would give me yet another 1,000,000+ credits project.
    NFS is doing this too. An update with values for the day, then later everyone has zeros.

  30. #30
    Multiple people with the same rank on E@H. Team Overclock.net. Team credit is out of order as well but its by the default sort.
    https://imgur.com/a/bX03KSL

    Other teams are also showing similar things.

  31. #31
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,615
    Blog Entries
    13
    I had a feeling this would happen...it's due to the new option at Einstein@Home where users have to opt-in to have their data exported for the 3rd party stats, they have just enabled it by the looks of it and I'm showing the #users in the xml has dropped from ~469,000 to just 2619!!!!

    Code:
    MariaDB [dcfree]> select active_this_run,count(*) from boinc_user where proj = 'eah' group by active_this_run;
    +-----------------+----------+
    | active_this_run | count(*) |
    +-----------------+----------+
    |            7269 |        1 |
    |            7270 |   466126 |
    |            7272 |     2619 |
    +-----------------+----------+
    3 rows in set (1 min 11.18 sec)
    Luckily I've deferred the automatic deletion of users which we are supposed to do, which is why the projranks and such are messed up but this is a total mess....

  32. #32
    Ah yeah. Makes sense. I had just went in and enabled stats export at LHC for this as well so I'm guessing stats will be garbled for that project at some point.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •