Results 1 to 36 of 36

Thread: New Stats Unit - jEMs

  1. #1

    New Stats Unit - jEMs

    Hey guys,

    I'm tinkering with the stats and doing some long over-due updates on the stats unit.

    Quote Originally Posted by DAVE NORRIS, 4 years ago
    "The formula is Cn^3 where it should really be Cn^2 log log n."
    This should fix the inflating score problem, which was only going to get worse over time.

    So don't be alarmed if you start seeing "jEMs" replacing "cEMs" on different stats pages in the next few days.

    I'm probably updating the graphs first, then the text stats, then the stat dump files (if anyone is still using those).

    Cheers,
    Louie

  2. #2
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,450
    Blog Entries
    13
    yes, I still use the stats files...

    http://stats.free-dc.org/stats.php?page=proj&proj=sob

    Let me know if you are going to change the format at all as it'll break my parser...

    Bok

  3. #3
    Quote Originally Posted by Bok View Post
    yes, I still use the stats files...

    http://stats.free-dc.org/stats.php?page=proj&proj=sob

    Let me know if you are going to change the format at all as it'll break my parser...

    Bok
    Cool. If you want to use the new unit, just use

    http://www.seventeenorbust.com/stats/textStats2.mhtml (new jEMs)

    instead of

    http://www.seventeenorbust.com/stats/textStats.mhtml (legacy cEMs)


    The new unit will be the only one displayed and used for ranking in the future so you may want to migrate when you get a chance. Really just have to use the new dump file and change references from cEMs to jEMs... but I understand it may be harder if you cache things too.

    Cheers,
    Louie

  4. #4
    For those of you interested in how these changes effects the aggregate stats:

    (old) cEMs


    (new) jEMs



    Because the formula to compute scoring were off by a pretty significant factor, the change is reflected in previous work being worth slightly more. Or in the case of *WAY* back work, significantly more... like 10x for the first year worth of stats.

    To be honest, I was actually worried that the stats were inflated even more than they were and that the project's true work-rate had collapsed even worse than it has in the past 3-6 months.

    Hopefully the faster, better client and the more fair scoring system will encourage more production in the near future to reverse that mini-trend.

    Oh, and ignore the stats going down to 0 on the far-right. I graphed it out into the future to March 31, 2009.


    Cheers,
    Louie

  5. #5
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,450
    Blog Entries
    13
    I'm not entirely sure how to handle this.

    I can easily change my parser, but am I right in thinking that most everyone's scores will change fairly significantly depending on when they did the bulk of their crunching ?

    Actually, scratch that, they already have !

    I've now jumped from 1.55P to 2.43P So it's still handling the old file ok.

    Am I right in thinking the team's scores have not been altered yet ?

    I'll change it to the new file in the next few days after the dust settles !

  6. #6
    OK, I'll bite - what's the "j" stand for?

    And as a relatively new contributor, I think I'm going to get knocked down quite a few notches at the next stats update. I guess that just means I get to pass the top-100 user milestone for a 2nd time!

  7. #7
    Quote Originally Posted by jjjjL View Post
    Hey guys,
    ...then the stat dump files (if anyone is still using those).
    /me does.. http://sob.qik.nl
    Now i also have to look for an simple solution to fox those scores.. thanks for the work :P
    Seriously, it's great to see so much activity from you these last couple of months, thumbs up !

    Quote Originally Posted by jjjjL View Post
    Cool. If you want to use the new unit, just use
    http://www.seventeenorbust.com/stats/textStats2.mhtml (new jEMs)
    I was already usning those.. itn't it possible to create a new one like textStats3.mhtml and keep the old one, so I/we can switch over in in a more controlled environment ?
    Last edited by DukeBox; 03-23-2009 at 04:41 PM.

  8. #8
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,450
    Blog Entries
    13
    I know I wasn't seeing things so I guess the values got passed to the textStats.mhtml accidently (albeit briefly) as they are back to normal now.

    I've just switched to the textStats2.mhtml so we'll see the next update.

    I still think the team values need adjusting as well....

    Bok

  9. #9
    Quote Originally Posted by enderak View Post
    OK, I'll bite - what's the "j" stand for?

    And as a relatively new contributor, I think I'm going to get knocked down quite a few notches at the next stats update. I guess that just means I get to pass the top-100 user milestone for a 2nd time!
    cEMs stood for "corrected Exponentiation+Modulus"

    The j in jEMs is for "jiggered"


    Cheers,
    Louie

  10. #10
    I like how the huge "double check dips" in the system-wide graph have vanished. That's a good sign that the stats unit is actually correct now... that even though the n-value composition of the test pool drastically changed at some points, the stats output didn't change.

    Also, it looks like 36.3% of all work done was previously under-counted before but is now validly credited. This means people who did work a long time ago or did a lot of double-checking will now get credit more in line with the true amount of computing power they expended.

    Cheers,
    Louie

  11. #11
    I like what I see!


  12. #12
    For people who want to know the difference between the cEMs and jEMs migration: http://sob.qik.nl/index3.php?date=2009-03-24
    The date is pure fictional, it's the stats difference between 2009-03-23 19:30:11 and 2009-03-23 19:45:50 when the change took place.

  13. #13

    DC-Vault all messed up now

    There are many teams in the DC-Vault wondering what is going on since the last update.
    Does the DC-Vault admins need to change something or are there stats correct?
    Last edited by Razor_FX_II; 03-24-2009 at 05:57 PM.

  14. #14
    Administrator Bok's Avatar
    Join Date
    Oct 2003
    Location
    Wake Forest, North Carolina, United States
    Posts
    24,450
    Blog Entries
    13
    I think the DC-Vault scripts are probably still correct. There was an adjustment in scores across the boards and it looks like it worked out more in favor of some of the teams below you, so I'm afraid your team lost some places.

    It increased the scores for some of the tests done with differing 'n' values. It has worked out more fairly now as per cpu time used for the numbers should be much more accurate. In a nutshell it means scores from way back when are now inflated to be on a par with scores being generated today.

    Luckily it looks like you can move up quite a lot of places quite quickly.

    Hope that helps.

    Bok

  15. #15
    Thanks Bok.
    This is like Congress changing the rules retro-actively on the T.A.R.P.

    Time to get moving up the charts again.
    Thanks again.
    Last edited by Razor_FX_II; 03-24-2009 at 05:57 PM.

  16. #16
    Sorry to go OT, but I didn't want to start a new thread for something so small, and who knows, maybe it's even related to jEMs.

    Anyone else been unable to contact the server since 7:00 this morning (PDT?)

    When I left for work, my computer was giving server too busy messages, and when I look at my (and other users) stats, it shows 0 jEMs/sec since this morning.

    Anybody know what's up?

  17. #17
    Quote Originally Posted by wolfemancs View Post
    Sorry to go OT, but I didn't want to start a new thread for something so small, and who knows, maybe it's even related to jEMs.

    Anyone else been unable to contact the server since 7:00 this morning (PDT?)

    When I left for work, my computer was giving server too busy messages, and when I look at my (and other users) stats, it shows 0 jEMs/sec since this morning.

    Anybody know what's up?
    The old client's server may have been down during that time. New server is running fine.

    Any reason you can't update to the new client software? It's posted on SB's homepage.
    http://www.seventeenorbust.com/

    Cheers,
    Louie

  18. #18
    .85*1 > 1*.5

    50% of the way through a 2 week work unit. . . Figured I'd upgrade when this one finished. Is there a better way to go about it?

  19. #19
    Quote Originally Posted by wolfemancs View Post
    .85*1 > 1*.5

    50% of the way through a 2 week work unit. . . Figured I'd upgrade when this one finished. Is there a better way to go about it?

    That's sound reasoning. Stick with it and finish then upgrade. The legacy server will still be around for awhile, although I will start emailing everyone about the software upgrade soon and may begin pestering those who don't upgrade with personal emails in a month or so.


    Cheers,
    Louie

  20. #20
    Member
    Join Date
    Jan 2003
    Location
    Norway
    Posts
    61
    There is a difference between the jEMs and cEMs user stats files. In the jEMs file it looks as if users that have switched teams is listed once for each team they have participated in, with the respective score for that team. In the old cEMs file, they are listed only once. At least my user and many others are listed twice.
    Is this change intentional?

    This seems to confuse the DC-Vault stats, which shows a significant drop in my score.

    EDIT: I now see that this issue is already posted in this thread
    Last edited by shauge; 03-25-2009 at 05:35 PM.

  21. #21
    Senior Member engracio's Avatar
    Join Date
    Jun 2004
    Location
    Illinois
    Posts
    237
    Quote Originally Posted by jjjjL View Post
    The old client's server may have been down during that time. New server is running fine.

    Any reason you can't update to the new client software? It's posted on SB's homepage.
    http://www.seventeenorbust.com/

    Cheers,
    Louie
    Louie,

    Got your email about the new client. How long do you plan to have the legacy client crunching and completely rollout to the new client? The reason I ask is majority of my crunchers are remotes and not sure when I will be able to access them or if at all. Thanks.

  22. #22
    I've just realised this has broken the stats signature, looks like I need to do some tweaking!

    Edit: It should now be working

    Last edited by Matt; 03-26-2009 at 04:39 AM.



  23. #23
    Quote Originally Posted by engracio View Post
    Louie,

    Got your email about the new client. How long do you plan to have the legacy client crunching and completely rollout to the new client? The reason I ask is majority of my crunchers are remotes and not sure when I will be able to access them or if at all. Thanks.
    The old server will be supported for a good long time to come for those who need extra time to upgrade. I'm sure most people will upgrade soon though since the new client is faster. But I understand there are reasons like yours where it will take longer to upgrade. Especially if their boxen are spread out or they have elaborate setups with SBqueue, etc.

    Cheers,
    Louie

  24. #24
    Senior Member engracio's Avatar
    Join Date
    Jun 2004
    Location
    Illinois
    Posts
    237
    Quote Originally Posted by jjjjL View Post
    The old server will be supported for a good long time to come for those who need extra time to upgrade. I'm sure most people will upgrade soon though since the new client is faster. But I understand there are reasons like yours where it will take longer to upgrade. Especially if their boxen are spread out or they have elaborate setups with SBqueue, etc.

    Cheers,
    Louie
    Coolest your the man!!! Let's go find that next prime.

  25. #25
    Quote Originally Posted by jjjjL View Post
    Wow. I would never have noticed this. Thanks guys.

    It's fixed now.

    The textStats2 should now behave exactly like textStats except for jEMs are cEMs.
    Louie, i think you just broke the stats.. it was working fine until your fix:

    Quote Originally Posted by Alien88 View Post
    Happy turkey day.

    Please look at this:
    http://www.seventeenorbust.com/stats/textStats2.mhtml

    It's the same idea as textStats.mhtml, however, this is grouped per user/team.

    EG:
    User 4044 Bok 1056123856 60 1068 1 29 1.12492637691787e+15 3076534010668.27 1074325317 1164262456
    User 4044 Bok 1056123856 201 432 0 0 43440648500256 0 1056124721 1075231400

    Show's Bok's info broken into each team he's been on.. same order of stats as textStats.mhtml.

    Does this work for you guys?
    Alien88
    Why did you change it and not just created a new one as mentioned before ?
    I'm creating SoB stats since 2003-06-14 but all is f*cked up now.
    http://sob.qik.nl

    Actually the stats (as i publish them on my site) are more accurate since points gathered in other teams are still counted for those teams.. the same as on seventeenorbust website itself.

    For example http://seventeenorbust.com/stats/tea...html?teamID=34 at the end of the page you have ex members, those scores are also added to the team score.
    Last edited by DukeBox; 03-26-2009 at 12:39 PM.

  26. #26
    Quote Originally Posted by DukeBox View Post
    Louie, i think you just broke the stats.. it was working fine until your fix:



    Why did you change it and not just created a new one as mentioned before ?
    I'm creating SoB stats since 2003-06-14 but all is f*cked up now.
    http://sob.qik.nl

    Actually the stats (as i publish them on my site) are more accurate since points gathered in other teams are still counted for those teams.. the same as on seventeenorbust website itself.

    For example http://seventeenorbust.com/stats/tea...html?teamID=34 at the end of the page you have ex members, those scores are also added to the team score.

    My bad. I didn't create the textStats originally so there has been a learning curve for me updating them. Also, I misunderstood your original request to "keep the old ones" meaning that you wanted a cEM version and a jEM version. I didn't know the textStats2 was being actively used.

    Can you use textStats3.mhtml instead of textStats2.mhtml? It looks to me like those 2 were nearly identical before I editied textStats2 to make it more like textStats. #3 already uses jEMs now too. Let me know if that works.

    Cheers,
    Louie

  27. #27
    Tnx it seems to work as it was..

  28. #28
    Quote Originally Posted by jjjjL View Post
    The old server will be supported for a good long time to come for those who need extra time to upgrade. I'm sure most people will upgrade soon though since the new client is faster. But I understand there are reasons like yours where it will take longer to upgrade. Especially if their boxen are spread out or they have elaborate setups with SBqueue, etc.

    Cheers,
    Louie
    One reason I can't upgrade all clients is the sheer number of shared libraries required by the new client. At least one of my crunchers will *never* be able to run the new client due to this. So I hope the old server will stay around for as long as possible.

  29. #29
    Quote Originally Posted by Vato View Post
    One reason I can't upgrade all clients is the sheer number of shared libraries required by the new client.
    I had some problems to.. but could fix it by copying all needed libs into the execution dir.

  30. #30
    Since I've been on a kick lately of updating long broken things, I decided today to update the cEMs -> THz and P90 year conversion factors.

    I did some benchmarking and back of the envelope math. It worked out that we were over-estimating both by almost exactly 8x (8.03...) so I updated the conversion factors. Part of that over-estimation was due to the change to jEMs so it was probably only off by around 5-6x before that... still a gross error... but now it was too gross to suffer any longer.

    Equivalent power (est.) 11.73 THz 10.84 THz
    Processing time used (est.) 428.6 P90-yrs 673914.0 P90-yrs
    Hope this doesn't bum out anyone who thought that the 2000 or so machine crunching SB were really running at a combined speed of 90THz.

    And for those of you who are still skeptical of even this reduced figure, those are P4 THz. So a modern Core2 Duo 1GHz might be something like 4.5GHz equivalent.

    Cheers,
    Louie

  31. #31
    P90s are so weak compared to what we have now that it probably would be better to use something like kilo-P90 years.

  32. #32
    Quote Originally Posted by jjjjL View Post
    Hopefully the faster, better client and the more fair scoring system will encourage more production in the near future to reverse that mini-trend.
    The last couple days of the system-wide graph look promising. Hopefully it will continue to climb!

  33. #33
    Quote Originally Posted by jjjjL View Post
    Since I've been on a kick lately of updating long broken things, I decided today to update the cEMs -> THz and P90 year conversion factors.

    (..)

    And for those of you who are still skeptical of even this reduced figure, those are P4 THz. So a modern Core2 Duo 1GHz might be something like 4.5GHz equivalent.

    Cheers,
    Louie
    Wouldn't it make more sense to either:
    - use P90 THz
    - use Pentium 4 years
    - put a note next to all THz figures that we're talking about P4 THz (just like it doesn't say "years", but always "P90 years")
    At the moment it strongly suggests you're using P90 THz (since it doesn't say otherwise and the line talking about P90s is right below it), while in fact it's not.
    I mean, since you're in the fixing mood anyway...

    ps. How about renaming the project into sixorbust? Would save a lot of typing and SoB would still be applicable!

  34. #34
    Quote Originally Posted by robvanwijk View Post
    ps. How about renaming the project into sixorbust? Would save a lot of typing and sob would still be applicable!
    lol

  35. #35
    BTW, I noticed that the Pending Tests section in the user preferences still says "cEMs/s":

    http://seventeenorbust.com/account/testsPending.mhtml

  36. #36
    Quote Originally Posted by umccullough View Post
    BTW, I noticed that the Pending Tests section in the user preferences still says "cEMs/s":

    http://seventeenorbust.com/account/testsPending.mhtml
    Thanks umccullough! I forgot to update that page -- it was still showing cEM/s.

    It's been updated now.

    Cheers,
    Louie

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •