Results 1 to 37 of 37

Thread: Unfair stats system?

  1. #1

    Question Unfair stats system?

    I was just looking through the stats and something dawned on me. The stats system is severly unfair when the proteins are so different.

    For example UserA has an output is 50,000 a day on this protein. Someone in front of him has an output is 100,000 a day.

    For 10 day UserA crunches 500,000 points

    UserB crunches 1,000,000 points.

    Now say UserA adds a few computer and is now putting out 150,000 points a day.

    Under tha same protein the UserA should catch up to UserB in 10 days right?


    So now the protein changes and the output drops 50% for everyone.

    Now it will take UserA 20days to catch up to UserB!!!!

    As this should show people with more power have a HUGE advantage over people with less power. This goes for teams aswell. Teams with more power can rack up a serious lead with a fast protein and pretty much dust the competition because the competition now has to work twice as hard to catch up each time the proteins change.

    Did that make any sence or should I put down the crack pipe?
    Crunchin D.F. for www.procooling.com

  2. #2
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    If you fail to adjust your notion of fairness to the reality of the Universe, you will probably not be happy.

    "When it rains soup, grab a bucket!"

  3. #3
    Fixer of Broken Things FoBoT's Avatar
    Join Date
    Dec 2001
    Location
    Holden MO
    Posts
    2,137
    Shocking!


    oh, wait, it has been that way since this project started, coming up on two years ago

    it promotes continued participation, if you slack off, you might miss out on a fast protein

    Use the right tool for the right job!

  4. #4
    Senior Member
    Join Date
    Mar 2002
    Location
    MI, U.S.
    Posts
    697
    How would you propose it be fixed? Give extra points to people with small farms? That's just ludicrous, even though I would get more points... (I only have 2 machines crunching most of the time, a P3-800 and a Barton XP2500+; there's the occasional output from a P4-1.8 at work, but that's very sporadic).

    Reality check... if you put more effort into something, you will get more out of it. That's just a fact of life, and regardless of how much liberals (and lately, a lot of conservatives... which is pretty sad) in the U.S. House and Senate have tried to change it with government handouts, it simply isn't going to change.

    Paratima -- would you mind if I used that quote (the "if you fail to adjust..." one) in a sig or two? With proper credit, of course.
    Last edited by bwkaz; 08-30-2003 at 09:09 AM.
    "If you fail to adjust your notion of fairness to the reality of the Universe, you will probably not be happy."

    -- Originally posted by Paratima

  5. #5
    The system folding@home uses rewards a different number of points for each different protein. They calibrate againast a reference machine. To quote directly from their FAQ:

    How do you determine how many points a work unit is worth? Before putting out any new work unit, we benchmark it on a dedicated 500MHz Celeron machine (this machine does not have SSE/3DNow). We plug the results of this into the following formula:

    points = 2 * multiplier * (daysPerWU)

    where daysPerWU is -- no surprise -- the number of days it took to complete the unit.
    I think it's a better scoring system, as it rewards point based on the speed of the hardware and the time put in folding. Their Work Units (WU) often take days to complete, where a Distributed Folding generation may only take minutes, but the scoring principle can still be applied.

  6. #6
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    Gosh, did I create some Deathless Prose©? I thought it was just a throwaway line!

    By all means, Kaz, help yourself!

    Jaydee116, if you're still with us, and I hope you are, I didn't mean to sound so insulting, and I apologize. It's just that this IS the way of the world, people with more resources tend to get more results. And yes, processing a fast protein DOES increase your advantage. But just a few months ago, someone was complaining that those of us who started earlier in the project (this was back in Phase I) had piled up a huge point advantage! Well, yeah! That's the incentive to start early and work consistently. Doing more work gets more reward! Like FoBoT said, that's the incentive. So.... rack up as many points as possible while they're easy to get. That's the cheese at the end of the maze! The longer you work the project, the more it all tends to even out.

    I just now took a peek at your stats. 12 million points at this stage of the game is a proud number! Build it up... Keep on crunching!
    Last edited by Paratima; 08-30-2003 at 12:49 AM.

  7. #7
    I was just making an observation. Wasn't complaining. Can get out of defensive mode already....

    I do this more for the project than the stats otherwise I would have stuck with SETI or Folding@Home. I see more potential in this project than others and that's why I will stick with it no matter how messed up the stats get. Non the less I like the competition as much as the next guy though. Just bringing it up for discussion in case there is a better way to do stats that will not mess up the real project.

    Have a good weekend guys/gals!
    Crunchin D.F. for www.procooling.com

  8. #8
    Fixer of Broken Things FoBoT's Avatar
    Join Date
    Dec 2001
    Location
    Holden MO
    Posts
    2,137
    it makes it more interesting, keeps you on your toes

    stanford/F@H has plenty of other problems
    Use the right tool for the right job!

  9. #9
    Originally posted by jaydee116
    I was just making an observation. Wasn't complaining. Can get out of defensive mode already....

    I do this more for the project than the stats otherwise I would have stuck with SETI or Folding@Home. I see more potential in this project than others and that's why I will stick with it no matter how messed up the stats get. Non the less I like the competition as much as the next guy though. Just bringing it up for discussion in case there is a better way to do stats that will not mess up the real project.

    Have a good weekend guys/gals!
    Don't underestimate the power of stats. I bet there would only be about 10% of the contributors if it was not for the competition for points, from my reading of several team forums there is a large points contribution from gamers in distributed computing challenges, it's nice to think it's just for the science, but I bet that only plays an initail small part in selecting which distributed folding project to run with.

  10. #10
    OCworkbench Stats Ho
    Join Date
    Jan 2003
    Posts
    519
    We at OCWorkbench Fold for the Fun & Taunting, and as a way to optimize our systems and squeeze the last breatrh of life from them. The Stats System can never be perfect, but what it is, is consistent . Phase II is pretty ugly to older computers, but we still use them, especially if we have 20-30 of them...strength in numbers. The trick is to hang in there for the slow Proteins and use the fast ones to make your move..and if that fails, busk at the mall until you can afford a Farm
    I am not a Stats Ho, it is just more satisfying to see that my numbers are better than yours.

  11. #11
    Fixer of Broken Things FoBoT's Avatar
    Join Date
    Dec 2001
    Location
    Holden MO
    Posts
    2,137
    Originally posted by erk
    Don't underestimate the power of stats. I bet there would only be about 10% of the contributors if it was not for the competition for points, from my reading of several team forums there is a large points contribution from gamers in distributed computing challenges, it's nice to think it's just for the science, but I bet that only plays an initail small part in selecting which distributed folding project to run with.
    i am only a moderate stats whore, i can go a whole day without checking Dyyryath's stats, ...




    once in a while
    Use the right tool for the right job!

  12. #12
    Member lemonsqzz's Avatar
    Join Date
    Sep 2002
    Location
    Montain View, CA
    Posts
    97
    I'm just here for the chicks

  13. #13
    Originally posted by FoBoT
    i am only a moderate stats whore, i can go a whole day without checking Dyyryath's stats, ...




    once in a while
    Geez your a better man than I, an hour or two infont of the PC without checking the stats, and I start to go cold turkey.

    BTW. "Stats whore" is an interesting term, a friend of mine reckons that men only get into these distributed computing challenges to get a bigger "e-penis". It's all starting to make some warped sense.

  14. #14
    OCworkbench Stats Ho
    Join Date
    Jan 2003
    Posts
    519
    There are "Chicks" here
    I am not a Stats Ho, it is just more satisfying to see that my numbers are better than yours.

  15. #15
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    Originally posted by lemonsqzz
    I'm just here for the chicks
    Good to hear from #2 on the stats.

    I don't know what you guys are talking about!
    The stats systems provide a most important tool for you to keep an eye on your production, to ensure all your boxen are crunching.

  16. #16
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    Originally posted by IronBits
    I don't know what you guys are talking about!
    Yeah! RIIIIIIIIGHT!

  17. #17
    Senior Member
    Join Date
    Mar 2002
    Location
    MI, U.S.
    Posts
    697
    Originally posted by Paratima
    Gosh, did I create some Deathless Prose©? I thought it was just a throwaway line!
    It would seem you did.

    Thanks.
    "If you fail to adjust your notion of fairness to the reality of the Universe, you will probably not be happy."

    -- Originally posted by Paratima

  18. #18
    Well there isn't a perfectly fair solution. The current setup rewards people for sticking with the project over time.

    Part of the fun with the current system is that point outputs are widely variable. It keeps the project fun.

    If it were decided that the points needed to be more consistent, one way to do it would be to include the protein length in the calculations. But IMO that would be taking a lot of the fun out of it.

  19. #19
    Senior Member
    Join Date
    Jan 2003
    Location
    North Carolina
    Posts
    184
    In phase 1, the "stats" were the number of structures, so there was no good way to adjust for fast vs slow proteins.

    Now, however, you get 50*sqrt(n) for generation n. That "50" could be replaced by another number. Each protein could be benched, and then the "50" could be replaced by a number that reflected the results of the bench.

    The benching would be tricky, as there is a huge random variation between runs depending on how much the folding gets stuck. It should be possible to get a rough idea of how fast a particular protein is running, though.

  20. #20
    Administrator PCZ's Avatar
    Join Date
    Jun 2003
    Location
    Chertsey Surrey UK
    Posts
    2,428
    AMD_is_logical
    I think you have come up with an excellent solution to the variation in scores between different proteins.

    But do you make the slow protein score higher or the fast protein score lower ?

  21. #21
    Keep the stats as it is
    If someone isn't running from the start and are missing the fast ones, it's their own fault
    Proud member of The Genome Collective

  22. #22
    25/25Mbit is nearly enough :p pointwood's Avatar
    Join Date
    Dec 2001
    Location
    Denmark
    Posts
    831
    /me agrees with HansArne
    Pointwood
    Jabber ID: pointwood@jabber.shd.dk
    irc.arstechnica.com, #distributed

  23. #23
    Fixer of Broken Things FoBoT's Avatar
    Join Date
    Dec 2001
    Location
    Holden MO
    Posts
    2,137
    Originally posted by HansArne
    Keep the stats as it is
    If someone isn't running from the start and are missing the fast ones, it's their own fault
    it adds a strategic layer to the project (from a stats prospective)
    vs. a purely tactical approach

    if you "normalize" stats for protein speed, then the strategic part of deciding when to "pour on the gas" goes away and it becomes a strictly "i have XXX Ghz and will always make more points than those with XXX/2 Ghz"

    look at SETI (in terms of points), right now somebody with 1,000 points can't reach someone with 100,000 points unless they aquire a 300Ghz horde

    oh, i guess that was the point of the guy that started this thread

    i would leave it , and at most reconsider this issue when phase III comes around
    Use the right tool for the right job!

  24. #24
    Senior Member
    Join Date
    Jan 2003
    Location
    North Carolina
    Posts
    184
    Originally posted by Galuvian
    The current setup rewards people for sticking with the project over time.
    The current setup rewards people who crunch for more than one project, and who can move boxes to DF when the protein is fast, and move to other projects when the protein is slow.

    For instance, suppose the protein is slow half the time, and 4 times faster the other half. Someone who put 2 boxes on DF while it's fast and no boxes while it's slow would end up with 60% more stats than someone that always had one box on DF, even though they're both contributing the same number of cycles.

  25. #25
    Fixer of Broken Things FoBoT's Avatar
    Join Date
    Dec 2001
    Location
    Holden MO
    Posts
    2,137
    Originally posted by AMD_is_logical
    The current setup rewards people who crunch for more than one project, and who can move boxes to DF when the protein is fast, and move to other projects when the protein is slow.

    For instance, suppose the protein is slow half the time, and 4 times faster the other half. Someone who put 2 boxes on DF while it's fast and no boxes while it's slow would end up with 60% more stats than someone that always had one box on DF, even though they're both contributing the same number of cycles.
    uh, so if you leave the 2 boxes on DF ALL the time, don't you end up in front of both those guys?

    i was referring to a group/team situation, an individual will always get the most DF points by putting all resources on DF and running 100% on fast and slow proteins

    so in your example, the guy with more boxen gets more stats isn't that the way its supposed to be?
    Use the right tool for the right job!

  26. #26
    Senior Member
    Join Date
    Jan 2003
    Location
    North Carolina
    Posts
    184
    Originally posted by FoBoT
    uh, so if you leave the 2 boxes on DF ALL the time, don't you end up in front of both those guys?
    Yes, but not by much. Assuming the 1:4 slow:fast ratio in my example, having boxes on DF half the time gives you 80% of the stats of having them on all the time, provided they are on DF for fast proteins and off DF for slow ones. Someone who crunches more than one DC project will be delighted that they can spend half their time crunching some other project, yet get 80% of the stats that they whould have gotten from crunching DF all the time.
    so in your example, the guy with more boxen gets more stats isn't that the way its supposed to be?
    No, stats should be proportional to cycles contributed. Using twice as many boxes for half the time should not give higher stats because the number of cycles being contributed is the same.

  27. #27
    Fixer of Broken Things FoBoT's Avatar
    Join Date
    Dec 2001
    Location
    Holden MO
    Posts
    2,137
    everyone is free to leave what they have on DF all the time, your scenario is only "unfair" if you assume that in one situation a person stops doing DF for a portion of the time in question

    apples =! oranges, big news

    "the number of cycles being contributed is the same" , but they aren't being applied to the same work/protein

    again, it is everyone's choice to run DF for a given protein or skip one or never skip one

    Use the right tool for the right job!

  28. #28
    Member lemonsqzz's Avatar
    Join Date
    Sep 2002
    Location
    Montain View, CA
    Posts
    97
    Wow.. there's another project I could be working on besides this ??.. anybody got a URL ???


  29. #29
    OCworkbench Stats Ho
    Join Date
    Jan 2003
    Posts
    519
    I am not a Stats Ho, it is just more satisfying to see that my numbers are better than yours.

  30. #30
    Someone who crunches more than one DC project will be delighted that they can spend half their time crunching some other project, yet get 80% of the stats that they whould have gotten from crunching DF all the time.
    That is good for the project. Some of those computers would not be used for DF. Now we get a few extra computers every time it is a fast protein.
    Proud member of The Genome Collective

  31. #31
    Originally posted by Grumpy
    www.foldingtomakepinkbunnies.org

    The site appears to be down Sounds like a great project, you got any details?

  32. #32
    Senior Member
    Join Date
    Jan 2003
    Location
    North Carolina
    Posts
    184
    Originally posted by lemonsqzz
    Wow.. there's another project I could be working on besides this ??.. anybody got a URL ???

    http://www.aspenleaf.com/distributed...-projects.html

    Groups like FreeDC, Ars Technica, and Dutch Power Cows have teams on a number of different projects.

    IMO, If DF succeeds in finding a good way to fold proteins, then it will have been by far the most useful in the long run. It's not at all certain that they will succeed, though. F@H, OTOH, will produce useful results, but they already have so many boxen crunching for them that adding a few more has less of an impact. The other projects vary in how useful they are.

    There are other considerations. For instance, DF runs faster on machines with 256MB (or more) of memory, it runs on a wide variety of machines, and it can run offline. F@H wants to be online, and runs faster on machines with SSE.

  33. #33
    OCworkbench Stats Ho
    Join Date
    Jan 2003
    Posts
    519
    And AMD SSE support needs to forced on F@H to make it work. 3D Now is a lot slower

    I always like to go with the UnderDog
    I am not a Stats Ho, it is just more satisfying to see that my numbers are better than yours.

  34. #34
    Senior Member
    Join Date
    Jan 2003
    Location
    North Carolina
    Posts
    184
    Originally posted by Grumpy
    And AMD SSE support needs to forced on F@H to make it work. 3D Now is a lot slower
    Yes, the F@H client needs the -forceasm switch to use SSE, and the DF client needs the -rt switch to make use of more memory.

  35. #35
    OCworkbench Stats Ho
    Join Date
    Jan 2003
    Posts
    519
    I was being sarcastic with my Pink Bunnies Site in case people clicked on it, I should have checked in case it actually existed. Would have been a naughty site 4 sure
    I am not a Stats Ho, it is just more satisfying to see that my numbers are better than yours.

  36. #36
    Originally posted by AMD_is_logical
    http://www.aspenleaf.com/distributed...-projects.html

    Groups like FreeDC, Ars Technica, and Dutch Power Cows have teams on a number of different projects.
    Of course Team rechenkraft.net has teams on other projects, too
    The German DC Community : Team Rechenkraft.net - Join now ! Rechenkraft.net

  37. #37
    Originally posted by Grumpy
    I was being sarcastic with my Pink Bunnies Site in case people clicked on it, I should have checked in case it actually existed. Would have been a naughty site 4 sure
    That was really cruel, Grumpy! I got all excited about a new DC project which sounded really interesting and worthwhile and then you pull away the carpet and say you were joking!

    How do Pink BUnnies fold anyways? Sounds like a great name for a team.... much better than OCWorkbench

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •