Reply With QuoteHere Here!Originally posted by Dyyryath
The fact that he's given the whole stats thing as much consideration as he has puts him head and shoulders above his peers in other projects in my view.
I feel pretty open minded about the whole thing. Zero the stats, don't zero the stats, create an aggregate of the stats, give us stars for past work, they all work just fine for me. My only concern is how any given decision will impact the project by irritating users. FoBoT's right, no matter what choice is made, people aren't going to be happy.
If it were me, I'd probably do it something like this...
Each Phase becomes a game in the larger 'World Series'. Given future refinements and Howard's dedication to continually improving the client and algorithms, what makes us think this will be the last 'Phase' anyway? And even if it's not, does that have to be a bad thing?
I'd have a set of stats for each phase that's finished, active stats for the 'current' phase, and 'overall' stats that are the sum of all the project phases to date.
My way of doing things wouldn't be the simplest way, though, and I'm sure that there is a limit to just how much time/energy/resources Howard is willing/able to to put into the way stats are handled.
The fact that he's given the whole stats thing as much consideration as he has puts him head and shoulders above his peers in other projects in my view.
It'd be cool if there was a way that those of us willing to could help out with the 'official stats' in some way, but that's probably more hassle than Howard wants to deal with as well.
Here Here!Originally posted by Dyyryath
The fact that he's given the whole stats thing as much consideration as he has puts him head and shoulders above his peers in other projects in my view.
~~~~ Just Passin' Through ~~~~
For a bit of trivia, the OCworkbench poll has so far recieved 20 replies or 1/4 of our active users. 7 Want the stats zeroed (35%) 7 want a system so you can choose old&new OR just new (35%) and 6 want to have just the old stats (30%). Not a large number of replies, but given the wide range of users we have from all over the world, I feel these numbers would remain the same no matter how large the vote becomes. The results.....people want the old stats to continue to be updated (65%) and 70% want to see the new stats by themselves .
I am sure you can interpret this a million other ways but i do not want to know
As Fobot said, you will always make someone unhappy.
I guess I've got to stop over here more often. I don't care all that much if you reset or re-calibrate your scoring system but I do have a few observations, so bear with me...
I think you need to carry something forward. People want acknowledgement for time served and effort put forth. Just a week or so ago FoBoT was looking to find his reg. date for example. I am closing in on my first half billion; that's something I'll want to remember, and celebrate somehow.
Whatever you do, you are doomed to piss a few people off. If you turn this into too much of a democratic process, you will end up with a mediocre committee-type decision. At some point you are going to have to just go with your gut. Personally, I can't stay mad, for long anyway, with someone who does that.
My home team, OCN, is currently engaged in some SETI nostalga(sp). If you reset scores we will probably never see the top ten again, and that's Ok with me. I am currently positioned just past the top ten, and crunching at about that same level. If nothing happens to change the current scoring my next year will be spoent going up a few places, then going back down about the same...
HaloJones has been creeping up on me for quite a while now, if you blow the counts away just before he passes, I'll give you a shiny Looney.
Participants are always coming and going. For a relatively young project there are a fair number of flat-liners, even in the top ranks. Getting them off the active stats pages will be good in some ways, and bad in others. I tend to set goals for myself that relate to passing people, flat or otherwise. I have Data and Jodie to thank for my humongous electric bills as much as myself. If we all go back to zero, we will still have new people enter the project, and leave it as well; that's just life.
I would like a nice "group photo" at the end of the current festivities. This has been a good group to compete with, and that too, deserves some acknowledgement.
edited to add that Howard had better bring this to an end soon, or I will be knocking him out of his own top 10.
Last edited by TheOtherZaphod; 01-19-2003 at 08:23 PM.
Don't Panic
Y'see Howard. Zero the stats before I pass him and he loves you. But I'll be seriously p*ssedHaloJones has been creeping up on me for quite a while now, if you blow the counts away just before he passes, I'll give you a shiny Looney.
(Actually, on current production, I won't catch you, Mr !=Beeblebrox. And HaloJones was (will be?) definitely a girl (although I'm not? you will be!) )
Old stats still available, new stats available and a joint stats available. How can anyone be unhappy with that as a solution?
Your assertion B is false. People rarely complain when they see their computer producing better numbers. A while ago the client was updated and the new client generated about twice as many structures per CPU-hour. I don't recall anyone complaining about that. People like seeing their computers producing better numbers.Originally posted by FoBoT
howard is in a no win situation here
if he starts a new set of stats for DFII , X% of the current crunchers will be pissed off
if he comes up with some magic formula to equate DFII production to DF1 production, then X% of the current crunchers will say it is unfair because , either
A- the new scoring is biased towards the old phase 1 people or
B- the new scoring is biased towards the new phase 2 people
Thus, the assertion that Howard is in a no-win situation is also false. All he has to do is to make sure the new client produces better numbers than the old one.
OTOH, if word gets around that DF zeros the stats every now and then, a lot of stats-loving people who are looking for a project will look elsewhere. I like DF and will continue even if the stats are zeroed. But, back when I was deciding which project to move to, if someone had told me DF zeros it stats every now and then, I would probably have chosen some other project (or maybe even have stayed with Stanford's Genome).
I would like to see the cumulative stats updated as often as the stats are updated now. One of the things I really like about DF is that the stats are updated every 10 minutes like clockworks. And the cumulative stats are the ones I would be looking at. The ephemeral stats for the current phase would be of only passing interest to me.Originally posted by horus2
What I would like to see is 3 different stats databases. One with the first phase stats, one with the new phase and another one which is just a sum of the two. The final database would not even have to be updated as often.
what about a person that currently has 100 million under the old system, but has recently dropped from 7 PC's to use down to just one?Originally posted by AMD_is_logical
Your assertion B is false. People rarely complain when they see their computer producing better numbers.
their current rate under a scoring system that is relatively faster will be lower and thus they will lose position at a higher rate due to "the unfairness" of the new scoring system
if there is the smallest change, somebody will bitch about it, it seems to be human nature
if the only consideration is the least "noise" then that option might be the path of least resistance, but if what is best for the project overall is paramount, then perhaps another option will be chosen
luckily i don't have to decide, i am just another cruncher
Use the right tool for the right job!
As a long time lurker, dedicated F@H and G@H, I'm finally glad to see someone post some sense.Originally posted by Dyyryath
I feel pretty open minded about the whole thing. Zero the stats, don't zero the stats, create an aggregate of the stats, give us stars for past work, they all work just fine for me. My only concern is how any given decision will impact the project by irritating users. FoBoT's right, no matter what choice is made, people aren't going to be happy.
If it were me, I'd probably do it something like this...
Each Phase becomes a game in the larger 'World Series'. Given future refinements and Howard's dedication to continually improving the client and algorithms, what makes us think this will be the last 'Phase' anyway? And even if it's not, does that have to be a bad thing?
I'd have a set of stats for each phase that's finished, active stats for the 'current' phase, and 'overall' stats that are the sum of all the project phases to date.
<snip>
It is ridiculous to view any serious scientific DC project as never ending. As Dyyryath suggests treat is as a world series, or world cup. Set a date, tell us when the playoffs are and after the game we all shake hands and start over again. Perhaps with enough down time for a good disk defrag
Then do it every year.
Originally posted by TheOtherZaphod
Whatever you do, you are doomed to piss a few people off. If you turn this into too much of a democratic process, you will end up with a mediocre committee-type decision. At some point you are going to have to just go with your gut. Personally, I can't stay mad, for long anyway, with someone who does that.
wise words
i hope we can get through this and into the next step of the project with a minimal of bad feelings
Use the right tool for the right job!
Maybe you won't like it, but, the star WILL in effect represent the billion or so structures you have crunched in the past. So we won't be taking them away from you, simply presenting it in graphical formatOriginally posted by reader50
. A bonus spelling-bee star is not what I had in mind. Not even if it's in color.
Because evidently we plan ahead better than they do...Side note for Howard: Why do foldingathome.com, foldingathome.net, and foldingathome.org all point to the DF project? Shouldn't they point to Stanford?
We owned those before they even existed
Howard Feldman
If Dyyryath could announce that he will do old/new/joint stats, I think most people would be satisfied. Or course, we would all have to continue with our current handles but I have seen no intention for that to change.
This is simply about how to score isn't it? Some people will say "I'm #xx in the new stats" others will say "I'm #yy in the joint stats" What's the harm in that?
But who is gonna say "I'm #xx at DF and hey, I've got a little green star against my name!" ???
I'd definately be willing to do multiple sets of stats (phase 1, phase 2, total). It should be no problem at all. I don't know if it would really satisfy everyone since they wouldn't be the 'official' stats at the DF project website, but if it helps at all, I'll be happy to do it.Originally posted by HaloJones
If Dyyryath could announce that he will do old/new/joint stats, I think most people would be satisfied. Or course, we would all have to continue with our current handles but I have seen no intention for that to change.
This is simply about how to score isn't it? Some people will say "I'm #xx in the new stats" others will say "I'm #yy in the joint stats" What's the harm in that?
But who is gonna say "I'm #xx at DF and hey, I've got a little green star against my name!" ???
Actually, I don't know if they'd even be interested, but I'd probably be willing to provide either the software I'd be using to generate multiple stats sets to the project itself to use as they see fit, or to completely generate and host an 'official' version of the stats (much like we do this forum) themed to match the rest of the DF site.
I don't forsee the 'handle' thing being a problem. If by 'handle' you mean your username, then you could change it to whatever you want. Your unique ID would remain the same and I could still find you in the old & current stats (to create the 'total' stats) with no problem. If by 'handle' you mean the handle we use to identify our clients to the project, then it would work just like it does now. If you use a different handle on some clients, you're creating a new user which will be tracked differently in the stats than your original user.
Well, I've posted twice already. Seems to have made zip difference. (ie: ignored) The stats will be zeroed and replaced with little stars.Originally posted by reader50
Wish more people would jump into the "No" side of things.
The last time I checked, little colored stars only meant something in Kindergarten.
I think it rather odd that this much "opinion" could be generated by a quantity stat!
No one has even mentioned that the most important stat gets zeroed at every protein change. No stats page ( that I use ) keeps track of your best fold to date ( only current protein ). Yet no one seems to be getting worked up over that stat!
Sorry, just thinking out loud.
PinHead, I think you are looking for a graph of your Best RMSD over time? If so...
http://teamstats.macnn.com/dfold/sta...low=1032600458
Last edited by reader50; 01-20-2003 at 10:57 PM.
Get Addicted
Bookmarked that one!:shocked:Originally posted by reader50
PinHead, I think you are looking for a graph of your Best RMSD over time? If so...
http://teamstats.macnn.com/dfold/sta...low=1032600458
That is more stats than any one person should be allowed.
Thanks for the link.
Well, after all the posts, all the fuss, it seems obvious that users still want the stats to continue on, but also want a separate count of the new format. (Our little survey has the combined system at 40% now, though it has way too small a user base to be anything but a sample).
The solution seems simple. Distributed Folding has the new stats with the stars, and Dyyryath looks after the combined system..after all, his stats site is THE site for stats, having put a lot of work and effort into it. Most would agree that they go to his site to check their personal and Team progress.
The only piece of the puzzle missing is a system to ensure the new units are within the range of the old client so passing/being passed is not abnormally slow or fast..it would be best if the new stats system is calculated to do this rather than devising a conversion routine.
hello everybody, I had a brain storm come to mind of how we can reset the stats, but maybe make people less angry about it. There are a zillion posts on this subject, and didnt have time to read them all, but i figured i'd put in my idea and see if it makes sence to everyone. I read that each machine would have to have its own ID, drastically slowing down the production for users that have DF running on multipule machines, what if on the stats list, next to your user name, you have a number.... kind of like the star idea (which is great) That number represents the number of machines that belong to you. for the individual id's, set them up in kind of like a Personal Team, not a group team, but saying all these ID's belong to you. Than if you wish you could join a larger Group team, and have full statistics on how many accuall machines are being used in that team. This would let everyone start off strong, and the people who would be effected by the zeroing of the stats would still have their edge. I hope i didnt leave anythign out, but i'm sure if i did, i'll be hearing for you.
I think I would like to put some links to the 'unofficial' stats pages on the main web site, with your consent of course. Im sure there are people who would like to make use of them but do not know about them yet. I will only do this for people keeping stats for all participants though, not if you're just tracking your own team. Let me know the exact link you want me to put up and I'll put it right under the stats button on the front page, or something like that and call it Unofficial Stats.
Thanks.
Howard Feldman
I certainly don't mind you linking to my pages. The whole point in building them is to help generate interest in the project, so...the more the merrier.
Calling them 'Dyyryath's DF Stats' or 'Arachnid Stats System' or something similar would be fine. The link should probably go to the status page at http://stats.zerothelement.com. That page links down into all of the stats pages and generally contains information about the system. It'll soon contain some 'help' information as well.
If there's anything you'd personally like to see added or changed, let me know. As I said, the whole point is to help support the project.
Personally I don't have a preference, but I can see the concerns that others have.
I can understand the idea of resetting the stats from just a project effeciency point. I also think Howard's idea of tracking the new stats but have a way of adding in the old stats for a total production # is good too. It is very important I think to have some method of giving credit for previous efforts from phase I.
Still the entire idea will have an impact on a lot of individuals. dFold is not the only project facing this kind of a situation. SETI is facing the same situation with the pending release of SETI2 and BOINC. Other projects may eventually face the same issue as well. As software and hardware technology improves, it may be periodially necessary to so radically revamp platforms that it becomes necessary to also change how results are tracked.
I vote for a reset for the new, but with a total production "number" not stars, that also reflects what both individuals and teams have previously contributed.
Also, when the change is made, ensure that clients are available for all of the major OS platforms at the same time so that no group or individuals are left at a disadvantage.
Last edited by Shaktai; 01-21-2003 at 01:25 PM.
I'm fine with that, but I don't know if we are tracking every team right now. I'll see if it can be done, not sure if the server can handle the load or not.Originally posted by Brian the Fist
I think I would like to put some links to the 'unofficial' stats pages on the main web site, with your consent of course. Im sure there are people who would like to make use of them but do not know about them yet. I will only do this for people keeping stats for all participants though, not if you're just tracking your own team. Let me know the exact link you want me to put up and I'll put it right under the stats button on the front page, or something like that and call it Unofficial Stats.
Thanks.
I know we are running them for all the major teams.
I'll tell reader50 to stop by.
Scott, let me know when you plan to add the $15K of extra servers, and the SCSI RAID 10. (reader50 rubs hands together)
We do not qualify. We track 14 teams today, and expect to track 10 to 50 teams in full detail at any given time. Even if we wanted to track everyone, our server capacity is perhaps 10% of what we would need. Should this change in the future, we can certainly give Howard a call.
Never. Raid5 I am planning on, but I'm not getting raid10.Originally posted by reader50
Scott, let me know when you plan to add the $15K of extra servers, and the SCSI RAID 10. (reader50 rubs hands together)
Ok.We do not qualify. We track 14 teams today, and expect to track 10 to 50 teams in full detail at any given time. Even if we wanted to track everyone, our server capacity is perhaps 10% of what we would need. Should this change in the future, we can certainly give Howard a call.
You Mac folks do have some great stats, I checked them often when I was with Ars TSF. I have since moved to ExtremeDC and I do miss your stats, but I think overall it would be great if the official page listed the "unofficial" stats that have been set up. All the effort that has been put into those stats should be recognized.
I think the analogy of what SETI is coming up on was a good one. When a phase ends, we should start tracking over again. I believe I speak for ExtremeDC's folding team when I say that we are in favor of a new system, but will continue with the project either way. After all, it's only stats.
ExtremeDC...Do YOU Have What it Takes?
I'm with reader50 on this one. I don't feel that zeroing the stats is a fair approach to this situation. I put some major effort into really improving my stats over the summer while I had access to machines that are normally not mine to use. Trashing my stats is not something I'd like to see happen. I worked for those stats, as I'm continuing to work on improving them right now. If my stats are going to just vanish, I have much less incentive to dedicate CPU time to dFold as it currently stands when there are other projects whose stats have not been deleted in years.
And then, when phase3 starts, what's going to happen? Will the stats be deleted again? I just want to have some ranking system available to me that displayes my total, all-encompasing dedication to the DF project. It's just not fair to delete the stats I've been working on for so long.
I wouldn't mind having a new stats category for phase 2, as long as the main category of stats was merely phase1 + phase2. That way, if people are interested, they can see the difference between the two. But, the aggregate stats should still be the deciding factor in determing ranking.
And, I'm sorry, but I think colored stars next to our names displaying our "overall stat total" is ridiculous. Just keep the numbers as they are! Or even edit them so they are more fair/compatible with how the new stats will be. If it is going to take 10 times as long to get 50 stat points in the future, then multiply the new stat number by 10! Or even divide our current totals by 10. Just keep it fair, and let those of us who have really worked on our stats to keep them.
Maybe even go a Folding@Home route where 1 stat unit is equivalent to maybe 1 minute of crunching time on a 1GHz P3. That would make the stats fair forever, through any changes, and I'm sure you could figure out some way of converting the current stats over to something like that.
Crunch Something
FlowerKid, here
The_Equivocator: I think the problem is that it is very difficult (if not impossible) to make it completely fair.
While I agree it is impossible to find something completely 100% fair, I still agree that in this situation, there is an A fairer than B.Originally posted by pointwood
The_Equivocator: I think the problem is that it is very difficult (if not impossible) to make it completely fair.
Crunch Something
which is to be more fair to those already crunching or those who starts now?
What is best for the project? What will make the project grow most?
I think that it is fair to both parties to just keep the stats. No new members are going to say, "This isn't fair! Why do I have 0 stats points even though I haven't yet contributed any processor time to DF? These other people have tens of thousands of stats points and have dedicated hundreds of hours of CPU time and they have higher stats than me? Unfair!"Originally posted by pointwood
which is to be more fair to those already crunching or those who starts now?
What is best for the project? What will make the project grow most?
As long as a scale is set so that the stats gained from 100 hours of computing time in phase 1 = the stats gained from 100 hours of computing time in phase 2, everyone should find that fair.
Crunch Something
Just as a little food for thought I did a quick, counting-on-my-fingers-type, survey of the current project production. (forgive the rough math, but I really didn't do more than just glance at the numbers)
The top five contributors produced over 10% of last weeks total.
Going down thru the top twelve or so, you get about 20%.
It takes the top 30 to get 1/3 of the production.
Just about the top 80 contributors are needed to get to 50%.
The top 200 users did just over 2/3 of the weeks work.
Frankly, I was surprised to see that it wasn't more heavily weighted towards the top. If this were a game-show I would have guessed that the top 50 players did half the work, and that the top 100 did two thirds or more.
Please draw your own conclusions, but to me this says that even though heavy hitters make an obvious difference to the project, that more work than you would think is done by individuals, most running just a single machine.
Don't Panic
It is not out of the question that WE could host 3rd party stats on our servers, provided you were willing to give us your code of course, and provided it didn't eat up a lot of our bandwidth which is all that we are short on. We have no shortage of CPU or disk space reallyOriginally posted by reader50
Scott, let me know when you plan to add the $15K of extra servers, and the SCSI RAID 10. (reader50 rubs hands together)
We do not qualify. We track 14 teams today, and expect to track 10 to 50 teams in full detail at any given time. Even if we wanted to track everyone, our server capacity is perhaps 10% of what we would need. Should this change in the future, we can certainly give Howard a call.
Howard Feldman
i think that is why there is a "distributed" in "distributed computing"Originally posted by TheOtherZaphod
Please draw your own conclusions, but to me this says that even though heavy hitters make an obvious difference to the project, that more work than you would think is done by individuals, most running just a single machine.
if the project grows in participants, those number will continue to skew away from the "Big" producers and even more heavily towards the "little guy" , there just aren't that many people with access to large numbers of PC's (or the fortitude to stick with baby sitting large # of PC's, it takes a lot of time/effort, especially if the box isn't directly connected to the internet)
Use the right tool for the right job!
That's an incredibly generous offer. It's not something that I would need to take advantage of (my systems already run on several load-balanced server class systems), but for those who need it, that could be a real boon.Originally posted by Brian the Fist
It is not out of the question that WE could host 3rd party stats on our servers, provided you were willing to give us your code of course, and provided it didn't eat up a lot of our bandwidth which is all that we are short on. We have no shortage of CPU or disk space really
It's certainly not something I can think of any other project offering to do...damn, that was cool, Howard!
TheOtherZaphod: thanks for doing that analysis, I hadn't taken the time to do one in a LONG time.
As the project grows the trend should continue to be towards the masses doing more and more work in comparison to the 'big hitters,' as someone else pointed out.
TSF was able to finally catch Free-DC and KWSN! because Ars just brought in so many users. As the project has matured and as it gets additional publicity, this will continue to be the case, for the project as a whole and for teams as well.
Especially with other projects ending (RC-5, ECCp109, GAH1 [not official, but as good as dead] and soon to be SETI 1).
'tis a good thing not to rely on a small number of people for production.
BTW, Douglas Adams fan?
Of course not, especially not if the new WU's are a bit faster, but will some of the old users not complain then?Originally posted by The_Equivocator
I think that it is fair to both parties to just keep the stats. No new members are going to say, "This isn't fair! Why do I have 0 stats points even though I haven't yet contributed any processor time to DF? These other people have tens of thousands of stats points and have dedicated hundreds of hours of CPU time and they have higher stats than me? Unfair!"
Which is exactly what will be difficult (if not impossible) to do. Please correct me if I'm wrong.As long as a scale is set so that the stats gained from 100 hours of computing time in phase 1 = the stats gained from 100 hours of computing time in phase 2, everyone should find that fair.
Have you read the whole thread? It think this has already been discussed (without a conclusion though).
I think it is fair to say that the issue of zeroing the stats has been discussed ad nauseum
Seriously though, it will be implemented in such a way that any or all of the suggestions discussed COULD be implemented, at any time after the fact. So regardless of what we choose to do, it may change or be enhanced if need be. The beta testing will hopefully identify any potential problems as well. So in the words of another wise man 'Don't Panic'.
Apparently no one has any other issues which was the original question of this thread, which is a good thing I guess. I'll be doing some serious alpha testing next week and hopefully have it ready for distribution shortly thereafter. The initial beta will likely be for Linux and Windows only, and only teh text client, but all supported OSes will be released together when it becomes official of course.
In response to Zaphods analysis: thanks, that was cool, and pretty much what we expected. That's exactly why we made the screensaver cooler, trying to attract more of the 'little guys'. After all, no matter how many computer lemonsqzz, or M2K1Guy or Brian the Fist can scrounge together, it will still be piddly compared to the ultimate potential available from 'average Janes and Joes' running the screensaver on their computers, and it is this sort of audience who we have to try to appeal to now to continue to swell our ranks (since we've cornered the maket on you hard-core folks already). The most difficult task will be making this sort of user aware of our project and its goals in the first place, so the more attention we can draw to our project, the better.
P.S. some CASP5 results are up on the web site
Howard Feldman
As an "Old User", I think that I would prefer my stats to be worth less, as opposed to my old stats being worth nothing at all.Of course not, especially not if the new WU's are a bit faster, but will some of the old users not complain then?
As long as a scale is set so that the stats gained from 100 hours of computing time in phase 1 = the stats gained from 100 hours of computing time in phase 2, everyone should find that fair.
Please correct me if I'M wrong, but no attempt was ever made to equalize stats since the begining of the DF project. I.E., any given machine, running since the inception of DF, might have folded 4500 folds per hour, on one target. 11,000 fold per hour, on the next target. And 3,200 fold per hour on another target.Which is exactly what will be difficult (if not impossible) to do. Please correct me if I'm wrong.
Posting Permissions
