Page 2 of 3 FirstFirst 123 LastLast
Results 41 to 80 of 84

Thread: SB's stats: open discussion

  1. #41
    Senior Member
    Join Date
    Jan 2003
    Location
    UK
    Posts
    479
    Originally posted by alpha
    I second this. I always submit work from a .com host but am located in the UK.
    I also agree it would be so much better to use the user profile information to obtain country stats. Interestingly this info is already in the user public profile, problem is it's a freeflow text field rather than a selection from a list.

    I also live in the UK but will always be resolved as .com (or un-resolved depending upon where I am). Never .uk.
    Last edited by MikeH; 01-10-2003 at 12:05 PM.

  2. #42
    Sieve it, baby!
    Join Date
    Nov 2002
    Location
    Potsdam, Germany
    Posts
    959
    As it's said on the accordant page:

    "They are based on the two-letter country code of the IANA IP address registry of the address from which the work was submitted and could be slightly off."

    Some countries indeed seem to do more than they are credited on that page, but I'm not sure if it's better to let the members decide, as everybody would have to do it.

    Though an advantage would be that you can work abroad and still the country you're from get's credited.

    Maybe a solution already mentioned is best:
    Take IANA, unless there's a country entered in the user profile.

  3. #43
    Junior Member
    Join Date
    Jan 2003
    Location
    Netherlands
    Posts
    14
    I posted a reply here saying I didn't like .com being the US, but I see now that this has been fixed already, ignore.

  4. #44
    The IANA IP lookup is far more accurate than whois.

    For example, here is MikeH:

    62.190.xxx.xx uk (ip address censored to protect the innocent)


    I think this is better than asking users which country they are from. People will just list phoney countries like "Togo" and exotic ones when given a list of all of them. I've run other DC projects and I'VE DONE THAT. I think all of us have that hotmail account somewhere where we're "Julio Jamal" from the Easter Islands.

    And some users submit from multiple countries (such as Payam, from Iran and Pakistan). So in some cases, even if a "real" country is listed in his profile, it will still make the stats less correct.


    -Louie

  5. #45
    Senior Member
    Join Date
    Jan 2003
    Location
    UK
    Posts
    479
    Being "the inocent" here, can we just clarify which method is being used to generate the stats right now?

    This 62.190.xxx.xx address with a simple hostname resolution will result in a .com. So right now is this contributing to the US or UK stats.

    And thinking about it, I do agree that allowing the user to select is not so good. If only because users select one that no one else has, then, from their perspective it becomes they vs the world.

    So yes, please go with (or stick with) IANA IP lookup if it gives good results. Which brings me back to the question - which method is being used right now?

  6. #46
    Sieve it, baby!
    Join Date
    Nov 2002
    Location
    Potsdam, Germany
    Posts
    959
    Do quote myself:

    As it's said on the accordant page:
    "They are based on the two-letter country code of the IANA IP address registry of the address from which the work was submitted and could be slightly off."
    annexe: This page is meant. And it's on the bottom.

  7. #47
    Senior Member
    Join Date
    Jan 2003
    Location
    UK
    Posts
    479
    My humble apologies. I had read the "Take IANA, unless there's a country entered in the user profile" argument as if it was still being proposed. Missed the bit at the bottom of the country stats page.

    All clear now. Sorry.

  8. #48
    Sieve it, baby!
    Join Date
    Nov 2002
    Location
    Potsdam, Germany
    Posts
    959
    No apologies.
    The bottom of a long page is rarely looked at. That's why it's the perfect place for the most unpleasant parts of the page.

  9. #49
    I love 67607
    Join Date
    Dec 2002
    Location
    Istanbul
    Posts
    752
    Well, it seems that this IANA IP Lookup thing didn't work well for me.

    I think I am the only participant from Turkey and country stats were almost perfectly correlated with my user stats until a week or so ago, and Turkey's 24-hr rate suddenly dropped to 0 and never came back.

    This is my user stats. I'm showing the link because my user id is different from Nuri.

    I guess this is due to the change from whois to IANA. I guess this is bad luck on my side.

    I agree that allowing the user to select is not good and an IP lookup method should give better reults. So, since you agree that IANA lookup is far better, it seems there is nothing to do unless Louie manually specifies my IP adress to belong to Turkey.

    Anyway, just wanted to share that with you.

  10. #50
    Junior Member
    Join Date
    Jan 2003
    Location
    Netherlands
    Posts
    14
    Another data point to show that the current country stats don't work: I started a bit over a month ago. The first time I saw the country stats, the Netherlands were in #5, just like they're now.

    About 10 days ago, I started asking around, and some friends joined my team, PINO. They're all from the Netherlands. Together we do 1M+ cEM/s daily rate. The Netherlands are still at #5, and in fact the whole country's daily rate is now below our team's every day! Even if there are no Dutch here apart from us (and there must be quite a few more), the stats are still too low.

    Actually, the system's daily rate is at about 95 McEM/s, whereas all the daily rates in the country stats added together gives about 49 McEM/s...

    I'd much rather see a few fake Afghanistan entries than the current situation, I don't think a significant amount of people would fill in another country.

  11. #51
    Member
    Join Date
    Jan 2003
    Location
    Norway
    Posts
    61
    I second Scarblac.

    I submit from the one major provider in Norway at work and from another big provider at home and none are registered to my country.

    We are participating in this project for a good reason. Some friendly contest between countries would just be fun.

  12. #52
    Hater of webboards
    Join Date
    Feb 2003
    Location
    København, Denmark
    Posts
    205

    Angry Current country stats suck

    [QUOTE]Originally posted by kugano
    [B]A few things:

    1) I've already implemented one of the ideas I mentioned in my first post. A breakdown of work, rate, and so on by country is now available at this URL:

    http://www.seventeenorbust.com/stats/byCountry.mhtml

    And those suck!

    Apperently I've done 118% of the work done i Denmark.

    (And or top producer biff has done 1039% of the work done in Sweden? - At least his profile says he's swedish)

    The only thing I can see that would explain this, is that most of the computers I'm using are behind a firewall and have
    private IP-addresses (10.2.1.*). But the firewall have an public IP (which I'll gladly give you in private mail) which have a nice reverse DNS-looup ending in .dk.


    A feature I would like is an number telling me how much of the team (and country's - if those stats can be made meaningfull)'s work I've done.

    .Henrik

  13. #53
    I'm sort of late in posting this (I haven't been keeping up with forums since I stopped sieving) but I'd pay a good 25-30 $US (more than that and I'd have to start negotiating with my money-saving side) for one of those posters! I've got a nice blank spot on my wall that I think it would fit into quite nicely...

  14. #54
    Senior Member
    Join Date
    Jan 2003
    Location
    UK
    Posts
    479
    If I understand correctly how SB country determination works, if you go here and enter your IP addresses, this should indicate the country that will be used for the stats.

    For people like Nuri, Scarblac and shauge, if this shows your country correctly, but stats are wrong then there is an SB problem that needs sorting.

  15. #55
    Junior Member
    Join Date
    Jan 2003
    Location
    Netherlands
    Posts
    14
    Originally posted by MikeH
    If I understand correctly how SB country determination works, if you go here and enter your IP addresses, this should indicate the country that will be used for the stats.

    For people like Nuri, Scarblac and shauge, if this shows your country correctly, but stats are wrong then there is an SB problem that needs sorting.
    This works correctly for me. It also works for the other people on my team (PINO) that I've tried (all from the Netherlands - and almost all either from the University of Groningen, or the @Home cable provider, and those IPs all work correctly there). Still, my team did 1.73McEM/s in the last 24 hours, almost double the total production of the country the last day (943.69 KcEM/s). Last time I already noticed that the stats on the country page don't add up to the total project stats at all.

    I have noticed other small problems with the stats, sometimes they just don't add up. Let me look...

    Say, team stats. Total production of PINO is now 5.04TcEM in 67.7 days, or an average of 5040000000/(67.7*24*3600) ~= 862 KcEM/s, close to what the overall rate table says (863.33).

    For team d'family, it says 4.48T in 62.7 days (average about 827), but the overall rate column says 679.04 - not close.

    I think I've noticed other things but I can't think of them at the moment.

  16. #56
    I love 67607
    Join Date
    Dec 2002
    Location
    Istanbul
    Posts
    752
    It's correct for me too. (195.174.xxx.xxx is Turkey)

    I don't really check for country stats with much frequency since I discovered that it was wrong. So, if anybody checks it frequently, can you please confirm if it is updating itself at all? AFAIK, the 37 countries figure did not change since the country stats were first implemented. Is it really likely that even just one person from any one of the remaining ~170 countries did not join the project within that period?


    As of now, total SB production is 593.34T cEMs. The total of operating systems is something like 585T cEMs (99% of total), which is pretty close, and acceptable. On the other hand, the total of 37 countries is 326.12T cEMs (55% of total). So, where did the remaining 45% come from?


    May be we should contact SETI researchers. It might be a kind of extraterrestrial intelligence trying to contact us, and perhaps we've found ETI before everybody else, as a byproduct of our project.


    Ok, seriously, I personally prefer not to have any county stats at all if even the total stats is 45% off. To my view, there is really no need to mention individual countries at all if the total is that misleading. As a last note, I really don't care much about the country stats. But, what troubles me more is about if the obvious error in country stats is harming the overall confidence in the project itself.

    By the way, Remaining tests n < 3000000 and Remaining tests n > 3000000 stats on the overall stats page is cool.

    Regards,

    Nuri

  17. #57
    Hater of webboards
    Join Date
    Feb 2003
    Location
    København, Denmark
    Posts
    205
    I don't really check for country stats with much frequency since I discovered that it was wrong. [b]So, if anybody checks it frequently, can you please confirm if it is updating itself at all?
    It does change.

    As of now, total SB production is 593.34T cEMs. The total of operating systems is something like 585T cEMs (99% of total), which is pretty close, and acceptable. On the other hand, the total of 37 countries is 326.12T cEMs (55% of total). So, where did the remaining 45% come from?
    It might be caused by failing reverse DNS-lookups. There are two things that might cause this:

    1. Unfortunenately not all IP's in use have proper reverse DNS.

    2. As I wrote yesterday, my work appears not to be counted (at least not for the country), probably because the computers I use, have private IP-adresses and the stats are based on IP's reported by the client. - Try performing a reverse DNS-lookup for 10.2.1.129 (the IP of the computer I'm using right now), 192.168.1.42 (my desktop computer at home or 172.28.10.67 (my laptop when I'm at home).

    Ok, seriously, I personally prefer not to have any county stats at all if even the total stats is 45% off.
    I agree.

    .Henrik

  18. #58
    Member
    Join Date
    Jan 2003
    Location
    Norway
    Posts
    61
    I just looked up my home ip-address, it got to the correct country. I have steadily produced more than 300K/s at home, that is more than my total country rate.

    I will check my work ip-address tomorrow.

    It looks as this issue may be possible to fix then. Good.

  19. #59
    Senior Member
    Join Date
    Jan 2003
    Location
    UK
    Posts
    479
    shauge wrote
    2. As I wrote yesterday, my work appears not to be counted (at least not for the country), probably because the computers I use, have private IP-adresses and the stats are based on IP's reported by the client. - Try performing a reverse DNS-lookup for 10.2.1.129 (the IP of the computer I'm using right now), 192.168.1.42 (my desktop computer at home or 172.28.10.67 (my laptop when I'm at home).
    I think these address you quote are just local LAN addresses, they are not the address by which you'll be know to SB. If you follow this link, you'll see your external IP address and the country (even if it can't be resolved).

  20. #60
    Hater of webboards
    Join Date
    Feb 2003
    Location
    København, Denmark
    Posts
    205
    Originally posted by MikeH
    I think these address you quote are just local LAN addresses, they are not the address by which you'll be know to SB.
    Why do think I have been calling them private? I know they are local to the LAN's. That doesn't change the fact that they are probably the only adresses the client knows.

    And since there are perfectly fine reverse DNS for the public addresses I use. Relying on IP's from the client is the most probable explaination for my work, not being included in the numbers for Denmark.

    If you follow this link, you'll see your external IP address and the country (even if it can't be resolved).
    Besides being a graduate students in mathematics, I'm also working as a systems & network administrator, so I know how to find my public IP address.

    .Henrik

  21. #61
    Senior Member
    Join Date
    Jan 2003
    Location
    UK
    Posts
    479
    hc_grove, no offence intended.

    What I was really trying to say is that it's unlikely that SB are recording your private IP address. Infact I know they're not from a previous posting with regard to my IP address. As such, this should not be the cause of the reports of country stats being wrong.

  22. #62
    Hater of webboards
    Join Date
    Feb 2003
    Location
    København, Denmark
    Posts
    205
    Originally posted by MikeH
    hc_grove, no offence intended.
    I didn't think so, so none taken.

    What I was really trying to say is that it's unlikely that SB are recording your private IP address. Infact I know they're not from a previous posting with regard to my IP address. As such, this should not be the cause of the reports of country stats being wrong.
    Then I'll be looking forward to an explanation from someone who knows the system.

    .Henrik

  23. #63
    They aren't using reverse DNS, they are looking at the country of the owner of the IP address. This is not necessarily the same as the person who has been assigned the IP address.

    From my memory, SoB started off doing rDNS, but there isn't a good way to get country data from that (where is .com? or .net? or .edu?). So, SoB switched to using the IANA IP address assignments.

    The owner of an IP can be researched by doing a WHOIS against the following websites:

    ARIN (www.arin.net) does the Americas & some of Africa.
    APNIC (www.apic.net) does the Asia/Pacific areas.
    LACNIC (lacnic.net/en/index.html) does Latin America & some of the Caribean.
    RIPE NCC (www.ripe.net) does Europe, the Middle East, Central Asia & some of Africa.

    Some of these will also point you to sub registrars if they've handed off blocks to others. Some ISPs will register IP assignments with the appropriate registrar but, in my esperience many don't. In that case the IP is shown as being where the ISP HQ is. I usually start with www.arin.net, but that's because I'm in the US.

    For example, my IP at home is 66.23.192.250 (Go ahead, blast away, I'm pretty sure I'm safe, but if you do find an obvious hole, be gentle ) If you do a whois at www.arin.net, you'll find that this IP is owned by Speed Factory (my ISP). It's "mine" in that it's been staticly assigned to my router by my ISP, but my ISP owns it and hasn't taken the time to register it to me with ARIN (not that I really expected them to or care if they do). In my case the ISP just happens to be in the same city, state and country as I am (their HQ is actually only about 2 miles from my house, one of the reasons I selected them). But that doesn't have to be the case. I know that my office's IP will be listed as being in NC and that the particular IP we have is listed as being assigned from them to another ISP who is listed as being in VA. My office is in GA. So, if SoB were to use the same routine they are using now, but extend it down to the state level, the units I completed at work would be credited to either NC or VA, but not GA.

    In doing research on tracking down spammers, I've run across many cases in Asia and Europe that the ISP is in one country but the IP is bing used in a different country. I think this is more prevalent in Europe and Asia where the ISPs are multi national.

    As usual, this is only my understanding of the way things work, so take it with a grain of salt.
    Mike
    -----
    Mike Faunce
    mdfaunce at hotmail dot com

  24. #64
    Member
    Join Date
    Jan 2003
    Location
    Norway
    Posts
    61
    I have now checked my IP addresses both at home and at work using both of the above methods, in all cases the ip address is linked with the correct country.

    Maybe Henrik is right. Maybe it is the ip address that the client think it is having that is used. Then everybody who is behind a router will not get the correct country, also all who has a combined DSL modem/router at home.

    I think we deserve an explaination.

  25. #65
    Junior Member
    Join Date
    Jan 2003
    Location
    Netherlands
    Posts
    14
    Originally posted by shauge
    I have now checked my IP addresses both at home and at work using both of the above methods, in all cases the ip address is linked with the correct country.

    Maybe Henrik is right. Maybe it is the ip address that the client think it is having that is used. Then everybody who is behind a router will not get the correct country, also all who has a combined DSL modem/router at home.

    I think we deserve an explaination.
    I don't think that's the reason, most people in my team are on our uni's network (129.125.*) or have cable modem, and they all have a normal IP address. I suspect there's just some small bug in the stats code somewhere.

  26. #66
    Hater of webboards
    Join Date
    Feb 2003
    Location
    København, Denmark
    Posts
    205
    Originally posted by MDFaunce
    [B]They aren't using reverse DNS, they are looking at the country of the owner of the IP address.
    No. That's also correct for the public addresses I use.


    .Henrik

  27. #67
    Senior Member eatmadustch's Avatar
    Join Date
    Nov 2002
    Location
    Switzerland
    Posts
    154
    a neat little tool (for linux and windows) is the cyberabuse whois (www.cyberabuse.org). I use it mainly for tracking down spammers. The neat thing about it is it only shows relevant information and it searches for the WHOIS server itsself. So it will show which country the spammer (or sob user or whatever) is from, it will show the abuse e-mail (useful if it's a spammer) and it will show other relevant information too. for example, MDFaunce (se his post earlier with his IP) is in the US, and if he spams you, you've got to complain to darryl@speedfactory.net
    EatMaDust


    Stop Microsoft turning into Big Brother!
    http://www.againsttcpa.com

  28. #68
    Hater of webboards
    Join Date
    Feb 2003
    Location
    København, Denmark
    Posts
    205
    Originally posted by Nuri
    I don't really check for country stats with much frequency since I discovered that it was wrong. So, if anybody checks it frequently, can you please confirm if it is updating itself at all? AFAIK, the 37 countries figure did not change since the country stats were first implemented.
    I just saw that the list now includes 38 countries. (But I don't know the list well enough to tell which country is the new one.) That shows that new IPs are in fact resolved.

    As of now, total SB production is 593.34T cEMs.
    The total of operating systems is something like 585T cEMs (99% of total), which is pretty close, and acceptable. On the other hand, the total of 37 countries is 326.12T cEMs (55% of total).
    Now the total production is up to 736.388T cEMs, the total of the 38 countries is around 360T cEMs. That 48% of the total so that stats have gotten even worse!

    That means that only 28% of the work done in the last weeks have been attributed to a country.

    Fix those country stats or remove them. In the present state they are worthless (at best).

    .Henrik

  29. #69
    Fixer of Broken Things FoBoT's Avatar
    Join Date
    Dec 2001
    Location
    Holden MO
    Posts
    2,137

    Re: SB's stats: open discussion

    Originally posted by kugano
    4) Take-over times for user and team rankings, i.e. an indicator of "how long it will be at your current rate to overtake the person ahead of you in the rankings."
    this is something i really like in some other DC projects, it would be especially helpful for SoB due to the units the stats are measured in, i am not too smart and don't really understand what the stats mean, so if you made it plain "FoBoT will pass XYZ in 3 days and 4 hours" that would be super cool
    Use the right tool for the right job!

  30. #70
    IPs are pulled off the first packet in the login transmission by the server.

    Countries are pulled from IANA listings only. If IANA has no country associated with a record, it can't be used.

    -Louie

  31. #71
    Hater of webboards
    Join Date
    Feb 2003
    Location
    København, Denmark
    Posts
    205
    Originally posted by jjjjL
    IPs are pulled off the first packet in the login transmission by the server.

    Countries are pulled from IANA listings only. If IANA has no country associated with a record, it can't be used.

    -Louie
    Would you mind explaining what you mean by IANA listings? IANA doesn't hold much data themselves, but leave it to the RIR's.

    Of course, that would explain why 72% of the work being done at the moment isn't counted.

    .Henrik

  32. #72
    Fobot: the data has not been collecting for long enough for truely accurate long-term predictions, but try this link:

    http://marcc.no-ip.org/SoB/User.php?User_ID=360

    MarcC from the Ars Technica DC teams has been working to port his ECC2 stats over to SoB.

    Hopefully that will atleast alleviate a bit of your stats problems Fobot.

  33. #73
    Sieve it, baby!
    Join Date
    Nov 2002
    Location
    Potsdam, Germany
    Posts
    959
    That looks really awesome!

    Can't wait to see it being integrated.

  34. #74
    Member
    Join Date
    Jan 2003
    Location
    Norway
    Posts
    61
    I really like this stats.

    Impressive!

  35. #75
    Member Cmarc's Avatar
    Join Date
    Dec 2002
    Location
    SF Bay Area
    Posts
    70
    Thanks for the kind comments.
    I still have a couple of small problem at the moment:

    -How is rank calculated for the official stats? I can't seem to get a match for all the users with no work.

    - How do I get the Total and average Work values for team members? a person's work does not follow him from one team to another but to accurently calculate totals I would need more history than I currently have. Is there another place I could find this information (without parsing all team pages every hour) ?

    Thanks,
    Marc

  36. #76
    Hater of webboards
    Join Date
    Feb 2003
    Location
    København, Denmark
    Posts
    205
    [QUOTE]Originally posted by Cmarc
    Thanks for the kind comments.
    I still have a couple of small problem at the moment:

    -How is rank calculated for the official stats? I can't seem to get a match for all the users with no work.


    They seems to be alphabetically sorted.

    It's really nice.

    .Henrik

  37. #77
    Originally posted by Cmarc

    Is there another place I could find this information (without parsing all team pages every hour) ?
    Yes! Read this http://www.seventeenorbust.com/help/textStats.mhtml

    I'm sure Mike would much rather have you leeching just the text dump of the database info than parsing every page.

    -Louie

  38. #78
    Member Cmarc's Avatar
    Join Date
    Dec 2002
    Location
    SF Bay Area
    Posts
    70
    Thanks Louie. I already use this page for my stats (and I must say it is the best stats dump I've seen so far. certainly the only one that comes with detailed explanations).

    leeching all pages is, of course, not an option. my problem is that the values for users are totals and I have no way of determining what portion of this total was produced while the user was a member of his current team. I can reflect team changes that happened since I started collecting data (about 3 weeks ago) but not what happened before.

    Marc
    Last edited by Cmarc; 02-28-2003 at 12:35 PM.

  39. #79
    FWIW: Your (Cmarc's) stats have an odd update pattern: Once an hour, either on h:20 or on h:33 in turn. Is there a reason for that?

  40. #80
    Member Cmarc's Avatar
    Join Date
    Dec 2002
    Location
    SF Bay Area
    Posts
    70
    Yes that is a bit strange. I pull the stats from the page listed above every hour at h:33. From the start I've seen this strange pattern. changing the update time does not help much as far as I can tell. If I move the update time the choices become :20 and :00. Given the large number of updates and the fact that there are only two different period lengths the variance does not matter much in the long run, just a minor curiosity.

Page 2 of 3 FirstFirst 123 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •