stats question (plus question about sieving)

[Keroberts1]

when will the number of test remaining below 6 (or 7) million and the number of tests over 6 (or 7) milion return? I like monitoring that stat because it allows me to see my sieving efforts impacting the project. Also could anyone tell me if I've had alot fo excluded factors submitted lately. One of my machines has been acting up and i rana stress test today and found that it is failing horribly. It fails the test in less than a minute most times. This machine has been used for sieving only and i don't know of the top of my head exactly what ranges it had but i can upload the fact.txt file to any one who would be able to tell me if the CPU has ben missing factors.

[Death]

you can post a fact.txt here and somebody can resieve some ranges from one factor to another to check missing factors.
i bet you found a couple of friends here =))

[Keroberts1]

well someof the ranges that this computer had were rather large adn if factors were missed thenit probably wouldn't be worth resieving the range for quite a while. Also i don't know how long the problem was going on so i was just hopping someon would be able to hopefully look at the result.txt file or look under my ip addresses and see if any particular one was generating alot of excluded factors or perhaps finding many fewer than expected.

[Keroberts1]

Please some attention to the first question though.

[Nuri]

see bottom of the page for your stats http://www.aooq73.dsl.pipex.com/ui/3423.htm


FactorsE means excluded factors.

FactorsD means duplicates.

FactorsU means unique factors. i.e. first time factors.




see bottom of the page to compare your average factors per G with others. http://www.aooq73.dsl.pipex.com/ui/9999.htm


Example:


211000 - 211500 500 40 1 0 86.616 jimmy
211500 - 211600 100 4 0 0 8.463 frodo42
211600 - 211700 100 7 0 0 14.817 rosebud
211700 - 211800 100 8 0 0 16.939 Kroberts5 (Keroberts1)
211800 - 213000 1200 71 0 0 94101.869 hc_grove
213000 - 213100 100 7 0 0 14.913 alexr
213100 - 214000 900 66 1 0 77792.140 Mat67


211000 - 211500 has 0.080 factors/ G (=> 40/500)
211500 - 211600 has 0.040 factors / G
211600 - 211700 has 0.070
211700 - 211800 has 0.080
211800 - 213000 has 0.059
213000 - 213100 has 0.070
213100 - 214000 has 0.073

[Keroberts1]

i guess that seems fine then because the range you used as an example was one of the most recent completed by the faulty computer. The sieve client is still crashing rather repetitivly though. I might simply have to do with the fact the it has been rather hot around here lately adn the computer in question is in the hottest part of the house. I'll place a couple fans around it and redo some of the stress testing in a day or so.

[Keroberts1]

still hoping kugano will bring back the remaining tests lines on the stats page. Hopefully remaining below 7 mil and remaining over. But if i don't hear from him after this post I'll assume it has something to do with the new queing system and I'll shut up abnd stop pestering. I just can't get over my paranoia that he may have completely missed my earlier posts.

[Mystwalker]

The secret stats at least show you the number of remaining tests <8M...

[kugano]

still hoping kugano will bring back the remaining tests lines on the stats page. Hopefully remaining below 7 mil and remaining over. But if i don't hear from him after this post I'll assume it has something to do with the new queing system and I'll shut up abnd stop pestering. I just can't get over my paranoia that he may have completely missed my earlier posts.
I hadn't planned to bring back the numbers that were on the "secret" page. It no longer makes sense to measure "secret tests," since there's no such thing. Instead there are just queues, as explained earlier, and a mapping from queues to users.

What is it exactly that you want to know from these stats? If I know what you want to measure, I'll be more than happy to find a good way to do it and post it somewhere.

P.S.: It also doesn't make sense to say how many tests there are "above" a certain number. There are an infinite number of tests remaining above 7 * 10^6, and there will be until the far future when we've eliminated the other 11 multipliers...

[royanee]

The lines that used to be on the Project Stats page about "Remaining test below 6 million" is what he wants back. I thought it was a fun number to watch also. It is getting closer and closer to zero. :)

[Keroberts1]

i believe royanee got my number pretty good and the remaining tests above 6 million up to 20 million was the previous stat. It allowed for a prespective on the production of the sieve too. Allowed me to see my factor dumps effect the big picture. Kinda makes me feel like my little bit of CPU time has a bigger effect. the secret page was fun too but so are all numbers that count down to zero. Of course you must know this because this project was started with only a countdown in mind. That one is at eleven but I hope to see it reach zero too. Perhaps it would be nice to have a number of untested canidates remaining listed by millions. like .. left between 4 and 5 million, ... between 5 and 6 and so on.

[kugano]

So it sounds to me like what you really want are ways to answer the questions:

1. "How many k/n pairs with n < X are still candidates?"

2. "How much work has been saved by sieving to p < X?"

I can think of lots of different ways to measure and display these. For #1, we could have a graph with X on the x axis and the number of remaining candidates on the y axis. This would be an always-increasing curve which would appear to slide to the right over time, as the search bounds increased.

The graph wouldn't exactly "count down to zero," but it would give a lot more visual information. To display this with numbers, we'd have to pick "landmark" values of X &ndash; say, in intervals of 10^6 &ndash; and display the number of remaining candidates below each landmark value.

We could even do both!

#2 is a trickier one (at least insofar as computing it goes, based on the current database schema). This one, too, could be presented in graph or numeric format. A graph might show "work saved by sieving to p < X?" with X on the x-axis and work on the y-axis.

Both of these could be done now, but it'd take a lot of work. Honestly I'd rather wait until the client is in alpha, at which point I hope to start working on yet another major website/stats overhaul, akin to the last big improvement in December '02.

But I'll do some fiddling and see if I can come up with a temporary 'quick fix.' =)

[Mystwalker]

Originally posted by Keroberts1
i believe royanee got my number pretty good and the remaining tests above 6 million up to 20 million was the previous stat. It allowed for a prespective on the production of the sieve too. Allowed me to see my factor dumps effect the big picture. Kinda makes me feel like my little bit of CPU time has a bigger effect. the secret page was fun too but so are all numbers that count down to zero. Of course you must know this because this project was started with only a countdown in mind. That one is at eleven but I hope to see it reach zero too. Perhaps it would be nice to have a number of untested canidates remaining listed by millions. like .. left between 4 and 5 million, ... between 5 and 6 and so on.

Maybe this (http://www.aooq73.dsl.pipex.com/scores_p.htm) page of MikeH's site is what you want. The first table shows just how many k/n pairs are left - of course, no PRP test related countdown is shown here. But this could be even better for your purpose...

[Keroberts1]

I am very familiar with mike's site but what i was thinking about was actually dave's first option (#1). I think it would be a wonderful idea. I want to see the sieving and the main project integrated more and thats wh it'll be nice for sieving results to be visible on the main stats page. I think dave understands what i want.

[vjs]

The intergration of seive and p1 with prp stats would be great, it would also help level out the computer power for the project.

I really liked the idea of the old secret page with how many n per k in a particular range.

It could be done graphically and that would a nice visual but I always prefer numbers to create my own visuals, also they can more easly be mutated into "aftermarket" stats pages and software etc.

Check the benchmark thread in the sieve section

[Keroberts1]

is the old tally going to be added back to the page? Nothng different than what was on the stats page before is needed. I was just hopping to get a stat that says x many tests left below 7 million and x many tests left above 7 million below 20,000,000.

[Keroberts1]

i would still like to see the old stats figure tests less than 7 million and the figure tests above 7 million replaced on the stats page. I know the tests over 7 million only goes up to 20,000,000 but that is exactly what i ould like to see. This way when ever factors are submitted above the active window they would immediatly effect the number on the stats display. It was on the page before so i assume it would be easy to repalce. (I believe the figure was tests with out residuals not just thenumber left in the ques) Can we see these numbers again. Just for fun? Pretty pretty please. It was my favorite stat and I'm sure others also enjoyed watching it.

[Nuri]

You can easily follow this from Mike's project stats page on a per million per k basis.

[Mystwalker]

Mike's page neither shows how many tests have been eliminated by PRP tests nor how many tests have to be done from now on till the end of the M.

So, I understand Keroberts1's demand. And I think it should be a matter of minutes to reimplement it into the project status page...

[maddog1]

One more vote for Keroberts1's demand.
This was a fun stat indeed-bring it back plz!

[kugano]

Well, in fact it's not as easy as you think. The number that was on the page before was a hack, built from a database table that doesn't exist anymore. I doubt it was completely accurate anyway.

While still possible, it's not a 2-minute job. And remember that every minute I spend reading the forums and tweaking already-questionable stats is a minute I'm not spending working on the new client, server and website!