Since the tests are getting larger all the time and they seem ot increase in CPU time very quickly is there any thought of (or even possability of) future test being divided among multiple computers?
Printable View
Since the tests are getting larger all the time and they seem ot increase in CPU time very quickly is there any thought of (or even possability of) future test being divided among multiple computers?
this is surely possible. All you have to do is transfer the cache and zXXXXXXX file. I think this should be done, the tests will get too long soon!
I really don't think that it will be done. The cost of sending all of the data required is prohibitive.
And think about it like this:
Computer A finishes a part and sends in all of the tempory results. WHat is computer A going to do now? It does not make sense for it to download a part of some other number to keep working does it? It may be of some use if that machine was exiting the project but even then the extra space/processing required on the server would probably outweigh the benifits.
cjohnsto is right:
Splitting the work only makes sense when
a) there are more computers than work units (not the case)
b) computing power gets lost because of aborted test <-- happens from time to time, but this should be the minority of tests - I think Louie spoke of less than 1%
I don't know the current size of the cache files, but it should be ~400KB, right? Imagine the bandwidth it would take for almost 1,000 computers to send 400KB maybe twice a day. That would be nearly 800 MB/day or 10KB/sec. However, it should take less (or not much more) than a KB to signal a block or send the result.
Cache file transfer occasionally makes sense in a LAN environment where there are fewer computers working and bandwidth is not that an issue.
No only that. It must be possible to process the different parts in parallel, which is not the case.Quote:
Originally posted by Mystwalker
cjohnsto is right:
Splitting the work only makes sense when
a) there are more computers than work units (not the case)
The effects we would get from splitting work units are
1) Lots of wasted bandwidth
2) Every test takes longer (because no computer can work on it, while the cache is being transferred).
I don't think we want that.
.Henrik
That is the hard part. A prp test concists of N-1 (or was it N-2?) iterations. The result of one iteration is the starting value of the next.Quote:
No only that. It must be possible to process the different parts in parallel, which is not the case.
It is possible though to perform parts of a single iteration on multiple processors. This way, a single test finishes faster, but it's more efficient (memory bandwith etc) to run a single test on each cpu.
Plus, that means the slower a machine is, the more unlikely it contributes any work for the project at all. ;)
very soon we'll be getting to tests that are so large even the fastest computes will take weeks to finish them. How are we going to deal with this situation. As we approach n values above 4-5 6? million this issue will become more apparent. However, before we finish finding all of the primes we wil be reaching values for n that are several factors of 10 larger than this. These tests have the potential to take years. Are we going to have computers that are going to be running the sametest for years at a time. I doubt many people would like that idea. If anyone has any ideas about an efficient way to reform te tests that would allow for them ot be sub divided them we should start planning for such a situation. Its only a matter of time before its a necessity.
I agree with most of the comments regarding the problems with splitting. On top of that, I want to add two more points.
GIMPS is already testing for primes that is at around n=18-19 million, and as far as I can see they are not facing any problems about the time it takes to finish a test.
I don't know how long it will take until we will start numbers that big, but when we reach those numbers, we'll have much more powerful computers. In other words, the time to finish tests will hurt us less than it is hurting GIMPS right now. If you are talking about numbers as big as n=100 million, we have enough time to think about that until we reach there.
Apart from technical difficulties, another reason why I think we should not be splitting tests is the problem of crediting users when a new prime is found. Will you credit everbody that has done some part of that test equally? (no matter how small or large his/her participation in that test is). If not, how will you credit different users that computed different proportions of that test?
I'm pretty sure that, if for any reason we decide to split tests, a new type of users (or cheaters you might call) will emerge. They will run their client for the smallest portion of a test that is possible, and jump to another test, trying to maximize their chances of "touching" the number that turns out to be a prime.
A couple of thousand exponents around 33 million are already tested by GIMPS. I know, some tests have taken more then a year to complete. A few numbers much larger than that have already been tested, and i've read somewhere that a test (and a double check at the same time so they can compare at intermediate points) well above 70 million (possibly 79 million) is underway.Quote:
GIMPS is already testing for primes that is at around n=18-19 million, and as far as I can see they are not facing any problems about the time it takes to finish a test.
OTOH, A Lucas Lehmer test can be performed faster then a proth test (i think twice as fast, but not sure about that)
Numbers are getting larger yes, but whats the point of testing a number half way, returning it for someone else to complete, and start testing a simmilar sized number half way again? You might as well finish the original number youreself, and save the time it takes to send an intermidiate file back (and someone else retrieving it). No testing willl be done during that time.
Like i said before, with a proth test it's not possible to start a test on one computer, and at the same time start another computer half way the test, and when both are done, perfom some magic to see if the number was prime or composite.
Well, there is the problem with people wanting to shut down computers (i.e. withdraw them from SOB entirely). What are you going to do the next time you replace your box? There's no "stop after upload" switch here anywhere (that I can see).
I think there's quite clearly the need for some mechanism that allows me to tell the client "OK, send this all back to home base and then quit". The vast, vast, vast majority of tests will run completely one one machine, but the ability to forfeit a test without having to throw away all the crunching that has already been done on it will certainly become more and more important over time.
Myself, I'm happy to dedicate a box or two to this project for a few weeks or months -- but if I knew from the outset that it'll have to be at least two years (or whatever) or all my work is going to be for naught because there's no mechanism to interrupt in the middle without wasting all the CPU power expended to date, I wouldn't even start.
Maybe one could implement "save points" - i.e. an option to send the cache file to the server in case the user withdraws from the project. Possibly a backup at regular intervals like once a week, too.
As said before: splitting work units doesn't make sense. But saving the work from time to time could be wise when tests get so large that an abandoned test discards a lot of computing power, let's say the above mentioned week.
Shouldn't be a problem of time to implement this. It's rather a question of bandwidth...