RNA World

[Michael H.W. Weber]

I really do not know how to give you the info you are asking for.

If you cancelled them manually, then you will find their names in the message window of the BOINC manager which you could paste in here.

All I know Is I Do not want to work on Long WU's
I see them I abort them And as long as I see them I wi9ll not port mor power here

I understand. Since my last posting some weeks ago, the WUs are mainly around 1.5 hrs in size with only few exceptions and as I believe these can easily be handled even without checkpointing. More importantly, they do not consume these tremendous amounts of RAM as some earlier batches did.

And Michael you avoided the question of
Have you fixed this Problem yet ?

So far, checkpointing is there only for x86 Linux as before. As soon as this changes, we will of course announce it.

Michael.

[LAURENU2]

Well Michael I just Lost another 250 Hrs of computer time with 8 more Looonng WU,s I aborted
My Error page on your sight has just doubled in size

http://www.rnaworld.de/rnaworld/resu...ames=0&state=5

I am sure you can fine the names of the WU's on the above Page

Having such long WU' floating around WITHOUT any save points is Not a good thing Michael
I am sorry to say that If I keep getting the few exceptions as you put it I will flick the switch on RNA Till you get it fixed

[Michael H.W. Weber]

(1) The current run times, i.e. shortest, longest and average you can always look up at any time from our server status page [at present: 0.92 hrs average (0.03 min - 12.61 max)] and then decide whether or not to participate. However, I have seen many projects without any checkpointing and long running WUs while we at least offer checkpointing for x86 Linux. So, given the stability of our project I am currently quite happy with it but of course it always depends on how you like to use your machines.

(2) As detailed before, we will implement checkpointing on the basis of a functional VM as soon as possible.

(3) I will consider implementing a run-time selector such that, if possible at all at the BOINC server software's end (that I will have to check), users can select how long the biggest WU is allowed to be. I think this might be a useful compromise at this stage.

Michael.

[yoyo]

Well Michael I just Lost another 250 Hrs of computer time with 8 more Looonng WU,s I aborted
My Error page on your sight has just doubled in size

http://www.rnaworld.de/rnaworld/resu...ames=0&state=5

Hi Laurenu2,
nobody has access to this url, only you as logged in user. Or I must hack me into your account, which I do not want. So please say at least one ID of an LONG wu.

yoyo

[LAURENU2]

OK here is a screen shot of one page



Most were aborted 50% or less a few befor they started

I'm sorry I have a hatred of Long WU's
I did one for UD that ran for about 2500 HRS 24/7 and it FAILED
I was So Pi$$ed off I promised never to do another

This is also being reported on the RNA forum
I thought it was also said you FIX or stooped the Long WU's in the Past
Why are they Back in the system again

[Michael H.W. Weber]

I'm sorry I have a hatred of Long WU's
I did one for UD that ran for about 2500 HRS 24/7 and it FAILED
I was So Pi$$ed off I promised never to do another

This is also being reported on the RNA forum

Let me clarify a few things here: A long WU in RNA World does definetely NOT mean that you are at risk of not completing it without errors except of course you abort it manually (you shouldn't have done that because if you hadn't you would have seen that they finish and give credits). I hope the time of error reports as we had in the past is over now. At least, the malfunctional WUs are COMPLETELY out of the queue and, again, the RAM requirements are more than moderate. You need to note, however, that with Windows you will loose your current calculation results if you restart the system due to lack of checkpointing with this OS.

I thought it was also said you FIX or stooped the Long WU's in the Past
Why are they Back in the system again

As said above, they are not back in the system and the problematic ones have indeed been deleted from the work queue. Still, some WUs are long runners. But that really is a minority. You need to understand that RNA World unlike other projects where e.g. a fixed number of MD simulations are computed over and over again is a project with very heterogenous WUs. And this will remain. But, anyway, I think I gave enough of explanations concerning all this. It ultimately is up to you what you like to support and what not.

Michael.

[LAURENU2]

This statement may not be true

Let me clarify a few things here: A long WU in RNA World does definitely NOT mean that you are at risk of not completing it without errors except of course you abort it manually
Michael.

I saw WU's in my systems that listed 244hrs
What if it ran for 9 days Power fails start over
run for 7 more days again and bam another power outage
are you WU's OK to rum for 20 or 30 days ?

Or what about the little guy who does not work 24/7 and shuts down there PC overnight
it would be a endless loop for them with out save points until your projects aborts it
and then gives them a new Long Wu to wast there time on

I want to support your project But I think a WU that runs for more then 4 hrs with out even 1 save point is risky
And one that rum for 24 hrs I do Not want to do even if they have save points
I am sorry Michael I made a promise to myself 10 years ago Not to do Long WU's
Thats why I just aborted 600 to 700 hrs of my computer time
I hope this makes you understand how strongly I feel about this matter of Long WU's

[gopher_yarrowzoo]

This statement may not be true
Or what about the little guy who does not work 24/7 and shuts down there PC overnight
it would be a endless loop for them with out save points until your projects aborts it
and then gives them a new Long Wu to wast there time on

I want to support your project But I think a WU that runs for more then 4 hrs with out even 1 save point is risky
And one that rum for 24 hrs I do Not want to do even if they have save points

QFA Lauren - I currently don't do RNA World I was thinking about it but I am a little guy who don't run BOINC 24/7 on my main PC and well my backup PC is getting used more as my Main PC needs a major overhaul now.. So I wouldn't want to have to abort days of work if there are no save points - I mean how can that work I run multiple projects on both machines what happens when they switch projects - same thing it starts from 0 again?
I need to run windows as I have software I use for work on my home pc so I can do remote reseting of the systems we got and to see where the problem lies.

[yoyo]

I checked this 189,000s workunit http://www.rnaworld.de/rnaworld/work...p?wuid=1015952 from LAURENU2 list. You see on the workunit page that this one is estimated with 318,000s runtime on the reference system. There is already one result finished for this wu, which needed 206,000s.

I checked also this http://www.rnaworld.de/rnaworld/work...p?wuid=1015891, which you aborted after 111,000s. This is estimated with 402,000s (4d 15h) on the reference system and finished by one host after 117,000s.

Yes, they are long, but not 10 days.

Maybe your Boinc DCF (duration correction factor) is to high and therefore the remaining time is so high. This time anyway is mysterious calculated by Boinc client.

yoyo

[LAURENU2]

Yes Yoyo I did abort 1 or 2 that were 90% complete and a run time of 50 & 95 hrs
It has to do with the promise I made to myself read above

The 10 days came from the est time of a WU listed here that I aborted
!0 days or 1 day the same looping thing can happen without Save points

What is holding you back from having SAVE POINTS
It Seems like most all other BOINC have them
You just tell it to save every 10 Min

I know nothing about a (duration correction factor) Boinc is a stock install with no tweaking

[Michael H.W. Weber]

I checked this 189,000s workunit http://www.rnaworld.de/rnaworld/work...p?wuid=1015952 from LAURENU2 list. You see on the workunit page that this one is estimated with 318,000s runtime on the reference system. There is already one result finished for this wu, which needed 206,000s.

I checked also this http://www.rnaworld.de/rnaworld/work...p?wuid=1015891, which you aborted after 111,000s. This is estimated with 402,000s (4d 15h) on the reference system and finished by one host after 117,000s.

Yes, they are long, but not 10 days.

Well, I see. His machines really picked the biggest ones that remained in the system (and will remain in the future as well for analyses of other organisms).

What is holding you back from having SAVE POINTS
It Seems like most all other BOINC have them
You just tell it to save every 10 Min

No, you do not 'just tell it to save'. If you want checkpoints (i.e. 'save points') then you need to write (or re-write) the entire code of your science application de novo. I have explained this many times (even in our FAQ). Most scientific applications that run on high performance clusters are not desiged to have checkpointing. With RNA World, we run such software and it is technically impossible to change the code to write the checkpoints: it would have to be re-written completely with a different design. Moreover, it is a waste of time to do this for all the implemented and the many, many upcoming software modules in the future. I know other DC projects have checkpoints, but these usually have one single science core client and the entire system was from the very beginning designed to run as a DC system - that is a completely different starting situation compared to what we are dealing with at RNA World. Hence, we thought about methods to develop a universal checkpointing, i.e. 'writing save points' at the system level to apply it to exactly these applications which do not allow for checkpoing generally - and these applications are undoubtedly the majority. This poses a different problem, namely that you require admin rights to save the entire RAM content to disk (it would be as if you send your laptop to sleep mode). Would you grant admin rights to a DC project? No. See, and that's the problem. We now have decided to use a virtual machine approach within BOINC. But that project is not yet complete.

I know nothing about a (duration correction factor) Boinc is a stock install with no tweaking

Yes, and as this stock install it has problems known for long. One is this DCF issue which means that the run time estimate BOINC displays in the form of your progress bar is adjusted by (your stock install) BOINC manager depending on how much your machines spends on tasks different from the BOINC tasks. If you play a game while RNA World computes, this gaming is at cost of RNA World WU progress. BOINC detects that and re-calculates the remaining run time. As such a neat idea. In practice, however, when you stop playing, BOINC takes a long time to re-adjust this correction (factor). As a result, it displays run-time estimates that are far too high. And this is exactly what happened with you RNA World WU. One could say that BOINC actually 'betrayed' you in telling you the wrong run time estimate.

Well, I think it should all be a bit clearer now. For me it is clear that running RNA World on machines that do not run 24/7 is a problem as long as we do not have checkpointing. And please note that I am by no means 'angry' or so because you have put forward some criticism here. The only thing I can do about this is trying to explain why we do the things we actually do and why some things are not yet implemented.

Michael.

[LAURENU2]

Well thank you Michael
That was a vary clear and explanation of how RNA works within BOINC
And I know you are not upset Nor am I upset It is just that I can't Do Long WU's
And was worried about the 8/5 user's stuck in a Loop

As for the DCF only 3 PC's out of the 60 I run here are used by people the 58 are only doing DC 24/7
And the 3 do not run RNA because they need a reboot from time to time

I will continue to Run RNA but I will keep a eye out for long WU's and send them back to you Like this one

2473870 1059802 12 May 2010 21:06:46 UTC 17 May 2010 13:36:43 UTC Aborted by user 327,886.09 296,718.66 1,285.62

I know thats a lot of points to flush but my word is my honor

[Michael H.W. Weber]

http://www.rnaworld.de/rnaworld/forum_thread.php?id=51

Michael.

[Michael H.W. Weber]

Hi guys, so the conference turns out to be a remarkable success. We have met a lot of people that are very interested in our project. Even better, we have acquired new collaboration partners that can help us in experimentally validating our computetaional results on a broader scale. Also, posters presented by independently working groups indicate that we are absolutely on the right track with our analyses. So, I am quite confident that we will soon make it for the first publication in a scientific journal. Moreover, we have scheduled the next conference presentation of RNA World which will take place in September in Dresden/Germany. Details on that at a later time.

Michael.

[Michael H.W. Weber]

We have released an OSX client today.

Michael.

[Norman]

We are currently working on reorganizing the CMSEARCH application into two variants of which one will run a combined package of many very small WUs while the other runs the rest (the default CMSEARCH) WUs. With this we intend to achieve three goals. First, the combination of many tiny WUs into one package tremendously reduces the client-server communication load: a big relieve for our server which will make it significantly more stable even during team challenges and races. Second, the packaged WUs will write a checkpoint after each completed sub-WU. Third, because the users will be allowed to opt in or out for each of these two CMSEARCH variants there will be much more flexibility at the user's end to control maximum CMSEARCH runtimes.
Michael.

@tm a few of the old cmsearch-wu are in the pipeline and after that the project can do final changes. (choice: long runners or the shorties )
new apps with more performance are out for MAC OS 10.4 +, Linux and Win.
also we need urgend testers with MAC OS 10.4 and 10.5 with intel 64bit.
( best you write directly here: http://www.rechenkraft.net/phpBB/viewforum.php?f=74 )
happy crunching

ps: you can see how many tiny wu are in a archive:
cms_GA-p[DZ-Lin64s]_7_Desulfitobacterium-hafniense-Y51_AP008230........
here are 7 in one archive.
can be over 100 or 1000.

[Norman]

yoyo has doubled some old wus, to get these archives finished. But this means, that these tasks got the same HR class as the existing ones.
So some HR-classes are now blocking the send queue and must be send first, before you get new work.
a few more 32-bit linux are helpful.

[Norman]

any problems, bugs or suggestions?

[LAURENU2]

any problems, bugs or suggestions?

Yes saving checkpoints

[Norman]

hehe
Oooh yes! that's a big request from me.
A man developing it and I hope as soon as possible done.

[Norman]

Hello!
We will kill the remaining 10 introns WU likely.
The users will get full credits for the terminated work.
The runtimes can then be placed finaly in short (s) and long-XXL.
The few introns are then again under XXL.
When it happens there are also news to be read on the project's site.

[Norman]

the introns should now have been broken off from the project and the user have get their credits.
Now is the separation of short (S) and long (XXL) is underway as it was being considered.

edit:
The remaining introns were canceled and credits should have been granted or are there still.
if this is not the case is please to announce.
Next, the cmsearch XXL adapted so that it functions as the S.
a few test-WU will follow and the introns are again redistributed among XXL.
WU-generator is also adapted for the big yet.
Right now, the XXL a test app (so who wants to allow)

[Norman]

our database is crashed again and will repaired soon as possible..
sorry.

[Norman]

database crash is over and now the new virus-WU are in the serverpipeline

[Norman]

it seems to be a little bit unclear for users about xxl and s and what they do/we mean.
so we have modified the apps-name to:

cmsearch XXL (large) 1.0.2
cmsearch S (small) 1.0.2

and we hope it´s now easy to understand what we mean with xxl and s
is it better ?! any suggestion ?

[Paratima]

Yes. Now how can I tell BOINC that I do NOT want to run the XXL types?

*EDIT: Never mind - set preferences on your site. Fixed.