RNA World

[LAURENU2]

So I guess that means hold at a 40% commit mark

[Michael H.W. Weber]

New CMCs for Windows x64 have been fed into the server and will be delivered today. We expect these to run without complications unlike some WUs we delivered to Linux x64 a few weeks ago.
Note that the CMS packets coming up tomorrow will require broadband internet connection. Transfer volumes will be one to two orders of magnitude higher for some WUs compared what you are used to. Computation times will range between a few minutes to up to 100 hrs on a 955 BE (3 GHz). The human genome will also be analyzed.

Michael.

[vaughan]

Computation times ... up to 100 hrs ...

Michael.

Count me out then. I detest long running molecules.

[Michael H.W. Weber]

Maybe a few more details. This time we have a total of 1024 genome files that will be analyzed with two RNAs. Of these 1024 files,

14 are sized > 200 MB (only 3 of these 14 > 300 MB)
126 are sized > 100 MB
319 are sized < 1 MB

All given file sizes are in uncompressed format, i.e. the files actually delievered will be smaller by at least 30%. Run time distribution will approximately scale with to genome sizes but I do not have a detailed run time distribution table (an idea that came to my mind today, but which would cost considerably efforts to process, so I postponed this for a while). So, we picked out very large, intermediate and small genomes and tested these with a set of around 10 RNAs, amongst these of course also the two we use for today's test run. Maximum run time is below 100 hrs.

Michael.

[edit]: Some WUs seem to require well above 1 GB of RAM. Yet we have no means to determine the memory requirements prior to simply running the WU and checking things out in practice. So please report any issues, observed RAM requirements, etc. to me if possible such that we can act quickly if required (e.g. tweak the memory settings as we successfully did with the CMC WUs).

[Angus]

I had to abort all the WUs and go NNW on this project.

The new CMS WUs use stupidly large amounts of RAM. My system used all the physical and page file space, and basically just stopped functioning. It's a pretty vanilla box, Win XP, 2GB RAM, and a dual core CPU. Probably representative of a high percentage of all the machines running BOINC.

I guess this will be a high end, 64 bit machines only project.

[zombie67]

It would be nice is RNA had a page like this, showing requirements by app:

http://boinc.umiacs.umd.edu/apps.php

[Michael H.W. Weber]

It would be nice is RNA had a page like this, showing requirements by app:

http://boinc.umiacs.umd.edu/apps.php

Thanks for the link, it indeed looks quite informative and worth to adapt to our needs as well.

One more thing: We have a number of CMS WUs that unexpectedly demand really immense amounts of RAM. Could you please keep an eye on whether crashes of such WUs occur exclusively on 32-bit machines? We already have a suspicion what might go on here.
Also, if you could provide RAM requirements observed for individual WUs on your systems, we may construct an improved WU assignment system such that the big ones are just not getting delivered anymore to machines that cannot handle them properly. As said before, we cannot yet forecast RAM consumption. So, annoyingly we have to go for tests and then adapt the project settings accordingly.
If all these measures do not help, we must consider splitting the WUs into smaller chunks but that will result in "downstream result puzzle" efforts which I'd prefer to avoid if possible.

Michael.

[LAURENU2]

Out of all my nodes I only have 1 nodes that can get and do work on this project.
With that I will have to pull off this project till you can provide smaller bits of food for my Nodes to crunch

[outlnder]

Unfortunate for Lauren, but great for the rest of us. We will finally get some work without a server crash.

[LAURENU2]

Unfortunate for Lauren, but great for the rest of us. We will finally get some work without a server crash.

Well see Good things come to those who wait
But as Arnold said I Will Be Back

[Michael H.W. Weber]

I think we found a solution for the RAM issue. But this might take some time to implement.

Michael.

[LAURENU2]

I think we found a solution for the RAM issue. But this might take some time to implement.

Michael.

So Michael have You /RNA made any progress on the solution for the RAM issue And the No Checkpoint yet ?

[zombie67]

Thanks for the link, it indeed looks quite informative and worth to adapt to our needs as well.

Any progress on this? I know it seems low priority. But the memory requirements confuse potential crunchers. Something like this would really help.

[LAURENU2]

so michael have you /rna made any progress on the solution for the ram issue and the no checkpoint yet ?

BUMP

[Michael H.W. Weber]

Since I am still in India, just quickly two notes concerning our project from the server's NEWS page:

http://www.rnaworld.de/rnaworld/forum_thread.php?id=46
http://www.rnaworld.de/rnaworld/forum_thread.php?id=47

So, the WUs in process right now should be safe for (hopefully) any machine.

Michael.

[LAURENU2]

Since I am still in India, just quickly two notes concerning our project from the server's NEWS page:

http://www.rnaworld.de/rnaworld/forum_thread.php?id=46
http://www.rnaworld.de/rnaworld/forum_thread.php?id=47

So, the WUs in process right now should be safe for (hopefully) any machine.

Michael.

Well since your in India I will speak loud NO IT IS NOT SAFE
RNA still have a 1000 WU's to burn off

http://www.rechenkraft.net/phpBB/vie...117573#p117573

[Michael H.W. Weber]

Well since your in India I will speak loud NO IT IS NOT SAFE
RNA still have a 1000 WU's to burn off

http://www.rechenkraft.net/phpBB/vie...117573#p117573

As I wrote in the NEWS section (see links above), there may indeed be a few WUs around of the remaining batch. Their number, however, is definetely not 1000 but in the worst case scenario a few hundreds. Moreover, these will exclusively be sent out to machines with large RAM availability. We have improved the WU delivery settings and also added a failure counter that prevents a WU from being sent out again if it has not passed validation for a certain number of times. Finally, if you take a look at the server status page you will see that even in the worst case scenario, the fraction of WUs from the old batch will be significantly below 10% of what the server currently is sending out and should be zeroed out shortly.

Michael.

[rilian]

Project is quite safe.. Any failing WUs on 2GB-doublecore machine since 10 march

[LAURENU2]

As I wrote in the NEWS section (see links above), there may indeed be a few WUs around of the remaining batch. Their number, however, is definetely not 1000
Michael.

Yes just after you posted the 3817.7 MB message stopped and I got WU"S
AS soon as I complete my other project WU's I will start to Suck you dry
Do you think you(RNA) can keep up?

[Michael H.W. Weber]

Yes just after you posted the 3817.7 MB message stopped and I got WU"S

Do you think you(RNA) can keep up?

Well, at present a few hundred thousands of WUs are waiting on the server to be processed for delivey and many more can be generated.

Michael.

[outlnder]

Ha, Lauren, it might take you a couple of days.

So much for mister fast pants.

[LAURENU2]

Ha, Lauren, it might take you a couple of days.

So much for mister fast pants.

I'm like a big train
Slow to start and hard to stop

Turning On or Off a project is fast once I sign up all the Nodes
It is the New projects that take time for me

I could turn on RNA on all nodes in about 60 Min or so

[zombie67]

If you could get BAM to work, it would take 30 seconds...

[LAURENU2]

If you could get BAM to work, it would take 30 seconds...

But then What Would I do for the next 59.5 Min
OH Well It Was A Thought

[outlnder]

I like to administer my pharm manually also. It's so much fun to plug and unplug the monitor, mouse and keyboard.

[LAURENU2]

I use KVM's for the home 1 keyboard and 1 mouse and VNC to the Garage

[zombie67]

I use VNC for everything, even the boxes in the same room.

[LAURENU2]

I use VNC for everything, even the boxes in the same room.

Yes, But I have 2 DSL Networks And I don't understand how to bridge them.
So I need a KVM to switch from 1 PC to the other to control both of my networks
I have a 8 port KVM with 6 PC's just in my DC cave alone (big closet)

[LAURENU2]

Ha, Lauren, it might take you a couple of days.

So much for mister fast pants.

Well it looks like only a couple days will get me to 1st place now that I am up to speed that is if Michael's team can keep my heard happy
Was that fast enough for you outlnder

[outlnder]

Wait a couple of hours. You may break it yet!!

[LAURENU2]

Wait a couple of hours. You may break it yet!!

Well thats why I am only at partial commit So not to bog down there network
I wonder about RNA database with all the small WU's taking up all the DP space

[LAURENU2]

_Free-DC_
Takes 2 ND Place.

[outlnder]

WOW, salty rain, or is that the gods crying??

[Michael H.W. Weber]

As described in the projects NEWS section in more detail, we had to transiently disable the WU generator.

Michael.

[Michael H.W. Weber]

The WU generator is working again, the small WUs will now be processed on the server, except for very, very few which we will deliver to some remote machines. Please also note that the server response delay has been drastically reduced such that connection issues should be resolved by now as well. We have plenty of WUs, will not run out of work and hope for your further support which is very much appreciated.

Michael.

P.S.: Our new concept to resolve the massive RAM requirements of our "CMSEARCH Monster WUs" has resulted in a program which is currently in testing phase. So, even here I expect good progress in the upcoming weeks...

[Michael H.W. Weber]

Not a single error-report in two weeks it now seems that our project is really running reliably and smoothly. Given the vast amounts of WUs in the server queue, we are actually in need of much more support. So, if you could spare some cycles it would be highly appreciated...

Michael.

[LAURENU2]

Not a single error-report in two weeks
OK I will see if I can Brake your new toy
I Just had my Bandwidth increased on both DSL lines here

[LAURENU2]

Not a single error-report in two weeks
OK I will see if I can Brake your new toy
I Just had my Bandwidth increased on both DSL lines here

Michael H.W. Weber I thought you FIXED the problem with LOOOONNNG WU's
I just aborted 6 WU's with 300 HRS of my time and the WU's still had 500 hrs to go
A WU that runs for 100 to 200 HRS is absurd Michael
Have you even put check point in Yet to save all that time ?

[Michael H.W. Weber]

Michael H.W. Weber I thought you FIXED the problem with LOOOONNNG WU's
I just aborted 6 WU's with 300 HRS of my time and the WU's still had 500 hrs to go
A WU that runs for 100 to 200 HRS is absurd Michael
Have you even put check point in Yet to save all that time ?

Well, we do not have such WUs in the pipeline (except for Yoyo would be doing some testing). Hence, I assume that this could just be one of these BOINC-related run time mispredictions. Could you please let me know a few more details on these cancelled "WUs" if still available?

Michael.

P.S.: Test apps are disabled, right (we are currently working on the user job submission forms)?

[LAURENU2]

I guess you could find them here http://www.rnaworld.de/rnaworld/resu...ames=0&state=5
I really do not know how to give you the info you are asking for.
All I know Is I Do not want to work on Long WU's
I see them I abort them And as long as I see them I wi9ll not port mor power here

And Michael you avoided the question of

Have you even put check point in Yet to save all that time ?

Have you fixed this Problem yet ?