Page 1 of 2 12 LastLast
Results 1 to 40 of 54

Thread: Client SPEED enhancement

Hybrid View

Previous Post Previous Post   Next Post Next Post
  1. #1

    Cool Client SPEED enhancement

    I've been thinking about this for awhile but haven't gotten around to it until now. Anyhow, with surprisingly minor code changes, I have sped up the client 200%! :shocked: Yes, that's right, twice as fast, that's not a typo.

    The catch? Well, it loads the entire protein.trj into RAM at once, rather than reading one residue at a time. Thus it will require a total of roughly: 25 + 0.64N MB RAM where N is the number of residues. Thus a 200-residue protein (about the largest we will ever attempt in the near future) would suck up 143MB RAM. Still, a small price to pay for speed doubling, especially if you use your machine only for DF, or if you have 512MB RAM. Note that's EACH process, so a dual machine would need 286MB RAM. Needless to say, this is an 'optional' switch. If should speed up the screensaver as well so an option will be added there as well.

    Just though I'd whet your appetites for what's coming up.
    Howard Feldman

  2. #2
    Senior Member Richard Clyne's Avatar
    Join Date
    Dec 2001
    Location
    Fife, Scotland
    Posts
    621
    :shocked: WOW, you could have knocked me down with a feather when I read that.

    Nice one Brian

    When can we expect this to be available?
    Last edited by Richard Clyne; 07-20-2002 at 02:09 PM.

  3. #3
    Target Butt IronBits's Avatar
    Join Date
    Dec 2001
    Location
    Morrisville, NC
    Posts
    8,619
    Me wants needs it now!
    Don't let Dyyryath have it until I pass him again
    Great work Brian!

  4. #4
    Senior Member KWSN_Millennium2001Guy's Avatar
    Join Date
    Mar 2002
    Location
    Worked 2 years in Aliso Viejo, CA
    Posts
    205
    I would like to beta-test it for say... 10 protein changes... before it is released to the general population or anybody else.

    60 percent of my machines have only 128 megs, 30 percent have 256 and the remaining handful have between 512 and 4gigs. I think I would only be able to make use of it on about 10 boxes. But that is still better than nothing.

    Kool!

    Ni, NI, and Ni!

  5. #5
    That's ashame, 2K1guy, since all 100 of our duals all have 512 MB.
    It will be incorporated just as soon as I test it for a few months on our cluster - mwu-ha-ha-ha. But seiously, should be on next Tuesday.
    Howard Feldman

  6. #6
    Originally posted by Brian the Fist
    It will be incorporated just as soon as I test it for a few months on our cluster - mwu-ha-ha-ha. But seiously, should be on next Tuesday.
    Howard, will this be an option on the client for those machines that can't handle using that much RAM? That is great for people with lots of RAM.

    Jeff.

  7. #7
    Out of curiosity, what would happen if the program overflows RAM and goes to swapfile? I've got plenty of 256MB machines in my little farm, and it seems to me they could benefit greatly from a speed increase even if it is only temporary. Will OS thrashing to disk negate or make worse any benefit from running the program in RAM?

  8. #8
    Vorlon Ambassador to F-DC Kosh's Avatar
    Join Date
    Mar 2002
    Location
    Toronto, Ontario, Canada
    Posts
    88
    Time to get those much needed RAM upgrades
    You've done some really nice work with the client

  9. #9
    Wow.

    Fortunately I hate running anything under 256meg or RAM, so I should see the benefit of this one.

    Great news!

    Sort of nice, seeing the client speed up and not down with updates (ala SETI).

  10. #10
    Ancient Programmer Paratima's Avatar
    Join Date
    Dec 2001
    Location
    West Central Florida
    Posts
    3,296
    Well, that's great news. However, I believe there may be a middle ground solution.

    Currently, Howard, my guess is that you're allocating memory for each residue, reading in the residue and working it over, then re-allocating your work-area for the next residue before/while reading it in.

    On a machine where there is enough RAM to go around, all of this doesn't actually thrash the disk at all, because protein.trj sits happily in the disk buffer. So you're really just "reading" from memory. This can be observed by just watching your hdd light. It lights up for a minute or so when the client starts, then :sleepy:.

    What you will also see, if you watch the memory allocation, is some pretty frantic activity. I believe the problem is that you're spending a lot of time allocating and reallocating the memory. I suspect that a lot of the speed differences we see between OS's is a measure of the relative efficiency of their malloc libraries!

    So what's the middle path? Calculate the largest buffer you need at program init and pre-allocate that amount of memory. You'll save all that time in malloc and realloc, but it won't require a huge area that a lot of boxen don't have.

    Of course, you can still do it both ways, but if the switch says "Don't allocate huge", you can still allocate big-enuff and give everybody a new set of plugs, rotor, and cap.

  11. #11
    Psycho Penguin dnar's Avatar
    Join Date
    Dec 2001
    Location
    Perth, Australia
    Posts
    111
    Originally posted by Halon50
    Out of curiosity, what would happen if the program overflows RAM and goes to swapfile? I've got plenty of 256MB machines in my little farm, and it seems to me they could benefit greatly from a speed increase even if it is only temporary. Will OS thrashing to disk negate or make worse any benefit from running the program in RAM?
    I doubt the DF client data would be pushed to swap, as stuff not accessed for a long time is pushed first.

  12. #12

    Talking



    This is why we love you Howard!

  13. #13
    Does 200% faster mean 3 times as many results to upload?
    That will be a real pain on my existing bandwidth.

    Will the download interval take into account the speed (or even be user settable? )


    Mind you, on second thoughts it might not be an issue since I only have four machines with more than 256Mb (out of 63).

  14. #14
    Bottom of the Top Ten TheOtherZaphod's Avatar
    Join Date
    May 2002
    Location
    zone 5 west
    Posts
    100
    One of my favorite math police crimes:

    200% faster than = 300% as fast as

    Let's hope Howard is not guilty.
    Don't Panic

  15. #15
    Originally posted by Gunslinger
    Does 200% faster mean 3 times as many results to upload?
    That will be a real pain on my existing bandwidth.

    Will the download interval take into account the speed (or even be user settable? )
    200% faster should mean 3 times as many results to upload.

    There is no download interval, do you mean upload interval? ie: how often it will upload a batch of structures?

    With the current version of the DF Client there is a new -s witch you can use to set the upload interval. The lowest being 999 which means use the server default, and then you can range it from 1000 to 10000 to manage how often your client will upload structures.

    Jeff.

  16. #16
    If you actually READ my post (why am I thinking broken telephone here..) it says it is 'sped up 200%, meaning twice as fast'.
    Thus we'll be doubling production (roughly) no tripling.
    Im not releasing the extra 100% speedup from 200% to 300% for awhile so you still have something to look forward to
    Howard Feldman

  17. #17
    Senior Member Richard Clyne's Avatar
    Join Date
    Dec 2001
    Location
    Fife, Scotland
    Posts
    621
    Originally posted by Brian the Fist
    If you actually READ my post (why am I thinking broken telephone here..) it says it is 'sped up 200%, meaning twice as fast'.
    Thus we'll be doubling production (roughly) no tripling.
    Im not releasing the extra 100% speedup from 200% to 300% for awhile so you still have something to look forward to
    You have to give us something to look forward to next Tuesday

  18. #18
    Originally posted by Brian the Fist
    If you actually READ my post (why am I thinking broken telephone here..) it says it is 'sped up 200%, meaning twice as fast'.
    Thus we'll be doubling production (roughly) no tripling.
    I was so excited to see the 200% speedup that I stopped reading there and probably didn't want to think that it was really only a 100% speedup like you meant to say.

    Jeff.

  19. #19
    Howard, don't listen to them; you may call it whatever you like as long as I generate structures twice as fast. I have a few people on my team that I must thwack!

  20. #20
    Psycho Penguin dnar's Avatar
    Join Date
    Dec 2001
    Location
    Perth, Australia
    Posts
    111
    Originally posted by bubbadog
    Howard, don't listen to them; you may call it whatever you like as long as I generate structures twice as fast. I have a few people on my team that I must thwack!
    Doof, and I was looking forward to a client that would process in half the time. Oh well.

  21. #21
    I can understand the confusion on 100% and 200% but where did 300% come from ? 2x or 4x the performance but 3x the perfromance would be 150% no ?

    It's all a bit mute anyhow as Brian mentioned doubling in the first post and we all knew what he meant, except for the guys who still insist that 2001 was the start of the new millenium and hadd a very lonely party with lots of uneaten jelly and mini pork pies.




    Regards

    Andy

  22. #22
    Registered User
    Join Date
    Apr 2002
    Location
    Singapore
    Posts
    23
    WIll it be equavalent to the use of RAM-disk ? I used ramdisk on my AMD 1.2GHz, but I only get about 10% reduced in time.

  23. #23
    Senior Member wirthi's Avatar
    Join Date
    Apr 2002
    Location
    Pasching.AT.EU
    Posts
    820
    Originally posted by RipItUp
    I can understand the confusion on 100% and 200% but where did 300% come from ? 2x or 4x the performance but 3x the perfromance would be 150% no ?
    No, you are wrong about that. 100% AS fast = equally fast. 200% AS fast = twice as fast. 150% AS fast = 1.5 times as fast = right in the middle between equally and twice as fast ....

    OR:

    100% FASTER = twice as fast. 200% FASTER = three times as fast . 150% FASTER = 2.5 times as fast (example: 1000 strucutes before, now 2500).

    So Howard actually said "three times as fast", while he meant "twice as fast" (as far as i remember)

    Greets,
    Wirthi

  24. #24
    Senior Member
    Join Date
    May 2002
    Location
    New Jersey USA
    Posts
    115
    What Mr Fist didn't tell you is that
    all new proteins will be 400% slower

  25. #25
    Howard - excellent work!!!

    Michael.
    http://www.rechenkraft.net - Germany's largest distributed computing community

    - - - - - - - - - -
    RNAs are nanomachines or nanomachine building blocks. Examples: The ribosome, RNase P, the cellular protein secretion machinery and the spliceosome.

  26. #26
    Psycho Penguin dnar's Avatar
    Join Date
    Dec 2001
    Location
    Perth, Australia
    Posts
    111

    Will the server(s) take the extra speed increase?

    Everyone is very excited about the new cleint speed-up of 200%/2 times/3 times as fast but I must ask, will the server(s) handle the extra load of work being returned twice as often? It appears that we have not yet seen an update period where the server(s) have handled the load without issue....

    Just a thought.

  27. #27
    Ok, I bit the bullet and ordered a T1 link today. Will be a few weeks in coming, but should sort out the bandwidth issues at my end ( especially if there are only 100% more results to upload )

    Can't do much about the RAM situation though - I just hope that it doesn't thrash too much on the lower memory machines as that will interfere with the normal users. I did some experiments today with a simple memory allocator - problem is that the low priority active task swaps out high priority 'idle' tasks, so there is a noticable delay when the user goes active again

  28. #28
    Psycho Penguin dnar's Avatar
    Join Date
    Dec 2001
    Location
    Perth, Australia
    Posts
    111
    Originally posted by Gunslinger
    I did some experiments today with a simple memory allocator - problem is that the low priority active task swaps out high priority 'idle' tasks, so there is a noticable delay when the user goes active again
    Your must be using Winblows. Try that on Linux my friend!

  29. #29
    Originally posted by dnar

    Your must be using Winblows. Try that on Linux my friend!
    I would love to, unfortunately due to the nature of the technology we develop we are pure Micro$oft/Intel (the latter helps loads for DF tho' ).

    I'm beavering away on using something different for the next generation of servers. The changes to the licensing are giving me a lot of leverage at the moment - I think they have shot themselves in the foot by being too greedy

  30. #30
    Psycho Penguin dnar's Avatar
    Join Date
    Dec 2001
    Location
    Perth, Australia
    Posts
    111
    Originally posted by Gunslinger

    I think they have shot themselves in the foot by being too greedy
    M$ Greedy? No!

    GNU = Good!

  31. #31
    Not using the -rt parameter :-

    138AA
    46800 structures per day
    mem useage 25MB

    using -rt parameter

    138AA
    111600 structures per day
    mem useage 115MB



    Does what it says on the tin.

    Regards

    Andy

  32. #32
    Gosh - your machines are that memory-starved, M2kG? Wow... Sucks to be you... BWAHAHAHAHAHAHAHAHAHA! (Glad I put 256M in all the toasters. Downside is that they're all SDR. [pout])

    Hmmm, looks like as soon as I can find the time to change all those boxes over to the new option - you may just be gettin' a HURTIN' my favorite nemesis! BWAHAHAHAHAHAHAHAHAHAHA!


  33. #33
    Senior Member KWSN_Millennium2001Guy's Avatar
    Join Date
    Mar 2002
    Location
    Worked 2 years in Aliso Viejo, CA
    Posts
    205
    I just placed an order for 70 256meg sticks.

    RAM is dirt-cheap, and think of the performance increase the end users will get.

    Ni!

  34. #34
    Good idea M2k+1G...time to buy up some of those $20 128MB sticks and upgrade the farm...

  35. #35

    ???

    I've only got SDRAM...Whats this DDR stuff...
    By the way..I couldn't find a service.config file for the -dt? command line or whatever..???
    I installed Jeffs GUI v1.7 and ticked the box, and it seems to work o.k??
    I know how to cover things with chocolate, but I don't mind saying I'm a bit confused...Ah well each to their own..
    I still know that a Billion is 1,000,000,000,000 though...:|punch|:
    ATB
    Ian AKA GOLDENBALLS

  36. #36
    Senior Member KWSN_Millennium2001Guy's Avatar
    Join Date
    Mar 2002
    Location
    Worked 2 years in Aliso Viejo, CA
    Posts
    205
    DDR is just like SDRAM, but it sends data twice per clock cycle. Your chipset must support it for it to work for you. SDRAM is slightly cheaper at the moment, but the prices are jumping back and forth, so buy low, sell high.

    Ni!

  37. #37

    Cool Cheers M2K+1 Guy!

    NI!
    What happened to your outing with The KWSN who say Ni (without an ! ?)
    They must be a dour lot not to have risen to the infiltration!
    and they don't even have a message board for taunting...
    Oh well!
    One looks forward to the new ammo you will install in your boxen.
    Just don't hire the Wabbitt to do it!
    Is he still flirting with G@H??
    Sad Bunny...
    ATB,
    GBSY

  38. #38
    Jodie and KWSNM2001 make me laugh. What a couple of megalomaniacs

    Next they'll be upping the FSB speed for " the benefit of the end users " ..heh.

    My XP1500 at 1476 MHz outperforms my XP1800 at 1600 Mhz due to the increased FSB.

    No doubt Jodie will respond to KWSNM2001 's increased RAM by buying a 4 gallon pot of silver conductive paint for the L1's

    Regards

    Andy

  39. #39
    I have a tiny advantage - I don't have to care about the 'end users' 'cause there aren't any... My whole farm is devoted 100% to df...

    Every AMD I have is unlocked and OC'd - which is 90+% of the farm - I *am* on the OCN team, afterall!

    Heya, M2kG - did you add more machines a couple weeks ago? Just curious...

    And I'm not a megalomaniac - I just need to win. BWAHAHAHAHAHAHHHAHAHAHA!

  40. #40
    Bottom of the Top Ten TheOtherZaphod's Avatar
    Join Date
    May 2002
    Location
    zone 5 west
    Posts
    100

    Talking

    General rule of thumb: Just stay out of their way, and nobody gets hurt.

    RAM prices aren't nearly as "rock bottom" as they have been. Fortunately most of my machines were already 256MB.
    Don't Panic

Page 1 of 2 12 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •