Ogr-ng(26)

[Brucifer]

So are there any "in-the-know" people around here regarding the client development. While it appears that the PS3's aren't going to be crunching on this, is there any known work going on with the nvida video cards? Can they be used or are they going to fall victim to the same issue as the ps3's?

[LAURENU2]

Of Of-coarse there are people "in-the-know" here
Please continue to hold your Post is vary important to us and it will be answered in the order received

[PCZ]

I'm hoping for a GPU client as well.

BTW
Were is everyone, R@R is over report for duty immediately.

[alpha]

Someone wrote a CUDA core for RC5-72 capable of 125 Mkeys/sec but I don't think it ever got finalized or released to the masses. The author's comments on OGR at the time were:

I agree that OGR is much more interesting but reading the documentation surrounding the core algorithm, the non-constant execution times may present a significant challenge WRT getting good performance out of a GPU implementation. But, I haven't looked too closely at the underlying code, so my comments may be irrelevant.

I personally haven't heard of any developments in the GPU field since. It doesn't sound hopeful, but with a new project having only just started hopefully it will spawn more interest in the area.

[gopher_yarrowzoo]

I'm hoping for a GPU client as well.

BTW
Were is everyone, R@R is over report for duty immediately.

Sorry "Sweetheart" was only crunching this one till OGR-25 was over to try and maintain position, the kindly donated dualie loved it but right now is offline due to erm well it also loved eating massive chunks of power ... Yeah I saw the hike in power usage and knew it had to be that as I've got other running and the power usage isn't as noticeable - IF I could manage to transfer over all that on my one remaining Win98se box - i.e it's running FTP / Web / MySQL servers I'd do that and close down that box totally, but I'd need to recreated the directory structures etc, bit slow over the wire to transfer it yeah that and I'm a lazy sod - but it will get done one day , when I find a project worthy of the dual power, well the 98 box is a AMD X2200+ so it's not slow just old....

[Brucifer]

You let a little thing like power consumption stand in the way of the team accumulating high points in OGR?????

Actually, get rid of those old amd's and get a new quad or a couple 60 watt X2's, and your power will go down. I was running a bunch of the old stuff, and shifted over to a few quads and some 60w X2's, and my power bill dropped by half......... that totally impressed the spouse!!

[Brucifer]

Ahhh... my perproxy is working just fine.

So it looks like we be in #4 spot, but there are some teams ramping up now. Those cows have already got a (pardon the pun) herd of members.

[gopher_yarrowzoo]

Ahhh... my perproxy is working just fine.

So it looks like we be in #4 spot, but there are some teams ramping up now. Those cows have already got a (pardon the pun) herd of members.

Oh time to MOOOOve on from that baaaad joke..

Yeah I powered down a X2000+ for that very reason it was the really early model ate power i think, it's not the chip swap that bothers me it would be a total replacement system and where do I off load the old one.. Oh wait I think I know, just need a case :P can give it as a christmas pressie

[Chuck]

Ahhh... my perproxy is working just fine.

So it looks like we be in #4 spot, but there are some teams ramping up now. Those cows have already got a (pardon the pun) herd of members.

With the help of a few friends, A WHOLE LOT OF TRUST AND PRAYING.... I setup and switched to the D.net Perproxy ON MY K6-2!!!!!!....


Yes, K6-2 500Mhz, stripped down just fine w/2k install. It DOES have a 100 Mbit Enet card in it and given it's running nothing else, all the non-essential services are OFF.

Now, thanks to help from Brucifer and a bit of experience using the other proxies, I took a shot at the transition NOW instead of tonight. SO gang..... If I have LITTLE to NO points on OGR-NG, you will know why. I do think I botched one machine, but hope the fallback to the Dnet server caught it. It looks like it did.

Either way, The K6-2 sits there, quiet as a can be, serving up my movies (yes, it has a few *nice* drives on it) AND serving up my OGR packet needs within 2 seconds (typically 1) of the cruncher deciding it needs to send/receive packets!

I hardcoded the IP address in each cruncher to avoid resolv.conf time (every cycle counts) and it's working like a champ.

So, unless i *lost* data in the switchover, things are a whole lot more efficient here now. Given I put the K6 on the appropriate side of the routers and have a 'magic port' open *somewhere*, should the need ever arise at the end of the project, I will be willing to supply data for the thirsty to the limits of the proxy & Inet.


Now, if there were a proper eOn proxy..... .


Again, thanks to all. A K6-2 is now very happy again being able to do SOMETHING useful !!!


Tnx,
C.


PS: I know many of us wanted to do 'pet projects' after OGR-25, but would some consider coming back????

[Death]

there IS eOn proxy.

http://www.pubquizcentre.nl/DigiK-oz/dceon.aspx

well, sort of...

[Chuck]

Thanks,
However, the Target OS isn't compatible, IIRC. I'm a linux shop here with 2 W/2k boxen. Last I knew, and PLEASE correct me if wrong, the proxy does require the .NET which is W/XP or better?

Tnx,
C

[Death]

.net 2.0 that eon proxy use 2k compartible

http://www.microsoft.com/downloads/d...displaylang=en

System Requirements
Supported Operating Systems: Windows 2000 Service Pack 3; Windows 98; Windows 98 Second Edition; Windows ME; Windows Server 2003; Windows XP Service Pack 2
Required Software:
o Windows Installer 3.0 (except for Windows 98/ME, which require Windows Installer 2.0 or later). Windows Installer 3.1 or later is recommended.
o IE 5.01 or later: You must also be running Microsoft Internet Explorer 5.01 or later for all installations of the .NET Framework.

Disk Space Requirements: 280 MB (x86), 610 MB (x64)

[Chuck]

Thanks very much... that's how little I remember of the MS world..... Last I used it for anything more than simple web stuff was before I went to FC5.

Tnx again,
C

[gopher_yarrowzoo]

I've thrown a PC at this, dropped it from SoB to this, so should help out.

[IronBits]

I finally got around to adding the PS3 pharm (all 3 of them) on OGR and on RC5 at the same time.
Looks like I can make 3-4T per day on OGR without any impact on the RC5 output.
to the developers

[Chuck]

I've got another machine 'breaking in' on this + some extra horsepower left over from what isn't needed for my mathematics work..... It's been added

FWIW: If you can run the linux client, you can have about 30-40% more output. Windows just wastes too much due to the lack of separation of the threads, so they bounce around between the cpu caches like ping-pong balls.

Otherwise, I like this version and look forward to the polished final release client.

C.

[Digital Parasite]

Is it just me or has http://stats.distributed.net been down for the past couple of days?

Even http://downforeveryoneorjustme.com/ is saying they can't find a DNS entry for it.

[wirthi]

Check the plans: http://n0cgi.distributed.net/cgi/dne...?user=mikereed

Our stats system is down. We apologise for any inconvenience this may cause.
More to follow.

[Chuck]

I would think it's down. Their DNS is responding and resolving... You can see it if you watch it in the FF status bar.

I get the 'The server at stats.distributed.net is taking too long to respond.' message, which I don't get unless there is a problem.


C.


/me feels dumb now.... we posted at the same time...

[alpha]

The stats are back and fully up to date.

Ars Technica slipped past us at more than double our rate for yesterday.

[Digital Parasite]

The stats are back and fully up to date.

Ars Technica slipped past us at more than double our rate for yesterday.

I think a lot of people are bailing on their ABC@Home push and switching to other projects like OGR-26

[alpha]

Yeah, I saw the thread over there and was in awe at the amount of computing power which was collaborated in such a short time. I didn't realise either that 100% on ABC@home did not mean the project was over, so I'm glad I tracked the thread.

We've still only got a handful of people running OGR-26 so hopefully when the client gets out of pre-release we can rally up a bit more support.

[Digital Parasite]

We've still only got a handful of people running OGR-26 so hopefully when the client gets out of pre-release we can rally up a bit more support.

I see that the 1 and 3 day averages are showing project end in Jan and Feb so we might whip through this thing. I was originally planning on putting most of my systems on OGR-26 in January to help finish it off but it looks like I will start doing that next week.

I wonder when they finished the new client algorithm. Since the new algorithm makes that much of a difference they really should have used it part-way through OGR-25 as well unless it was just finished recently.

[Brucifer]

look:
http://n0cgi.distributed.net/cgi/dne...gi?user=bovine

10% done already.

[alpha]

That's an interesting update Brucifer!

We're working on updates for some of the remaining platforms (like
NetBSD and OpenBSD) plus some exciting new ones (like nVidia CUDA), so look forward to future announcements.

Digital Parasite, the amount of processing required for OGR-26 was supposed to be significantly greater than for OGR-25, but due to some changes (not sure if it was the algorithm itself or some other change or both) the quantity of work was heavily reduced.

IIRC, they said the new algorithm should be able to decide sooner when to stop processing the current stub, which effectively reduces the average stub size over what it would have been with the old algorithm. But also, it seems there must have been some other change to reduce the overall number of stubs to check because we went from 304,857,742 stubs for OGRp2-25 to 9,967,533 for OGR-26. Perhaps it wouldn't have been possible to have made such changes mid-way through a project.

[Brucifer]

That's an interesting update Brucifer!

I thought it was too! The CUDA info will be interesting to watch. Initially they said they had some severe memory constraints with the PS3 too, but they have a working client for that which is interesting too. As Ironbits mentioned that he could get a lot of rc5 work done at the same time as the ogr-26 without seeming to impact the ogr-26 crunching. So I'm watching the CUDA issue too.

The other thing that surprised me was that they said the project was about 10% done. That's rocking along, and it isn't going to be that long of a project. A lot of processing power that was on the ogr-25 effort at the end has moved back to other efforts, but even so, ogr-26 seems to be moving along pretty good. I wouldn't be surprised to see the completion date move closer as teams realize they aren't going to have as much time to do ogr-26 and other stuff as it initially appeared. Time will tell.

[alpha]

Actually, the memory problems with the PS3 client are still ongoing. With the latest version of the client, they just have it crunching OGR on the PPE but none of the six (available) SPEs.

If/when we have a working CUDA client unleashing the kind of processing power we expect we could be bashing through the OGR projects in no time, not to mention increasing the chances of finishing RC5-72 within our lifetimes! I'd be more tempted than ever to buy the appropriate hardware and get it running.

[Digital Parasite]

Too bad there is such good support for CUDA and nothing for ATI cards. If only NVIDIA or ATI would make CUDA compatible.

[alpha]

Wow, what a find when I wake up this morning:

• Updated PS3 client which now supports OGR-NG on the SPEs
• CUDA client for AMD64 Linux, RC5-72 support only! (no OGR-NG yet )

Find them here!

[IronBits]

I'm not ready to burn up my expensive gaming card to crunch numbers.
The video card puts out LOTS-O-HEAT and sucks up more watts than a cpu to.
Thanks for the linkage to the updated dnet client for CellBE

[Death]

what kind of cooler are in your video card?

[IronBits]

Stock, and I don't OC it either.
A GTX280 is plenty fast enough for what I need.

[Digital Parasite]

Wow, what a find when I wake up this morning:

• Updated PS3 client which now supports OGR-NG on the SPEs
• CUDA client for AMD64 Linux, RC5-72 support only! (no OGR-NG yet )

So they were able to squeeze the new OGR-NG algorithm into the SPEs then? IronBits, how fast does that puppy run on your PS3s?

[Digital Parasite]

I'm not ready to burn up my expensive gaming card to crunch numbers.
The video card puts out LOTS-O-HEAT and sucks up more watts than a cpu to.
Thanks for the linkage to the updated dnet client for CellBE

What?!?!? You mean you bought an expensive gaming card to play games on? What kind of a DC fanatic are you...

[the-mk]

[IronBits]

Holy crap!

OGR output went way up!
http://stats.free-dc.org/dnet/ogr/byemail.html
~101 Mnodes/s

RC5 output went down because OGR is using all 7 cores
http://stats.free-dc.org/dnet/rc572/byemail.html

dnetc.ini shows
project-priority=OGR-NG,RC5-72

I was using
project-priority=OGR-NG=1,RC5-72=1

[Nov 26 15:19:36 UTC] Automatic processor detection found 7 processors.
[Nov 26 15:19:36 UTC] Loading crunchers with work...
[Nov 26 15:19:36 UTC] Automatic processor type detection found
                      a Cell Broadband Engine processor.
[Nov 26 15:19:36 UTC] OGR-NG: using core #1 (KOGE 3.0 Hybrid).
[Nov 26 15:19:36 UTC] OGR-NG #a: Loaded 26/2-7-33-14-15 (131.86 Mnodes done)
[Nov 26 15:19:36 UTC] OGR-NG #b: Loaded 26/2-7-39-11-26 (171.58 Mnodes done)
[Nov 26 15:19:36 UTC] OGR-NG #c: Loaded 26/2-8-11-1-49 (169.85 Mnodes done)
[Nov 26 15:19:36 UTC] OGR-NG #d: Loaded 26/2-8-15-16-40 (171.72 Mnodes done)
[Nov 26 15:19:36 UTC] OGR-NG #e: Loaded 26/2-9-5-12-27 (167.87 Mnodes done)
[Nov 26 15:19:36 UTC] OGR-NG #f: Loaded 26/2-9-14-10-46 (172.27 Mnodes done)
[Nov 26 15:19:36 UTC] OGR-NG #g: Loaded 26/2-9-20-12-4 (169.37 Mnodes done)
[Nov 26 15:19:36 UTC] OGR-NG: 168 packets remain in buff-in.og2
[Nov 26 15:19:36 UTC] OGR-NG: 0 packets are in buff-out.og2
[Nov 26 15:19:37 UTC] 7 crunchers ('a'-'g') have been started.

[Chuck]

Catching up / verifiying if you don't mind? I would like to make sure I'm in sync about CUDA / SPE-type/based GPUs and OGR.


first the obvious.... The OGR algorithm is a Linear Algebra / Combinatorics problem, right? If so, then memory and access time (like texture maps) along with list construction/destruction (like display lists) uses just the core instruction/capability set of a GPU/GPU-set... and the big gain comes from having so many processsors (like IB's machine) ?

So, your results IB mean they are testing the support in the OGR code as well but not saying anything publicly yet????




ANd, if I understand CUDA correctly, two CUDA support cards can use the SLI as well to gang up a bunch/pair of medium grade cards using SLI, provided there's enough main application CPU data bus speed to feed them (and do other housekeeping) for yet more fun? That right?

If so, what about accessing the spare/extra P4 in the xboxen? We can use that too, can't we?

Regarding the ATI side.... AMD is doing the CUDA support now. Granted a bit late, but it is apparently coming for the new Firestorm side, but how much do you want to bet we end up with ""limited"" support for the rest of the line?

Thoughts? Knowledge? Ideas? Wish lists? WHO DO WE WRITE TO???



Thanks,
C.

[Bok]

IB's is a PS3 running linux (x3 - was x10 at one point )..so yes, more SPE's all being utilized now.

If Microsoft would release any code then sure, XBOX360 could be utilized, I just imagine with all the trouble they normally get this would make it worse...(red ring of death)

SLI Nvidia can be utilized as two seperate cards I believe, not sure if it is seen as one, though I'm sure that's possible..

ATI, most likely but down the line once the instruction set is more understood, seeing as it is only now being released...

Bok

[Chuck]

Thanks Bok,
Wasn't sure of the machine etc.... thanks.


And yes, I agree that MS is probably VERY reluctant to let us use any more of the xboxen given how many melted down the first time around, but it would be nice to hack it.... i've been told, but not seen it done, that accessing the processor is not a big deal... the issue is overtasking it and the heat it produces.... Please correct me if I understood my son wrong, but MS had to fix/recall/whatever a bunch of machines because the now 'spare' P4 inside (could be used for housekeeping and offload one of the properly cooled mains ??) was once allowed to be used at 'full strength'... hence the Red Ring of Death and overhaul that "didn't quite live up to expectations" ???

I would love to see what IB gets over the long haul on the machine.... I see the memory fragmentation that OGR currently causes.


Oh, and from what I know.... once the ATI weenies finish their brain dump and map the instruction sets.... the new development kit (like for the firestorm, etc) will be regular C code development-level... so perhaps a 'port' to it won't be too painful, but the compiler's internals are going to be a nightmare.

One can hope?????


Thanks again,
C.

[Brucifer]

I'm probably all wet, but as I understand it the CUDA/nvida is only being tested on the rc5-72 stuff right now on 64-linux. What I'm not aware of and wondering is what all hast to be loaded onto the host system for the cuda stuff to work??

As for the spe's on the ps3, that is supposed to be intentional with the latest beta release for the cell ogr.