Ogr-ng(26)

[Chuck]

Bruce, you are probably right.... probably nVidia, and I agree since it is the most common and currently best engine out there accessible to the mass user base.

I know I did read it is AMD64 and not X86-64, but that would simply imply it's AMD scheduled (for speed) and not EM64T (Intel scheduling for speed).

Please forgive all my questions as I am looking at this to properly understand how to best use the GPU (assuming it can't really be programmed into an FSM engine... and what would be the fastest way to do the math to a) eliminate non-Golomb ruler sets as well as b) validate a 'proper' ruler in as few machine instructions / memory accesses as possible. I know the guys at D.net have had 8 years to prepare for the '26' effort and I am still 'wet behind the ears' to the problem.... but it does have other applications that I am looking at.... and no, it's not prime finding (that will come).

Another thought is 'data compression'. Sounds strange at first... but if you did 'block compression' (like 1080p HD aka Blue-Ray or MP4 format) video to a PC over slow cable/ std dsl) where much of the individual pixels are the same, building in the CRC/ECC into the data stream for live feed for proper decoding and error correction without retransmit is essential to get the audio & video out efficiently and then display or encoding speed.

I'm just 'thinking out loud' and like you, wondering what all has to be loaded & what are the limitations. Also, I must admit that not having a PS3 here to use for DC apps and/or video is a handicap.... so the ultimate question is.... how much better (if at all) can a current CISC cpu be programmed for those of us who don't have PS3s, but do have 'middle -> upper end' graphics cards because we insist on good quality, dual head operation of our machines being the 'geeks' we are.

Eventually the 'other' math will come into play, but not yet. GPUs by design love to stream their data.... so how best to do it?

I've even wondered what the testing results for OGR were on multi-core machines where it puts one stub per core..... what if it were better to split/pipeline the stub between the cores/cpus (that is what the GPUs are doing). What happens if cpu/core 'a' filters out all the obvious non Golomb constructions and then 'b' determines the 'optimal' adjustments/compliance with other limits.. would we go faster or a lot slower? Could that be part of the 'algorithm improvements' that have happened since OGR-25 ?



Thanks,
C.

[Death]

afaik ogr stub size calculations is linear.
i mean they got a lot of cycles included in each other like

for i = 1 to 100
for j =1 to 200
for k = 1 to 150

compare size of current stub to shortest

next k
next j
next i

kinda like that.

this is a very VERY simplified view but it's real, so you can't make it parallel.

[alpha]

The source is available to download, if that helps, Chuck.

[Digital Parasite]

Oh, and from what I know.... once the ATI weenies finish their brain dump and map the instruction sets.... the new development kit (like for the firestorm, etc) will be regular C code development-level... so perhaps a 'port' to it won't be too painful, but the compiler's internals are going to be a nightmare.

Looks like ATI's new C-code like SDK will be included in the December release of the 8.12 Catalyst drivers. Details here:
http://www.behardware.com/news/lire/24-11-2008/

Should make porting your code to ATI a whole lot easier I would imagine.

[Chuck]

The source is available to download, if that helps, Chuck.

Looks like ATI's new C-code like SDK will be included in the December release of the 8.12 Catalyst drivers. Details here:
http://www.behardware.com/news/lire/24-11-2008/

Should make porting your code to ATI a whole lot easier I would imagine.

THANK YOU! I thought I had seen current/semi-current source available, but could not remember where. Thank you DP for the update on the SDK.

Now, having had *some* time to go over it.... in spite of the .cpp extensions (C++), it appears to be classic C code with the appropriate asm pieces where it is critical. I still have not mapped the entire algorithm flow and structure in my head yet, but think taking the code back to pure .c doesn't seem to be too difficult.... but then i haven't found that one piece of code (which has to be there SOMEWHERE) which would require rewriting...... Murphy's Law always prevails.

One thing I did see is the reference to gcc 3.4..... I currently have 4.1.2 which is a NICE step forward over the horrors of gcc 3.x (at times).

The 1-byte array of data is kinda wild. Haven't seen a monolythic table that big in a while. Making that available to / getting it sliced & stuffed into a GPU properly is going to be fun.

With the rest of the code being mostly regular C and the SDK coming out sooner than I was told it would be.... I have to ask.........Is it worth giving it a shot?


C.

[Digital Parasite]

With the rest of the code being mostly regular C and the SDK coming out sooner than I was told it would be.... I have to ask.........Is it worth giving it a shot?

The SDK itself is available for download now:
http://ati.amd.com/technology/stream.../sdkdwnld.html

Support for it built-into the ATI driver will be available in a couple of weeks.

Are you talking about would it be worth it for your project? Compared to what, writing the assembler code or CTM (Close To Metal?) style code that needed to be done before? This SDK will probably be the main way to write code for ATI cards for the foreseeable future so I would think it would be worth it. Debugging might be a lot easier too.

[Chuck]

Are you talking about would it be worth it for your project? Compared to what, writing the assembler code or CTM (Close To Metal?) style code that needed to be done before?

I'm talking about what it means to us here with OGR.... and getting that back to D.net for their 'blessing' as official. What else we do with the technique/technology for our next target project can be decided later. As for my personal project, it is even more compute intensive so a stub == a WU and that is clearly where to draw that line.

Using the new SDK (and standard they are giving us w/wo CUDA) is part of learning that is going on as well as what is going on in the main app so what is already ASM can stay where it is.... UNLESS it's part of what needs to be put CTM.

Granted, I would love to just put all but the I/O (comm + local files) onboard, but this is a learning process for me compared to my regular job. The 'learning' part is how to slice / scale down (i know that sounds funny) from having 100+ individual cpus/boxen handle the network traffic and 'node' management, where 1 is the 'master', and the nodes below it are grouped into a 4 'render' per 'combiner' and each 'render' node has two cards in it. So the overall structure is Master -> Combiner -> Render machine structure. The Combiner is that strange piece of 'middleware' that speaks both 'render' language as well as does a bit of work on its own and then sends the finished data downstream, status back up, while waiting for the next block/task to be sent to it.... All of this occurring at a crazy rate 75 full 'cycles' per second.

The raw data comes into the master and is parceled out to the appropriate combiner/render nodes, where the 'master' is driven by the application machines (5 + a coprocessor SGI machine)


What I'm looking at now is essentially a 'quad core' (master + combiner) -> 1 or 2 'render' nodes.

The nice part is it's all in one box here.....but also makes it tougher since we can't 'just add another combiner/render set' as needed and the application is part of that task load in the box.



I've sent out an email already to the guy who was the original architect of it all (plus my mentor) and see what advice he has. If I know him, he's already been playing with the new SDK. Heck, he could (and most likely) have had a hand in its development / testing / tweaking.

Now, if you want to grab a quad cpu, quad core machine with 2 cards in a box.... that is something i can mentally directly map to.....

So, given I have only potentially only 4 cores and 1 card is the 'scaling' I need to verify.

Either way, this will get a LOT closer to the metal.... It's just a question of WHERE to draw the line.

Does that make sense? Any thoughts / advice?

I would like to know how you (all) would feel about having one machine be the 'master' and parceling out 'sub-tasks' to the other machine(s) you've configured as part of a 'cluster' on your lan ? Should I even give that any thought? Present thinking for me says 'no' as these are sufficiently compute-intensive *AND* asynchronous compared to massively parallel 'rendering' *AND* the proxy server is really the 'machine' that parcels out the work to be done anyway. true?

I have the proxy on my K6, but run the app on all of my other machines. In my case, I only care that a stub gets done, not by which machine or when. Is this correct thinking?


Maybe that's the real question here..... how do we define a 'cluster' and where are the layer boundaries?


Thanks,
C.

[Chuck]

Rather than edit.... please let me say it here...


I said "75 'cycles' per second".... that would be, in OGR terms, the equivalent of 75 stubs per second.

and....

If you think about the GPUs on a Graphics Card with a single cpu driving it to drive a display (or multi-display group)..... expand that to be a group of cpus (or cores) driving multiple graphics cards to produce multple images, all in perfect sync with each other...

The transition from sync -> async and reducing multiple 22U tall cabinets into a machine or two (or whatever) which make up your pharm.

that help? it's a cpu cluster running one application driving multiple cores which in turn each drive a graphics card.

Sorry for the difficulty in explaining. It's a lot easier if I had a pic or two to show you.


thanks for understanding,
C

[Chuck]

First... hope you all had a great holiday weekend, travelling, relaxing, or whatever.


I downloaded the 508 source by accident, but found it very interesting.... Also got some things working pretty nicely.

Has anyone tried the same and, if so, can we talk about the results you got in pvt ?

Tnx,
C

[gopher_yarrowzoo]

I'll have to power back on this a bit as well for some reason the fan on my GPU has stopped working so I've got another fan on it to keep it cool but due to where it takes air in, I don't know if I can stand the noise of this at full speed in the same room, we'll see...
Well I don't know how it stopped as well playing a game, when screen flickered oddly and then monitor went to power save I was able to exit game and safely reboot windows. It came back up so thought PSU glitch but then it did it again and so I took the rig apart and dusted it, removed all the crap I could, I even took the card out carefully and check it removing some more dust in the process. I put it back in and made it go to BIOS to check and no fan movement thankfully I had a 2 slot fan cooler so I can put it near enough to keep it cool and it's working so far..

[Chuck]

Goph,
I ran into that with my sister's pc.. it was full of tumbleweed to the point both the cpu and gpu cooler fans were a mess. The CPU cooler had stopped and GPU was about to seize.

Luckily with CRC spray lube in the can, I got the fine grit out of the bushings/bearings and got it turning again. (I did alcohol wash the CRC over-spray off the main board (GPU) to prevent attracting other dust. The CPU fan was cleaned and washed externally (grateful for screws to remove it).

Then I put in a drop of 'sewing machine oil' (of all things).... and it has worked at full/proper speed since.

I did add a 'tumbleweed dust grill' to the main air intake and mounts on the outer surfaces of the chassis (120 & 90 mm flavors) behind the outer cover. It keeps the main cause.... pet hair/dander/field dust (corn fields all around) from getting inside.

Since then, there have been no complaints, unexplained shutdowns or other quirks with games..... no need to clean out the 'heavy stuff'.... they just externally vacuum the dust grill and give it an air can blast internally every once in a while.

I do similar for my machines if/when an issue pops up... it's always worked.... The Intel, nVidia (GPU + mobo) coolers are tough little fans to recover to easily.

Never had a problem with my AMD's but they are in a different air quality environment and use a sponge-like filter (washed monthly) across the face of the 4U rack units.

Hope that helps.


C.

[gopher_yarrowzoo]

Thanks Chuck... I'll try that, the thing is it being an AGP card the fan is underneath so a PITA to work on, I'll power down the machine and see what I can see might be a simple fix hopefully else. I guess i'm gonna have to keep the side on it it in future. Time to swap the current "hideaway" I'm using with something a bit more open, or maybe more air flow around about the case...
I'll swap the hideaway even if it looks like a cupboard...

[Chuck]

Goph,
It should recover well, and quite easily.

I did mistype a bit... I meant to write.....

"... they are tough little fans and easy to recover "..


As for recoverying AGP or even PCIE.... AGP's tend to be a bit harder to get the last bit of oil on, but even if you drip it in and let it run down behind the blades onto the main prop shaft, all will be fine... PCIE's tend to be just a tad easier.

FWIW: my ATI 9800SE (AGP) still works fine and has ad the treatment 1 time.

With regard to putting the 'tumbleweed grill' on the front. Yes, please do keep the side cover on. Compensate for cooling with a 90mm temp controlled fan on the rear (venting out), and a nice 120mm internal of that front metal chassis (so the case metal is sandwiched between the fan and dust grill) have that one drawing in air to positive-pressurize the chassis.

That always does the trick for me, as I have a 120mm in front of each stack of HDs inside the case, behind the foam filter for my 4U cases.

C.

[gopher_yarrowzoo]

I've managed to get it spinning back up I think the cleaning of it just took it off spindle a little but I will try and oil it, I got an 80 mm fan at the front as that's what I think the vent in the front is made for I got a rear fan as well but the thing is it goes up and up and up as it's in an fairly enclosed space in my hideaway 3 hdd's + dvd reader & a dvd writer don't help the cause either.
It's running cool now 50-52 rather than about 60 seen it peek at 62

[Chuck]

Goph,
Sounds good! If you can't get some good oil in there... then even some WD-40 or CRC type 'dribbled' in through the spray hoze/tip does help a lot.

I understand about the HD's and burner.... I have 10 drives on the front face, and then 8 more right behind it. The burner is on the front face. Lastly, there are 4 drives (2x2) mounted in a 3.5" frame which is part of the overhead superstructure and case structural integrity.

If I may suggest, check on T-take (thermal take) fans as some of them are the very quiet 'blower' type which can pull / or push a lot of air.

Re cleaning out the AGP.... i know some of those are a pain to clean... If the ductwork on the ATI 9800 gets clogged, I have a lot of work ahead of me.... sometimes even shoving the compressed air nozzle into the duct and giving it a full blast.

With the amount of heat you have.... Do you have an optional 2nd rear fan? If so, may I suggest a temp regulated one?

Here is what I use.... http://www.thermaltakeusa.com and feel free to browse the site and switch to the appropriate UK/EU site as needed.
The main site is http://www.thermaltake.com and it should auto-switch for you. I use the smart fans with remote sensors.... that way I can put the sensor where needed to pull air out & in at and as needed at the key heat locations.

C