PDA

View Full Version : Distcc & Gentoo: Thanks, Kosh!



Dyyryath
08-02-2003, 05:00 PM
A couple of days ago, Kosh reminded me about distcc (http://distcc.samba.org) as a way to speed up the massive amount of compiling that you tend to do under Gentoo. I'd used it in the past, but it never really dawned on me to use with Gentoo. :blush:

After Kosh mentioned it, I promptly kicked myself, and then set about getting it up and running. I wanted to use it with the fastest machines at my disposal, which would mean my gaming boxes. Of course, these boxes all run Windows XP, so I decided to pull out on of my favorite apps: VMWare (http://www.vmware.com). I loaded a stripped down version of Linux (about 180mb) into a virtual machine on each of my gaming boxes with the necessary tools to compile with distcc under gcc 3.2. I'm just letting the VMs run 24/7 on these boxes so my 'compile farm' is always available. When it comes time to play, we just 'Suspend' the VM, play, and then 'Restore' the VM when we're done. Works like a champ.

So now, when I go to compile something that used to take 8 hours, it only takes about 45 minutes. Smaller apps don't take much more time than it would take to install from binaries. :thumbs:

How do you know it working? Well, when you're compiling (and using a distcc monitor), you see something like this:


http://www.zerothelement.com/offsite-images/distcc-gentoo-small.png

If you're using Gentoo, and you have multiple boxes on your network, I'd strongly suggest you give distcc a try. It can definately make your life much easier! :thumbs:

You can find some information about running distcc with Gentoo right here (http://dev.gentoo.org/~lisa/distcc/distcc.html). If you need to run distcc on Windows machines, you can either use something like VMWare (http://www.vmware.com) (as I did), or if the $299 price tag is a bit out of your reach, you can use cygwin (http://www.cygwin.com/) (which was my first thought). While cygwin's bound to be more complicated, there's some info here (http://www.xraylith.wisc.edu/~khan/software/gnu-win32/cygwin-to-linux-cross-howto.txt) that might be of use.

Chinasaur
08-02-2003, 07:09 PM
Wow!! That's a hell of a time differential!!!

So how much faster is Gentoo over a Debian??

Chinasaur
08-02-2003, 09:08 PM
Gentoo article - http://articles.linmagau.org/modules.php?op=modload&name=Sections&file=index&req=viewarticle&artid=227

Dyyryath
08-02-2003, 10:53 PM
That was a good idea, but probably not the best execution possible. Some of it seemed to kind of scattered and haphazard. ;)

The major distros are pretty easy to test, you just use the stock kernel & settings. Gentoo, however, can be a real bear. Which kernel did you use? How was it configured when you compiled it? What optimizations & USE flags did you use when compiling the software you used?

Honestly, Gentoo's greatest strength can also be it's greatest drawback: if you start tinkering and don't know what you're doing, you can easily end up with a *slower* system. In the motorcycle world, we often see racers who go out and buy a good solid bike to race on. Then they begin 'adjusting' the timing, 'cleaning up' the head, 'rejetting' the carbs...I've seen guys spend months souping up a new race bike so that it's almost as fast as it was when it was stock. :D

Gentoo's a lot like that, too...

I've been experimenting with benchmarking on my own over the last couple of days. Chinasaur's questions on how much performance difference you can get with Gentoo got me curious. With that in mind, I think I'm going to do something similar to what they tried to do using Red Hat 9, Mandrake 9.1, Libranet Debian, and my current install of Gentoo.

I'll use the same box for each distro, reformatting a spare 40gb drive each time to ensure that the hardware isn't in question. For the Gentoo install, I'll just dd my current drive over to the test drive to keep everything even. I've made some concessions to performance for the sake of stability with my current Gentoo install, but it should be a pretty good representation of a moderately tuned version of the distro.

Obviously I'll use the Distributed Folding client's benchmarking as one test (we have certain priorities around here, you know ;)), and the standard Linux kernel compile as another. I'm partial to gaming (yes, even under Linux), so I'll probably throw some Q3 & UT2k3 benchmarks in there as well. I also do quite a bit of work with GIMP so I'll add a few filter & scriptfu benchmarks, too. I might even try to make a couple of big code editing macros in Komodo to test with. Beyond that, does anyone have anything specific they'd like to see?

If we can come up with something that makes sense to us all, I'll be happy to do the leg work...

Chinasaur
08-02-2003, 11:13 PM
When do you sleep?? :)


:notworthy

magnav0x
08-02-2003, 11:58 PM
WARNING....magnav0x has forgot to goto sleep and has left a long unuseful rambling beflow. Continue at your own risk:


Benchmarks! My second favorite statistics! Thanks for going out of your way to do this for us Dyyranth.

I for one feel that Gentoo could give a WIDE variaty in benchmarking. Due to scalability and customization (especialy if starting from stage 1 tarball when building it) can lead to inconsistant benchmarking numbers. Gentoo is not like Redhat, Debian, or Mandrake where a majority of "basic" linux users will choose some sort of default. Also, the distributions set their own USE, CFLAGS, etc which will make a difference to the machine. I would think it's safe to assume that most gentoo users at least set appropriate USE, CFLAGS, and CXXFLAGS that are best suited for their machine. Did the benchmarker do this? Or did you just fly through the install from a stage 3 and not bother to "tweak" gentoo as it was realy meant to be?

Honestly just from running Gentoo for a short period (only about a week now :Pokes: ) I'm no gentoo expert and I'm realy far from being an expert with Linux in general. I can see a great amount of speed increase over Redhat which was my linux flavor of choice before. Then again this can be due to the many extra services that Redhat likes to throw in for those who choose to just pick an "install package" and add whatever programs they want. I never realy took the time to try and tweak and minimalize other distributions, because I realy don't have a FULL understanding of all the aspects of linux filesystems and system files/dependencies as I would with a Windows platform.

Building Gentoo from the ground up has taught me a great deal more about the linux infastructure. Other than taking a year and a half (that's in magnav0x time ) to get the basic structure up and going it was not difficult as all. Gentoo provides a VERY in depth and helpful install guide on their website and their forums have just about any question you could ever think of covered. I could easily imagine a intermediate linux user being able to get their own box up and running gentoo even from a Stage 1 install. They may not fully understand the aspects of optimizing the distro for their machine's specific hardware, but I guarantee they would come out much more familiar with kernel compiling by the time they got it all set the way they like :fart:.

In short...I couldn't tell you if Gentoo is the greatest performer on the market. A gentoo install seems more or less to depend on the users choices during the "building" stages. However I will stick with Gentoo for the long haul and dub other distributions (especialy Redhat and Mandrake) as "obsolete". Of course everyone has different needs and those who just want to "run linux" and/or have a weak heart, I'd suggest to stick to what you are comfortable with.

This rambling has been brought to you by emerge....have you had to search for dependencies lately?

Kosh
08-03-2003, 03:22 AM
Originally posted by Dyyryath
Obviously I'll use the Distributed Folding client's benchmarking as one test (we have certain priorities around here, you know ;)), and the standard Linux kernel compile as another. I'm partial to gaming (yes, even under Linux), so I'll probably throw some Q3 & UT2k3 benchmarks in there as well. I also do quite a bit of work with GIMP so I'll add a few filter & scriptfu benchmarks, too. I mght even try to make a couple of big code editing macros in Komodo to test with. Beyond that, does anyone have anything specific they'd like to see?


Maybe something for compression as well ... time bzip2 or maybe something else. Sounds amazing :notworthy :notworthy

Whenever you get that done I might just use some of the same benchmarks for some varitions of a Gentoo install. I've been meaning to get around to see what 2.6-test brings to the 'folding table, and also play around with Resier4 and see what it can accomplish before it completely destroys the drive :p

About distcc ... I've never really bothered trying to setup up my cygwin slackers to help compile for my Gentoo boxes, thanks for the link :thumbs:.

magnav0x
08-05-2003, 11:13 PM
Just installed cgywin on my Win 2003 server box and I'm gonna see if I can get distcc up and going on it. I'll let you guys know how it goes :D

magnav0x
08-09-2003, 04:40 PM
Well I did get distcc and a cross platform compiler set up between my gentoo box and cygwin on windows. The only problem is when I compile something on the gentoo box it never outsources to the windows box for help. I had a long night getting cygwin set up to function correctly and I realy wasn't up for trying to figure it out. So I'll mess with it next time I have free time and I'm up for an adventure.