A NetBoot Notes thread

[rsbriggs]

I'm going to re-install from scratch this evening, and use this thread to keep notes on things as I go along - to better help someone that comes along behind me. We can make this a kind of Wiki if anyone wants - feel free to keep one step ahead of me....

The general philosophy of what I'm building here is discussed at this link:

http://www.extremeoverclocking.com/articles/howto/FAH_Diskless_Farm_1.html
Another resource along the same lines:
http://ws9.jobnegotiator.com/html/netboot.html

I'm going to separate my network into two pieces, or subnets, allowing me to completely separate my farm network from my Windows based home network.

Dual NIC machine to be used as a farm server 3.2 Ghz Dell 400 SC Poweredge server box. - 2 NIC cards, one a RealTek PCI card added in, the other is an e1000 intel "card" on the motherboard.

eth0 - (PCI card) Realtek RTL-8139 SMC EZ
eth1 - Intel Ether Express on the main motherboard.


Step #1 :thumbs:
Perform a full install of Red Hat Shrike - release 9 - select custom for the type,
and at the bottom of the install screen in the MISC section - select the box marked "EVERYTHING". and continue. This takes a while....

The install is being done from downloaded ISO images from http://www.linuxiso.org/distro.php?distro=7. Burned them onto CD. Downloading these takes a while, too. (DAYS on a slow connection)

NOTES:

During set up I set both NICs up as having static IP addresses, one lives on the Windows network, the other on the farm network.

eth0: static 192.168.0.2 - windows net
eth1: static 192.168.2.254 - farm net
hostname: DELL400SC
set the gateway to my internet routing machine, which lives at 192.168.0.1
no firewall configured

when asked about packages, go to the misc section at the bottom, and check the EVERYTHING box. (This really isn't necessary....)

Finish the install.... Nap. Eat dinner. Whatever.....

(probable next step, download the LTSP stuff - I'll have to refer back to PCZ's notes about exactly what version he installed and post the link. Feel free to cut-n-paste or enter that info into the next message here, PCZ...)

Edit by IronBits
Anything that does not pertain to PXE NIC, CD, Floppy, Net Booting, or helpful in any way will be removed from this thread to keep it concise and to the point for referencing. :Pokes:
End of Edit by IronBits

[rsbriggs]

some notes for step 2

Boot into your nice shiny new Red Hat install. Pop up Mozilla, and check the net connection. In my case, Success !! (At least the windows 1/2 of the network connection is working as expected. )

:thumbs: Step 2 - download LTSP stuff

Following PCZs directions, I headed over to http://ltsp.org/ and downloaded the following:

ltsp_core-3.0.9-0.i386.rpm
ltsp_kernel-3.0.9-0.i386.rpm
ltsp_local_apps-3.0.0-0.i386.rpm
pxestuff-3.0.5-i386.tgz

I also grabbed the following files

ltsp_initrd_kit-3.0.9-i386.tgz
and
ltspcfg from the 4.0 section. This is a new PERL script that is supposed to make setting up an LTSP system "easier".

But, ... Before I install any of this stuff, I'm going to go through the several hour process of running up2date, registering this machine with Red Hat, getting a demo account, and upgrading everything. I know for a fact that the kernel has been through a number of upgrades and fixes since this distribution was released, and I really want to get the SMP kernel updates that have been done in the last six months.

NOTE: Of course, the only time that I've ever seen the Redhat update service down is ... right now. Their update service is saying "due to technical difficulties, the service is temporarily unavailable."

Ran the ltspcfg script. Used it to build all the entries (253) in /etc/hosts and /etc/hosts.allow - it saved a lot of manual, by-hand work.....





======================
The rpm's are installed like this:
rpm -ivh ltsp_core-3.0.9-0.i386.rpm

The pxestuff archive has a readme in it telling you where to put the files.

I installed Red Hat 9 with the custom option and selected everything.
I didn't really need all the apps but I was playing safe.

I edited the config files using Jasons article as a guide.
I can help with the config files if you get stuck.

[PCZ]

Instructions for people who aren't Guru's


[Downloading the ltsp packages]

log into Gnome and run mozilla.
download these files from ltsp.org (http://ltsp.org)

ltsp_core-3.0.9-0.i386.rpm
ltsp_kernel-3.0.9-0.i386.rpm
ltsp_local_apps-3.0.0-0.i386.rpm
pxestuff-3.0.5-i386.tgz

[Installing the ltsp packages]

Double click the home icon and you should see the downloaded files.
Just double click on them and let the RPM package manager install for you.

Unarchive pxestuff-3.0.5-i386.tgz
Just double click on it and extract the pxestuff folder.

Open the pxestuff folder, click on Edit then Select_ALL Files from the Edit menu
All the files and the pxelinux.cfg folder should turn blue.
Click the right mouse button inside the folder and choose copy files from the menu.

Click the up button until you get to / and open the tftpboot folder
Open the lts folder, click the right mouse button inside the open lts folder and select paste from the menu.

All the files from the pxeboot folder should now be inside the tftpboot/lts folder including the pxeclient.cfg subdirectory.


[Creating some working directories]

Click back a couple of times until you are at /
Right click some empty space inside the folder and select New Folder
Rename the new folder DF

Open the DF folder
Right click some empty space inside the folder and select New Folder
Rename the new folder ws001

Right click some empty space inside the folder and select New Folder
Rename the new folder ws002

You need to create a ws folder for all your nodes, so if you are planning on having more than two, create more folders inside DF.
Just increment the trailing digit by one each time.

Click the up arrow and you will be back at /
Double click opt and it will open with one folder inside called ltsp.
Double click the ltsp folderand you will see a i386 folder.
Double click the i386 folder and it will open up.
Right click on some empty space inside the folder and select new folder.
Call this folder DF


end of part one

[rsbriggs]

Couldn't wait for Red Hat to fix their technical difficulties, so we continue on.

:thumbs: Step 3

As root, open a terminal window. Navigate to where the downloaded files were placed. Install the downloaded files by entering

rpm -ivh ltsp_core-3.0.9-0.i386.rpm
rpm -ivh ltsp_kernel-3.0.9-0.i386.rpm
rpm -ivh ltsp_local_apps-3.0.0-0.i386.rpm
tar zxvf pxestuff-3.0.5-i386.tgz

then do
cd /opt/ltsp/templates
./ltsp_initialize

and just apply the changes. Then change to the directory where pxestuff unpacked, and follow the instructions in the README file. The last step in the README is to copy the file /etc/dhcpd.example to /etc/dhcpd.conf and edit it. I left it as it was, except that I had to change all instances of 192.168.0 to be 192.168.2 AND
change the file so that the shared network WORKSTATIONS section looked like this:



shared-network WORKSTATIONS {
subnet 192.168.2.0 netmask 255.255.255.0 {
range 192.168.2.101 192.168.2.200;
}





(at this point, I'm impatient, so I'm going to try to start everything in a default configuration)

/etc/rc.d/init.d/portmap restart
/etc/rc.d/init.d/named restart (gave me an error message, but started)
/etc/rc.d/init.d/xinetd restart
/etc/rc.d/init.d/dhcpd restart
/etc/rc.d/init.d/nfs restart (some error messages, but started)

I tail -f /var/log/messages and start my reference board, a VIA EPIA 800 Mhz mini. I see several DHCPDISCOVER and DHCPOFFER 192.168.2.200 to the via, but it complains that no file name was received. At this point, I at least have DHCP going. Now to edit a few more config files, and get this puppy to boot....

[rsbriggs]

Looking around the net some, I find the following little trick that can be used in the /etc/dhcpd.conf file :



shared-network WORKSTATIONS {
subnet 192.168.2.0 netmask 255.255.255.0 {
range dynamic-bootp 192.168.2.100 192.168.2.253;
use-host-decl-names on;
option log-servers 192.168.2.254;

# trick from Peter Rundle <peter.rundle@au.interpath.net>
if substring (option vendor-class-identifier, 0, 9) = "PXEClient"
{
filename "/lts/pxelinux.0";
# NOTE: kernels are specified in /tftpboot/lts/pxe/pxelinux.cfg/
}
else
{
filename "/lts/vmlinuz-2.4.21-ltsp";
}
}
}



and my reference board TRIES to boot. Got a good message in the /var/log/messages file - rpc.mountd: refused mount request from ws253.ltsp for /opt/ltsp/i386 (/): no export entry.

OOOps - have to edit the /opt/ltsp/i386/etc/lts.conf file - change references to 192.168.0 to be 192.168.2

[Default]
SERVER = 192.168.0.254
should be 192.168.2.254

It's looking in the wrong subnet for the server

(Also change RUNLEVEL = 5 to be a 2 instead... This would change the netbooted startup from an Xterminal log into the server to just a local bash shell - when the board boots, you can actually run things locally on the netbooted board. X isn't set up correctly to run, so you have to do this, I guess.)

And one last thing I found that I missed, you have to change the 192.168.0.0 entries in /etc/exports to be 192.168.2.254, then restart the NFS server with /etc/rc/.d/init.d/nfs restart

fiddle with /opt/ltsp/i386/etc/lts.conf - add nearly everything including RCFILE_01 = startfah to the default section.

add startfah to /opt/ltsp/i386/etc/rc.d with a /bin/bash --login -i

AND MY REFERENCE BOARD BOOTS and runs a bash prompt....


whew... tomorrow fiddle with /opt/ltsp/i386/etc/rc.local to mount a local /home on the netbooted card, change my fah script to start something in the home directory, and I'm ready to run DF on netbooted boards.


Notice that I don't tie my MAC addresses to any entries in dhcpd.conf file - I just let server pass out addresses to the netbooted boards.... Guess I'll have to change that - if you don't do the assignment, the client board doesn't receive a hostname....

[rsbriggs]

Well, found that I really DO need to tie my mac addresses to a hostname in dhcpd.conf

Starting on trying to get the M7NCD Pro board booted - it uses the Nforce-2 chip set.... On first try, it simply hangs during the PXE boot up sequence.... PCZ had a fix for that....

Need to download the file syslinux-2.05.tar.gz from ftp://ftp.kernel.org/pub/linux/utils/boot/syslinux

unzip and untar the thing. find the file pxelinux.0 and copy it to the /tfptboot/lts directory.


Boots - but kernel panics - no NIC driver.... Hmmm. Easiest way to fix this???? Rebuild the kernel, or find one. Let me take a quick look around ltsp and k12ltsp web sites....


OK - the exact problem to be solved here is to get the file nvnet.o into the directory
/opt/ltsp/i386/lib/modules/2.4.21-ltsp-1/kernel/drivers/net

However, I'm going to try re-building the most current ltsp kernel, rather than to attempt to find a compatible nvnet.o -file. Who knows, we might run into some other module later (like on the next board we try) that needs to be loaded, so why not go for the most general solution....

Procedure is to download the file lbe-4.0-beta1.tgz
tar xvzf lbe-4.0-beta1.tgz
cd lbe
./build_all

this gets all the pieces as source code and compiles everything (including the compiler.) The docs say this takes a couple of hours... Once this completes, it should be possible to take the generated kernel and modules, and replace the other ltsp stuff. Might have to re-configure the kernel and re-build it afterwards, too. Don't know yet....

I'll let everyone know in a couple of hours.....

Edit:

What the docs fail to mention is the DOWNLOAD time - this new set up makes building LTSP look nearly exactly like building a Gentoo system! And, given my recent experience in that area - there are more than a few hours of DOWNLOADING to be done.... So, still going..... Currently starting a 56 Meg download of the XFREE subsystem. (Think about downloading the 188 Meg ISO image they have posted.)

Edit: It's a couple of hours later, and it's just started to download the kernel sources. Another update in a couple MORE hours.....

[PCZ]

Attempting to run net-booted clients gives me this error
Can't open terminal: Linux

The DF program is trying to use linux terminal to draw to the screen, the remote nodes don't support this

The Fix:

Before you let the clients run on the remote nodes run through setup on the server.
Go into the nodes working directory on the server [Delete handle.txt if it exists]
Run ./foldit and you will be prompted for your handle, type it in.
Go through the setup questions and say you dont want any output to terminal.
When setup is finished the program will quit.
Copy this directory and use it for all your other remote nodes.

[PCZ]

DF needs a larger Ramdrive than default
To increase the size of the Ramdisk:

1:edit /tftpboot/lts/pxelinux.cfg/default
Add the ramdisk_size=32768 command.

prompt=0
label linux
kernel bzImage-2.4.21-ltsp-1
append ramdisk_size=32768 init=/linuxrc rw root=/dev/ram0 initrd=initrd-2.4.21-ltsp-1.gz


2:edit /opt/ltsp/i386/etc/rc.local
Look for this section:

echo "Creating ramdisk on /tmp"
RAMDISK_SIZE=`get_cfg RAMDISK_SIZE 1024`
/sbin/mke2fs -q -m0 /dev/ram1
/bin/mount -n /dev/ram1 /tmp

Comment out this line: RAMDISK_SIZE=`get_cfg RAMDISK_SIZE 1024`

echo "Creating ramdisk on /tmp"
#RAMDISK_SIZE=`get_cfg RAMDISK_SIZE 1024`
/sbin/mke2fs -q -m0 /dev/ram1
/bin/mount -n /dev/ram1 /tmp

3:edit /opt/ltsp/i386/etc/lts.conf
If there is any reference to ramdisk size either change the value to 32768
or comment the line out with a #

[rsbriggs]

Originally posted by PCZ
Attempting to run net-booted clients gives me this error
Can't open terminal: Linux

The DF program is trying to use linux terminal to draw to the screen, the remote nodes don't support this

The Fix:

Before you let the clients run on the remote nodes run through setup on the server.
Go into the nodes working directory on the server [Delete handle.txt if it exists]
Run ./foldit and you will be prompted for your handle, type it in.
Go through the setup questions and say you dont want any output to terminal.
When setup is finished the program will quit.
Copy this directory and use it for all your other remote nodes.

The "better fix" (tm)

This is happening because the ncurses (libncurses.so.4, which is probably symlinked to libncurses.so.5) needs terminal info to be able to display anything on the screen. It USED to be in termcap, but is now stored in the terminfo directory.

1.) Find out where the terminfo directory is on your system (probably /usr/share)
2.) make the same directory under /opt/ltsp/i386/ if yours was in /usr/share, then do: mkdir /opt/ltsp/i386/usr/share/terminfo
3.) now copy all the contents of the one from your system over to the clients directory:

cp -R /opt/ltsp/i386/usr/share/terminfo /opt/ltsp/i386/usr/share/terminfo

boot the client. Now you can actually run ./foldit on a client machine, and watch the pretty ascii drawings, if you want.....

[rsbriggs]

Checklist for setting up net-boot farm server, providing separate sub-domains and doing IP masquerading. Each item on the checklist has an expansion message....


1.) Install Red Hat 9.0
2.) Download and install the LTSP (terminal server) software
3.) Setup PXE
4.)



Additional References:

General Overview:

http://ws9.jobnegotiator.com/html/netboot.html
http://www.extremeoverclocking.com/articles/howto/FAH_Diskless_Farm_1.html

[rsbriggs]

Checklist #1 - download and install Red Hat Linux 9.0

Use a download manager / download speed-up software like IDM (www.internetdownloadmanager.com) for doing big downloads (like downloading the distribution .ISO images.) It makes a BIG difference speed wise (4x in my case), and you can pause/resume downloads and recover from interrupted downloads due to power or ISP outages. At 256K DSL connection speeds, it will take about 4 hours per .ISO image, and there are three images in the Red Hat 9.0 set. One good place to download Linux distributions from is http://www.linuxiso.org

Why Red Hat? I spent WEEKS investigating other distributions, like Mandrake 9.1, Gentoo, Knoppix installed to the hard drive, Debian, SUSE via FTP install, and so on. Red Hat may not be the "sexiest" distro, or provide support for the absolute latest-and-greatest hardware, or even run things the fastest, but it is a very solid distribution. All of the additional packages we are going to install are readily available as Red Hat .RPM files, which simplifies the installation process considerably. Other distros may work for you - if you have a favorite then feel free to use it - but you are on your own as far as modifying the procedure presented here to work for your particular distribution.

Another note about downloading and installing - I had several CD's "go bad" on me for no apparent reason after an initial period of working with no problem (I'm now working off my third copy of the initial Red Hat install disk.) I recommend that you spend the few minutes it takes to do the CD test at the beginning of the install, rather than just skipping over it.

If space isn't an issue for you, then do an "everything" installation, by selecting "custom" when asked for the type of install. On the next screen, scroll down to the bottom, and check the "install everything" box. This will put a few megabytes of things out there you probably have no use for, like Slovakian and Ukrainian man pages, or Greek KDE support. If this is really bothersome, also check the little box at the bottom where it says "individual package selection". On the next screen you will be able to de-select packages that you don't want to install.

After installing, I recommend that you spend the time to connect to the Red Hat network and set up a demo entitlement. This will allow you to update anything that has changed or been upgraded since the distribution was released. In my case, it takes about 4 hours to download everything including the latest kernel release.

[rsbriggs]

Checklist Item 2

Assuming that you have correctly installed your new Red Hat, you should now be booted into your nice new install, and have network connectivity. Make certain that both of your network cards are set up, and active.

Fire up your choice of browsers, and head to the following link: http://sourceforge.net/project/showfiles.php?group_id=17723
(Since links change over time, this is where the "download software" selection of http://www.ltsp.org takes you)

Download the following .RPM files from the 3.0 section:

ltsp_core-3.0.9-0.i386.rpm
ltsp_kernel-3.0.9-0.i386.rpm
ltsp_local_apps-3.0.0-0.i386.rpm


and these additional files

pxestuff-3.0.5-i386.tgz (PXE boot files)
ltspcfg (from the 4.0 section. This is a new PERL script that is supposed to make setting up an LTSP system "easier".)


NOTE:
grab this file from ltsp.org: ltsp_kernel-3.0.10-i386.tgz
in place of the 0.9 version


As root, install / unpack the packages like so:

rpm -ivh ltsp_core-3.0.9-0.i386.rpm
rpm -ivh ltsp_kernel-3.0.10-0.i386.rpm
rpm -ivh ltsp_local_apps-3.0.0-0.i386.rpm (optional step)
tar zxvf pxestuff-3.0.5-i386.tgz

then do:

cd /opt/ltsp/templates
./ltsp_initialize

and choose to apply the changes.

The last step will create several files that you can use as a starting point for further modification, such as dhcpd.conf.example. After editing this file, it would be moved to the same name minus the .example on the end.

In fact, when you do an accept, the following files will be modified on your system (this doesn't include the X related files, since we won't be running X on the clients):


/etc/dhcpd.conf.example Example config file for dhcp
/etc/exports The config file for nfs
/etc/hosts.allow Configuration file for tcp wrappers
/etc/inittab Config file for init
/etc/rc.d/rc5.d/S60nfs Startup links for nfs
/etc/rc.d/rc5.d/S13portmap Startup links for portmapper
/etc/sysconfig/syslog Startup info for syslogd
/etc/xinetd.d/tftp Enable the tftp daemon


---end---

[IronBits]

Change to the directory where pxestuff unpacked, and follow the instructions in the README file. The last step in the README is to copy the file /etc/dhcpd.example to /etc/dhcpd.conf and edit it. Ok, I'm with you so far and I'll get to the README tonight :thumbs: :)

Ok, so I stayed home... ;)

I am all confused :blush:

Let me give you my network setup.
This way you can get me going in the right direction the first time :jester:

DNS Server 192.168.2.2 (dbestern.com)
DHCP 192.168.2.2 range 100-122 (dbestern.com)

Redhat SERVER 192.168.2.20 (epox.dbestern.net)
DHCP Server 192.168.2.20 range 50-75 (dbestern.net)
TFTP Server 192.168.2.20

GATEWAY 192.168.2.1

I want the PXE clients on the dbestern.net, using dynamic IPs.
I have followd PCZ and rsbriggs to a "T" I think.
except for Bob's perl script.

Give me the steps
1) edit this
2) edit that
3) go here edit this and that
4) reboot 6x, cross all toes, bang head on kitchen counter twice
5) anything I missed ? ;)

thanks! :thumbs:
I can feel the "POWER" coming soon :D

[rsbriggs]

Are you down through as far as this ?


As root, install/unpack the packages like so:

rpm -ivh ltsp_core-3.0.9-0.i386.rpm
rpm -ivh ltsp_kernel-3.0.9-0.i386.rpm
rpm -ivh ltsp_local_apps-3.0.0-0.i386.rpm
tar zxvf pxestuff-3.0.5-i386.tgz

then do:

cd /opt/ltsp/templates
./ltsp_initialize

I'll be continuing on from that point this evening - don't panic.....

Now what I'm describing builds something along the following lines, and assumes that you have a box with two NICs, and that you are going to be using a different subnet for the "farm" to keep it separate from the other network:

windows <-- 192.168.0.x ---[ net-boot server ]--- 192.168.2.x---> Linux-farm

The net-boot server will be doing IP-Masquerading and routing for the 'farm side". There is a separate NIC in the net-boot server for each side of the network. I believe that eth0 (->current windows network) on my box is a RealTek card, and that eth1 (->the new farm subnet) is an e1000 card on the server motherboard.

Funny that it finds the add-in card before finding the NIC on the motherboard.....

[IronBits]

Yes, done all that - this will only have
one NIC card,
one subnet, all on
one LAN, no need for Masquerading and routing ;)
Typical home LAN setup, protected by an SMC router/firewall. :)
I just need to know what files need to be edited and how to get it working on the first ws001 node, step by step...
Thanks!

[rsbriggs]

Then...maybe PCZ could jump in here??

I'm uncertain just how to set this up on a single sub-net. The problem being, I don't know how, exactly, to guarantee that a DHCP request gets handled by the Linux server, rather than being handled by whatever box is currently handling DHCP requests on your network (possibly, your router?)

( Assuming that there is a simple solution, I wouldn't mind running mine in that fashion, as well. Trouble is, I have an "internet connection sharing" box on the windows network that believes that it has the right to dole out IP addresses in response to DHCP requests.... )

[PCZ]

You do not need to setup 2 networks.

All you need to do is turn off the existing DHCP server on your network.

You can easily set up dhcpd to do the work that your existing DHCP server did as well as support PXE clients.

Here is my dhcp.conf.

The PXE strings should be one long line not split over 2 .



# Sample configuration file for ISCD dhcpd
#
# Make changes to this file and copy it to /etc/dhcpd.conf
#
ddns-update-style none;

default-lease-time 432000;
max-lease-time 864000;

option subnet-mask 255.255.255.0;
option broadcast-address 172.31.158.255;
option routers 172.31.158.250;
option domain-name-servers 172.31.158.100;
option netbios-name-servers 172.31.158.100;
option domain-name "itsoapbox.com";
option root-path "172.31.158.240:/opt/ltsp/i386";
option log-servers 172.31.158.240;

shared-network WORKSTATIONS {
subnet 172.31.158.0 netmask 255.255.255.0 {
range 172.31.158.1 172.31.158.99;
}
}
group {
use-host-decl-names on;
host ws001 {
hardware ethernet 00:E0:18:D9:1A:02;
fixed-address 172.31.158.101;
filename "/lts/pxelinux.0";
option vendor-encapsulated-options 09:0f:80:00:0c:4e:65:74:77:6f:72:6b:20:62:6f:6f:74:0a:07:00:50:72:6f:6d:70:74:06:01:02:08:03:80:00:0 0:47:04:80:00:00:00:ff;
}
host ws002 {
hardware ethernet 00:0C:6E:2D:DE:BD;
fixed-address 172.31.158.102;
filename "/lts/pxelinux.0";
option vendor-encapsulated-options 09:0f:80:00:0c:4e:65:74:77:6f:72:6b:20:62:6f:6f:74:0a:07:00:50:72:6f:6d:70:74:06:01:02:08:03:80:00:0 0:47:04:80:00:00:00:ff;
}
host ws003 {
hardware ethernet 00:0C:6E:2D:DE:93;
fixed-address 172.31.158.103;
filename "/lts/pxelinux.0";
option vendor-encapsulated-options 09:0f:80:00:0c:4e:65:74:77:6f:72:6b:20:62:6f:6f:74:0a:07:00:50:72:6f:6d:70:74:06:01:02:08:03:80:00:0 0:47:04:80:00:00:00:ff;
}
host ws004 {
hardware ethernet 00:0C:6E:2D:DE:8F;
fixed-address 172.31.158.104;
filename "/lts/pxelinux.0";
option vendor-encapsulated-options 09:0f:80:00:0c:4e:65:74:77:6f:72:6b:20:62:6f:6f:74:0a:07:00:50:72:6f:6d:70:74:06:01:02:08:03:80:00:0 0:47:04:80:00:00:00:ff;
}
host ws005 {
hardware ethernet 00:0C:6E:2D:DF:9E;
fixed-address 172.31.158.105;
filename "/lts/pxelinux.0";
option vendor-encapsulated-options 09:0f:80:00:0c:4e:65:74:77:6f:72:6b:20:62:6f:6f:74:0a:07:00:50:72:6f:6d:70:74:06:01:02:08:03:80:00:0 0:47:04:80:00:00:00:ff;
}
}

[rsbriggs]

Every instance of 172.31.158 needs to be replaced with your network address, like 192.168.2 (IronBits) or 192.168.0 (me).

Looks like IB has the same issue with DHCP servers:



DNS Server 192.168.2.2 (dbestern.com)
DHCP 192.168.2.2 range 100-122 (dbestern.com)

Redhat SERVER 192.168.2.20 (epox.dbestern.net)
DHCP Server 192.168.2.20 range 50-75 (dbestern.net)
TFTP Server 192.168.2.20


Might have to turn off the dbestern.com DHCP?

The example in PCZs file is assigning fixed hostname/IP address based on the
MAC address of the connecting node.



host ws002 {
hardware ethernet 00:0C:6E:2DE:BD;
fixed-address 172.31.158.102;



And passing out other addresses in the range of .1 to .99



shared-network WORKSTATIONS {
subnet 172.31.158.0 netmask 255.255.255.0 {
range 172.31.158.1 172.31.158.99;
}
}

[PCZ]

172.31.158.240 is the linux boot server.
172.31.158.250 is my ADSL router

[IronBits]

How do I find out the MAC address of the NICs as they boot up?
Is there a way around that, so that it will just pick one and use it?

[rsbriggs]

You have to look at the board. If they are in a case , you might be able to get around this by taking a look at /var/log/messages on the server as they attempt to boot up - I believe the requesting MAC address shows in the DHCP request message that gets logged.

You could not use MAC addresses, and simply allow the boards to get assigned an IP, but......

Maybe my system was mis-configured before, but I found that in that case they didn't end up knowing their host-name. If you want to set up host-specific directories for them to log into on the server, this is a problem....

(You can maybe do it via IP address instead of host-name? Hmmm. Interesting idea... )

[PCZ]

connect up an ethernet cable to the board boot it up and it will attempt to PXE boot
if it detects a network then you will see the mac address on the screen.

[IronBits]

Aug 19 13:57:35 epox dhcpd: DHCPNAK on 192.168.2.105 to 00:e0:4c:a6:70:ae via eth0
Aug 19 13:57:40 epox dhcpd: DHCPDISCOVER from 00:e0:4c:a6:70:ae via eth0
Aug 19 13:57:40 epox dhcpd: DHCPOFFER on 192.168.2.201 to 00:e0:4c:a6:70:ae via eth0
Aug 19 13:57:40 epox dhcpd: DHCPREQUEST for 192.168.2.201 (192.168.2.20) from 00:e0:4c:a6:70:ae via eth0
Aug 19 13:57:40 epox dhcpd: DHCPACK on 192.168.2.201 to 00:e0:4c:a6:70:ae via eth0
Aug 19 13:58:22 epox dhcpd: DHCPRELEASE of 192.168.2.201 from 00:e0:4c:a6:70:ae via eth0 (not found)
It did get IP .201 ok :)
Am I close? :confused:

[PCZ]

Dont forget to edit the hosts file. /etc/hosts
You must have an entry for each node for host declared names to work properly

Here is my hosts file.

# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 Red-Hat-Serv localhost.itsoapbox.com localhost
172.31.158.240 server
172.31.158.101 ws001
172.31.158.102 ws002
172.31.158.103 ws003
172.31.158.104 ws004
172.31.158.105 ws005
172.31.158.106 ws006
172.31.158.107 ws007
172.31.158.108 ws008

[IronBits]

Ok, done that - now this
Aug 19 13:58:52 epox dhcpd: DHCPOFFER on 192.168.2.201 to 00:e0:4c:a6:70:ae via eth0
Aug 19 13:58:52 epox dhcpd: DHCPREQUEST for 192.168.2.201 (192.168.2.20) from 00:e0:4c:a6:70:ae via eth0
Aug 19 13:58:52 epox dhcpd: DHCPACK on 192.168.2.201 to 00:e0:4c:a6:70:ae via eth0
It is getting IP .201, but doesn't boot from NIC yet.
HOSTS
127.0.0.1 epox localhost.localdomain localhost
172.31.158.20 server
192.168.2.201 ws001
192.168.2.202 ws002
192.168.2.203 ws003
192.168.2.204 ws004
192.168.2.205 ws005
192.168.2.206 ws006
dhcpd.conf
ddns-update-style none;

default-lease-time 432000;
max-lease-time 864000;

option subnet-mask 255.255.255.0;
option broadcast-address 192.168.2.255;
option routers 192.168.2.1;
option domain-name-servers 192.168.2.2;
option netbios-name-servers 192.168.2.2;
option domain-name "dbestern.net";
option root-path "192.168.2.20:/opt/ltsp/i386";
option log-servers 192.168.2.20;

shared-network WORKSTATIONS {
subnet 192.168.2.0 netmask 255.255.255.0 {
range 192.168.2.50 192.168.2.99;
}
}
group {
use-host-decl-names on;
host ws001 {
hardware ethernet 00:e0:4c:a6:70:ae;
fixed-address 192.168.2.201;
filename "/lts/pxelinux.0";
option vendor-encapsulated-options 09:0f:80:00:0c:4e:65:74:77:6f:72:6b:20:62:6f:6f:74:0a:07:00:50:72:6f:6d:70:74:06:01:02:08:03:80:00:0 0:47:04:80:00:00:00:ff;
}

[PCZ]

You have one of these missing } at the end of the file.

Restart the dhcpd service after you edit the dhcpd.conf.

From the start menu in the GUI
System Settings Server Settings Services

Highlight dhcp and restart if your config is wrong it wont start.

[IronBits]

It's there, I just didn't post the rest of the workstations below, thus the missing closing brace. dhcpd is running fine... PXE is not finding anything...
Biostar M7VIQ Rhine II VT6013 cards... maybe I should toss in a 3com PXE boot card ;)
Says on the boot screen

Intel UNDI, PXE-2.0 (build 082)
...
VIA Rhine II Fast Ethernet Adapter v2.13 (2002/08/09)

PXE-MOF: Exiting Intel PXE ROM.

BOOT DISK FAILURE <--- becuase I have everything else disabled for boot up sequence.
tail /var/log/messages shows
Aug 19 14:09:57 epox dhcpd: DHCPREQUEST for 192.168.2.201 from 00:e0:4c:a6:70:ae via eth0
Aug 19 14:09:57 epox dhcpd: DHCPACK on 192.168.2.201 to 00:e0:4c:a6:70:ae via eth0

[PCZ]

Try another card, dont forget it will have a different mac address.

Also there must be no spaces in the vendor options string and it MUST be one continuous line.

[IronBits]

OK, now I'm getting excited!!!

TFTP timeout! :) I'm close... I must not have done the tftp server part yet.

NEXT?!?!?! ;)

[PCZ]

Do you have a tftpboot directory ?
/tftpboot

there should be a directory called lts in there.

you need the files from the pxestuff archive extracted into lts.

make sure the tftp service is started.

[IronBits]

Ok, ran the tftp script rsbriggs had to get tftp server running ok. :D
See something about 16 RAM disks of 32678 each?
bunch of other stuff, then...
running dhclient
mouting root filesystem: /opt/ltsp/i386 from: 192.168.2.20

[PCZ]

you need to create some directories if you haven't already

This is what I did.

Create a directory called DF /opt/ltsp/i386/DF
Create a directory called DF /DF
Create subdirectories off /DF

/DF/ws001
/DF/ws002

and so on one for each node.

you need to edit /etc/exports
---------------------------------------------------------

## LTS-begin ##

#
# The lines between the 'LTS-begin' and the 'LTS-end' were added
# on: Tue Aug 5 08:08:30 BST 2003 by the ltsp installation script.
# For more information, visit the ltsp homepage
# at http://www.ltsp.org
#

/opt/ltsp/i386 192.168.2.0/255.255.255.0(rw,no_root_squash,sync)
/var/opt/ltsp/swapfiles 192.168.2.0/255.255.255.0(rw,no_root_squash,async)

/DF/ws001 192.168.2.0/255.255.255.0(rw,no_root_squash,sync)
/DF/ws002 192.168.2.0/255.255.255.0(rw,no_root_squash,sync)
/DF/ws003 192.168.2.0/255.255.255.0(rw,no_root_squash,sync)
/DF/ws004 192.168.2.0/255.255.255.0(rw,no_root_squash,sync)
/DF/ws005 192.168.2.0/255.255.255.0(rw,no_root_squash,sync)

## LTS-end ##

----------------------------------------------------------

[rsbriggs]

Is he going to need to change 192.168.2.0 in /etc/exports
to his servers address -> 192.168.2.20 ??

[PCZ]

No

The 192.168.2.0 is the subnet.
What this means is that any access from that network will get root rights

[IronBits]

Did the /etc/export edit PCZ
There is nothing in /DF/ws001 is that right?

Also, the Server locks up right after it says dfclient :confused:

[PCZ]

Have you edited /opt/ltsp/i386/etc/rc.local yet ?

Also you need to make the ramdrive bigger if it is still 1024 k.

[IronBits]

Originally posted by PCZ
Have you edited /opt/ltsp/i386/etc/rc.local yet ?

Also you need to make the ramdrive bigger if it is still 1024 k.

It says I have 16 RAM disks of 32 something each!

I'm swapping out motherboards/NICs - the damn thing keeps lockiing up... give me a few.

What would I edit rc.local to? Think NEWBIE I haven't a CLUE what I'm doing yet.

[PCZ]

are you trying to run the dfclient already and if so in what directory.

[IronBits]

I don't have a clue here :bang: /DF/ws001 I guess

On the client, I now see eth0: setting full duplex...
so I'm getting further - I'd settle for a shell prompt # or something at this point ;)

[PCZ]

IB

I thought you had the node booted to the bash prompt already.

Try and describe how far you are getting in the boot process.

[IronBits]

See something about 16 RAM disks of 32678 each?
bunch of other stuff, then...
running dhclient

mouting root filesystem: /opt/ltsp/i386 from: 192.168.2.20
etho: setting full-duplex mode

then it just sits there, doesn't go any further...

I noticed ROOT owns /DF/*.* and perms are 744

[PCZ]

What version are the files in the tftpboot/lts directory ?

Looks like a nic driver problem.

[PCZ]

If the files in your lts directory are .19
you need to upgrade to .21


grab this file from ltsp.org
ltsp_kernel-3.0.10-i386.tgz

[rsbriggs]

Usually the ram disk size results in a kernel panic....

How about trying this first:


Download the file syslinux-2.05.tar.gz from ftp://ftp.kernel.org/pub/linux/utils/boot/syslinux

unzip and untar the thing. find the file pxelinux.0
and copy it to the /tfptboot/lts directory.

That hung me up during boot-up....

[rsbriggs]

My bad - I think my instructions point towards the 0.9 version....

[PCZ]

IB

I had days of frustration with this but the end result justified all the pain.
Hang on in there.

[IronBits]

drwxr-xr-x 4 root root 4096 Aug 19 16:24 .
drwxr-xr-x 4 root root 4096 Aug 19 05:55 ..
-rw-r--r-- 1 root root 679237 Aug 19 07:42 bzImage-2.4.19-ltsp-1
-rw-r--r-- 1 root root 731855 Aug 19 07:42 initrd-2.4.19-ltsp-1.gz
-rw-r--r-- 1 root root 11300 Aug 19 16:24 pxelinux.0
drwxr-xr-x 2 root root 4096 Aug 19 08:17 pxelinux.cfg
drwxrwsr-x 6 1026 1026 4096 Jul 16 15:24 syslinux-2.05
-rw-r--r-- 1 root root 1617920 Aug 19 16:21 vmlinuz-2.4.21-ltsp-1
-rw-r--r-- 1 root root 1656832 Aug 19 16:21 vmlinuz-2.4.21-ltsp-lpp-1
I think step 3 hasn't been done in rsbriggs's setup :(
Still stops at the same place... duplex mode... :cry:
It also hogs up all the 100mbps network with traffic and makes it impossible to ssh to :(

[IronBits]

This is as far as I have gone ...


Originally posted by rsbriggs
Checklist Item 2

Assuming that you have correctly installed your new Red Hat, you should now be booted into your nice new install, and have network connectivity. Make certain that both of your network cards are set up, and active.

Fire up your choice of browsers, and head to the following link: http://sourceforge.net/project/showfiles.php?group_id=17723
(Since links change over time, this is where the "download software" selection of http://www.ltsp.org takes you)

Download the following .RPM files from the 3.0 section:

ltsp_core-3.0.9-0.i386.rpm
ltsp_kernel-3.0.9-0.i386.rpm
ltsp_local_apps-3.0.0-0.i386.rpm

and these additional files

pxestuff-3.0.5-i386.tgz (PXE boot files)
ltspcfg (from the 4.0 section. This is a new PERL script that is supposed to make setting up an LTSP system "easier".)


NOTE:
grab this file from ltsp.org: ltsp_kernel-3.0.10-i386.tgz
in place of the 0.9 version


As root, install / unpack the packages like so:

rpm -ivh ltsp_core-3.0.9-0.i386.rpm
rpm -ivh ltsp_kernel-3.0.10-0.i386.rpm
rpm -ivh ltsp_local_apps-3.0.0-0.i386.rpm
tar zxvf pxestuff-3.0.5-i386.tgz

then do:

cd /opt/ltsp/templates
./ltsp_initialize

and choose to apply the changes.

There is some additional setup to do for the PXE boot file that you just unpacked. That will be covered in the next section....


( Coming: Change to the directory where pxestuff unpacked, and follow the instructions in the README file. The last step in the README is to copy the file /etc/dhcpd.example to /etc/dhcpd.conf and edit it. We'll cover that in the next step. )

[rsbriggs]

Checklist item 3

In the previous step you unpacked the PXE file. In that directory there is a README file that tells you how to set up the PXE files. Enter that directory (called /root/pxestuff on my machine.) and do the following. Follow along in the readme if you wonder what each step does:

1.) cp bzImage* /tftpboot/lts
2.) cp init* /tftpboot/lts
3.) cp pxelinux.0 /tftpboot/lts
4.) mkdir /tftpboot/lts/pxelinux.cfg
5.) cp pxelinux.cfg/default /tftpboot/lts/pxelinux.cfg

6.) You now need to edit the file /etc/dhcpd.conf.sample (put there by the ltsp package) and add the contents of dhcp.sample in the pxestuff directory. Basically this is an entry for ws001 that has a long line of vendor option bytes in it. I changed it to be ws003, and inserted it just before the final closing } of the file.

7.) You can now copy the sample dhcpd.conf file over to be the real one:
cp dhcpd.conf.example dhcpd.conf

8.) The "real" version of /etc/dhcpd.conf will need some other changes. In particular, add a range of addresses to be served. Change the section that looks like this:


shared-network WORKSTATIONS {
subnet 192.168.0.0 netmask 255.255.255.0 {
}
}


to look like this:



shared-network WORKSTATIONS {
subnet 192.168.0.0 netmask 255.255.255.0 {
range 192.168.0.1 192.168.0.99;
}
}


This will server up addresses in the range of 192.168.0.1 to 192.168.0.99 automatically. We'll use 100 and above for "static" IPs assigned to workstations. Don't forget the ; on the end of the added line.

Also, change any instance of 192.168.0 to be whatever YOUR subnet needs (like 192.168.2)

9.) one last step before re-starting DHCP:

Download the file syslinux-2.05.tar.gz from ftp://ftp.kernel.org/pub/linux/utils/boot/syslinux
unzip and untar the thing. Find the file pxelinux.0 in the directory and copy it to the /tfptboot/lts directory.
cd syslinux-2.05/
cp pxelinux.0 /tftpboot/lts

Here are some additional notes from the README:
If you want to include additional kernel command line options, you
need to edit the /tftpboot/lts/pxelinux.cfg/default file, and include
the options on the 'append' line.

If you want to specify different configurations for each workstation, you need to create a config file in the /tftpboot/pxelinux.cfg directory. The name of the config file is made up from the IP address of the workstation encoded in uppercase HEX digits. That is, if the IP address of the workstation is 192.168.0.1, the filename should be 'C0A80001'

10.) Restart the dhcp daemon by executing:

/etc/rc.d/init.d/dhcpd restart

(we'll be starting and stopping lots of daemons - get used to typing /etc/rc.d/init.d/ or write a script that allows you to pass the name of the daemon to restart.)

Your Linux box should now be accepting DHCP requests, and assigning addresses.

[IronBits]

Aug 19 18:04:58 epox dhcpd: DHCPREQUEST for 192.168.2.201 from 00:04:75:1a:7a:90 via eth0
Aug 19 18:04:58 epox dhcpd: DHCPACK on 192.168.2.201 to 00:04:75:1a:7a:90 via eth0
Aug 20 01:04:59 epox in.tftpd[1955]: tftp: client does not accept options
Aug 19 18:05:08 epox dhcpd: DHCPDISCOVER from 00:04:75:1a:7a:90 via eth0
Aug 19 18:05:08 epox dhcpd: DHCPOFFER on 192.168.2.201 to 00:04:75:1a:7a:90 via eth0
Aug 19 18:05:10 epox dhcpd: DHCPREQUEST for 192.168.2.201 (192.168.2.20) from 00:04:75:1a:7a:90 via eth0
Aug 19 18:05:10 epox dhcpd: DHCPACK on 192.168.2.201 to 00:04:75:1a:7a:90 via eth0 tail /var/log/messages
Client shows booting up, loading up 16 RAM DISKs, going into duplex on the NIC, then nothing. If I hit return, it just scrolls up the screen.
ctrl-alt-del lets me reboot it. /me shrugs

[rsbriggs]

Two things.

1.) If I turn off DHCP on my main windows box, it hoses the entire network. Have to go with multiple sub-nets, which means that I have to take the server down AGAIN, and re-install the second NIC and re-configure my networking on that box.....

2.) In the /opt/ltsp/i386/etc/lts.conf file, change all the runlevels from 5 to 3 and try re-booting the client workstation.

[IronBits]

[Default]
SERVER = 192.168.2.254
XSERVER = auto
X_MOUSE_PROTOCOL = "PS/2"
X_MOUSE_DEVICE = "/dev/psaux"
X_MOUSE_RESOLUTION = 400
X_MOUSE_BUTTONS = 3
USE_XFS = N
LOCAL_APPS = N
RUNLEVEL = 3

#------------------------------------------------------------------------------
#
# Example of specifying X settings for a workstation
#
[ws001]
XSERVER = auto
LOCAL_APPS = N
USE_NFS_SWAP = N
SWAPFILE_SIZE = 48m
RUNLEVEL = 3

no luck, same place, just sites there after saying full duplex mode on the NIC.

[rsbriggs]

stuck for tonight, I'm afraid. Put the 2nd NIC card back into my server, and am running fine here. Should be able to try the VIQ tomorrow.... Must sleep


:sleepy:

[rsbriggs]

Success!!

Board comes up and net-boots just fine. It is folding as we speak, and I just opened the box that it came in about 15 minutes ago.....

Just one scare - you not only have to select LAN as one of the boot devices, you also have to make the BIOS selection to enable the LAN BOOT ROM.

[IronBits]

Anyone figured out how to run tftp/dhcpd without having to specify a MAC address for each client? :bang:

[PCZ]

It can be done but the workstations have to have names assigned by dhcpd.
You have to add all the names and IP addresses to DNS.
dhcpd will allocate names and IP's dynamically providing it can find them in DNS.

This command should be added to the global defaults:
get-lease-hostnames true;

The vender encapsulation options would also have to be moved to the global defaults section.

[rsbriggs]

The vender encapsulation options would also have to be moved to the global defaults section.
Probably should be anyway - all of these boards are using Intel PXE version 2.0 build 82, and not liking the options. (My log shows a log entry of "Client does not accept options" everytime each one of these boots, anyways...)

I've been able to net-boot thse boards without specifing a MAC, but could never get them a hostname assigned properly....

[IronBits]

My brothers, Tim and Rick, built me 8 Power Supply splitters.

I tested one for continuity, just to be sure it was ok...

Plugged two Biostar M7VIQ mobos into the one Power Supply and turned it on, viola!! :D

All wires are connected on the Y-adapters... 400 Watt Allied Power Supply.

Plugged one mobo into one end of the Y, then the other into the other end of the Y, then connected it to the Power Supply mobo connector and turned it on.

POST!

It's been running for several hours now, and none of the wires are hot!

Thanks Tim and Rick !!! :notworthy

Now to finish the whole project and get some pics :D

[rsbriggs]

Well, unless I can do the same somehow, I'm going to have to shut down at least a couple of comps. Just got a $300 electricity bill...... :eek: :eek:

I'm probably going to shut down my old gaming comp. Just the video card heat output alone can raise the temperature of my entire comp room by 6 degress F. Heat like that's got to be a little expensive. If I had a splitter, I'd probably buy another M7VIQ and retire it ASAP....

Got another awfully toasty 800 Mhz Slot-A that I'd replace with a VIQ, too, come to think of it...

[IronBits]

I tried to re-size it without losing too much quality...
Excellent quality work and nice looking to :thumbs:

[IronBits]

Sorry to hear that! :(

I'm sure mine is gonna be right close to that, but will wait one more month to see if I can keep it all turned on myself.
Last year, I paid over $500 for 4 months in a row before I took it offline :cry:
Speaking of toasty, I wonder if the heat shrink tubing will shrink further, in all the heat ;)

Oh hell ya! Upgrade them slow boats :thumbs:


Originally posted by rsbriggs
Well, unless I can do the same somehow, I'm going to have to shut down at least a couple of comps. Just got a $300 electricity bill...... :eek: :eek:

Got another awfully toasty 800 Mhz Slot-A that I'd replace with a VIQ, too, come to think of it...

[rsbriggs]

Would a Hydra have any use for it ?

[IronBits]

Magnav0x will take ANYTHING :D

[PCZ]

At last :D

I can use the built in nvidia nics with ltsp.

The latest ltsp kernel supports nvidia.
It includes the reverse engineered nvidia driver {forcedeth}

[PCZ]

The last piece of the puzzle just fell into place.
How to remotely reboot the nodes.

I have never got shutdown to work on the nodes as it runs a script which disconnects the NFS shares before poweroff and the script never finishes.

Well I discovered reboot -f
No more turning the power supplys off to reset them :)

This command shouldn't be run on a PC with HD's but as the nodes are diskless it is OK.