Mailing List Archive

Mark Dieterich wrote:

> Hi folks,
>
> I'm finally tired of dealing with floppy disks to control our remote installs
> and thought that PXELINUX sound like a great way to go. However, I've got
> multiple hurdles and I'd like some advice before I get myself dug in too
> deep... We currently have Solaris Jumpstart and Windows RIS running in the
> department and now we would like to intermix Linux PXELINUX booting.
> Any suggestions on how this should be designed?
>


What DHCP server do you use?


> I tried a really quick test, following the instructions to set up a tftp
> server and dhcp server, but the windows RIS server always answered the tftp
> request.


The TFTP request is a unicast, not a broadcast. The PXE stack will
download from whereever it is requested. This probably means the RIS
server is answering the DHCP request, or possibly the followon
pseudo-DHCP request to port 4011 which is part of the PXE protocol. The
latter is usually unicasted, but can be multicasted if the original DHCP
reply contained the appropriate magic.

We did have the next-server option listed in the DHCP server conf,
> it's almost as though it is ignoring this option. How are other people dealing
> with RIS and PXELINUX in the same environment? I looked through the past
> archives and couldn't find anything really appropriate. Any help/suggestions
> would be greatly appreciated.


A good question is perhaps: what do you expect to be keeping them apart?
Unlike the Solaris case there isn't a hardware difference to cue off
the servers. If you have a list of hosts, for example, that is RIS vs.
PXELINUX, make sure that *all* DHCP servers (including RIS, if
applicable) respect it and don't answer requests not directed to it. If
RIS is set up to answer all requests with a "PXEClient" tag, then RIS is
going to run on all PC machines.

-hpa

Thanks for the reply.

> What DHCP server do you use?

We are running a stock Redhat 7.2 DHCP server (dhcp-2.0pl5-8).

> The TFTP request is a unicast, not a broadcast. The PXE stack will
> download from whereever it is requested. This probably means the RIS
> server is answering the DHCP request, or possibly the followon
> pseudo-DHCP request to port 4011 which is part of the PXE protocol. The
> latter is usually unicasted, but can be multicasted if the original DHCP
> reply contained the appropriate magic.

The RIS server does not actually run DHCP. All of our windows clients
actually obtain their addresses from the linux DHCP server. I have to admit,
I don't know where the magic is that let's RIS work at all! As you suggested,
I suspect the RIS server is answering the followup request to port 4011.
I would have thought that the DHCP server option, next-server, would have
overridden this default for the linux boxes we are trying to use PXE with.

> A good question is perhaps: what do you expect to be keeping them apart?

I'm hoping we can keep them separate through the DHCP conf file. All of our
hosts are statically assigned in the DHCP tables and I was hoping to use
the group options to apply different parameters to the different groups of
host; perhaps this isn't possible?

> Unlike the Solaris case there isn't a hardware difference to cue off
> the servers. If you have a list of hosts, for example, that is RIS vs.
> PXELINUX, make sure that *all* DHCP servers (including RIS, if
> applicable) respect it and don't answer requests not directed to it. If
> RIS is set up to answer all requests with a "PXEClient" tag, then RIS is
> going to run on all PC machines.

Here are the pertinent pieces of the DHCP conf file (excluding all of the
static declarations):

#
# Our domain name
#
option domain-name "cs.brown.edu";

#
# Default time for a lease (in seconds). 36000 = 10 hours
#
default-lease-time 36000;

#
# Max time for lease (in seconds). If a client requests a longer
# lease than usual, let them have it if less than 1 day.
#
max-lease-time 86400;

#
# Fully Automated Install (FAI) - debian unattended install options
#
# option-170 is FAI_LOCATION, target is /usr/local/share/fai
# - this is the custom portion of the install
option option-170 "server:/fai";
option option-171 "install";
option option-172 "createvt sshd";

#
# For PXELINUX clients
#
allow booting;
allow bootp;

group {
next-server server;
filename "pxelinux.0";

host grendel {
hardware ethernet 00:00:00:00:00:00;
fixed-address client;
}
}

group {
... static assignments for the non-linux PXE clients ...
}


Thanks,

Mark

Mark Dieterich wrote:

>
> The RIS server does not actually run DHCP. All of our windows clients
> actually obtain their addresses from the linux DHCP server. I have to admit,
> I don't know where the magic is that let's RIS work at all! As you suggested,
> I suspect the RIS server is answering the followup request to port 4011.
> I would have thought that the DHCP server option, next-server, would have
> overridden this default for the linux boxes we are trying to use PXE with.
>


I wish it worked that way. It doesn't :-/

I would like to suggest doing the following: put a blank system on a
shared hub with another system running tcpdump to snoop packets, and see
what the packets going back and forth looks like. This should help
figuring out just how RIS is taking over your machines.


>>A good question is perhaps: what do you expect to be keeping them apart?
>
> I'm hoping we can keep them separate through the DHCP conf file. All of our
> hosts are statically assigned in the DHCP tables and I was hoping to use
> the group options to apply different parameters to the different groups of
> host; perhaps this isn't possible?


That should work, *assuming* we can figure out how to keep RIS from taking
over the universe. Worst case you might have to firewall off UDP ports 67
and 4011 on your RIS server from talking to hosts it shouldn't be talking to!

-hpa

There isn't really any such as a TFTP request. The box is going to TFTP
to whatever box you tell it (using the next-server line). Perhaps the
Windows box is answering the DHCP request and pointing it back to itself
for the TFTP server. Try looking at the PXE information when it gets
its IP, and see what it lists for its bootserver. Confirm that it is
the correct IP. Why don't you post a copy of your dhcpd.conf.

James

Mark Dieterich wrote:
> Hi folks,
>
> I'm finally tired of dealing with floppy disks to control our remote installs
> and thought that PXELINUX sound like a great way to go. However, I've got
> multiple hurdles and I'd like some advice before I get myself dug in too
> deep... We currently have Solaris Jumpstart and Windows RIS running in the
> department and now we would like to intermix Linux PXELINUX booting.
> Any suggestions on how this should be designed?
>
> I tried a really quick test, following the instructions to set up a tftp
> server and dhcp server, but the windows RIS server always answered the tftp
> request. We did have the next-server option listed in the DHCP server conf,
> it's almost as though it is ignoring this option. How are other people dealing
> with RIS and PXELINUX in the same environment? I looked through the past
> archives and couldn't find anything really appropriate. Any help/suggestions
> would be greatly appreciated.
>
> Thanks,
>
> Mark
> _______________________________________________
> SYSLINUX mailing list
> Submissions to SYSLINUX@zytor.com
> Unsubscribe or set options at:
> http://www.zytor.com/mailman/listinfo/syslinux
>

Well, I'm definitely getting closer... As hpa suggested, I started sniffing
packets as the machine tried to boot and here is what I saw:

**** NOTE: linuxserver is the dhcp server, w2kserver is the RIS server, and
workstation is the client I'm trying to get PXELINUX installs to work on ****

11:17:23.787348 0.0.0.0.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:4 flags:0x8000 file ""[|bootp]
11:17:23.790017 linuxserver.cs.brown.edu.bootps > 255.255.255.255.bootpc: xid:0xb4284a46 secs:4 flags:0x8000 Y:workstation.cs.brown.edu S:godzilla.cs.brown.edu G:router-33.cs.brown.edu ether 0:2:b3:28:4a:46 file ""[|bootp] (DF)
11:17:23.793547 w2kserver.cs.brown.edu.bootps > 255.255.255.255.bootpc: xid:0xb4284a46 flags:0x8000 S:w2kserver.cs.brown.edu G:router-33.cs.brown.edu ether 0:2:b3:28:4a:46 file ""[|bootp]
11:17:25.783960 0.0.0.0.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:4 flags:0x8000 file ""[|bootp]
11:17:25.786645 linuxserver.cs.brown.edu.bootps > 255.255.255.255.bootpc: xid:0xb4284a46 secs:4 flags:0x8000 Y:workstation.cs.brown.edu S:godzilla.cs.brown.edu G:router-33.cs.brown.edu ether 0:2:b3:28:4a:46 file ""[|bootp] (DF)
11:17:25.807018 arp who-has router-33.cs.brown.edu tell workstation.cs.brown.edu
11:17:25.807545 arp reply router-33.cs.brown.edu is-at 0:7:84:47:74:9
11:17:25.808081 workstation.cs.brown.edu.bootpc > w2kserver.cs.brown.edu.4011: (request) xid:0xb4284a46 secs:4 C:workstation.cs.brown.edu file ""[|bootp]
11:17:25.810521 w2kserver.cs.brown.edu.4011 > workstation.cs.brown.edu.bootpc: (reply) xid:0xb4284a46 C:workstation.cs.brown.edu S:w2kserver.cs.brown.edu sname "GIGAN.cs.b"[|bootp]
11:17:25.816358 workstation.cs.brown.edu.2070 > w2kserver.cs.brown.edu.tftp: 44 RRQ "OSChooser\i386\startrom.com"

... continues into RIS installation ...

Interesting that the w2kserver AND the linuxserver are both responding. I
know the w2kserver is NOT configured to hand out dhcp addresses for this
machine, so this must be a hidden RIS feature. My best guess at this point is
that the client is ignoring the next-server line in the dhcp.conf file and/or
the RIS server is handing out some piece of information needed by the client
to install, which is not being handed out by the dhcp server (running linux).

I got to poking around with dhcpd.conf file on the linuxserver and decided to
try the PXEclient install version. So I modified it to include this one
machine I was testing in a group:

group {
next-server linuxserver.cs.brown.edu;
filename "/tftpboot/pxelinux.0";
option dhcp-class-identifier "PXEClient";

host workstation {
hardware ethernet 00:02:b3:28:4a:46;
fixed-address workstation;
}
}

Afer making these changes to the dhcp config file, the workstation no longer
started a RIS install :) and RIS still worked on the rest of the clients!
So I went and got a copy of the PXE server from
http://www.kano.org.uk/projects/pxe and installed this. I managed to get
the PXE boot menu loading! However, the boot still fails because it can't
locate a boot server. Here is a dump of the network traffic:

15:24:35.912764 0.0.0.0.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:4 flags:0x8000 [|bootp]
15:24:37.886329 0.0.0.0.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:4 flags:0x8000 [|bootp]
15:24:37.909372 arp who-has router-33.cs.brown.edu tell workstation.cs.brown.edu
15:24:37.910920 arp reply router-33.cs.brown.edu is-at 0:7:84:47:74:9
15:24:37.911461 workstation.cs.brown.edu.bootpc > linuxserver.cs.brown.edu.4011: (request) xid:0xb4284a46 secs:4 C:workstation.cs.brown.edu [|bootp]
15:24:37.914232 linuxserver.cs.brown.edu.4011 > workstation.cs.brown.edu.bootpc: (reply) xid:0xb4284a46 secs:4 S:linuxserver.cs.brown.edu [|bootp] (DF)
15:24:42.807815 workstation.cs.brown.edu.4011 > SGI-DOG.MCAST.NET.4011: udp 548
15:24:44.751908 workstation.cs.brown.edu.4011 > SGI-DOG.MCAST.NET.4011: udp 548
15:24:48.706439 workstation.cs.brown.edu.4011 > SGI-DOG.MCAST.NET.4011: udp 548
15:24:54.638273 workstation.cs.brown.edu.4011 > SGI-DOG.MCAST.NET.4011: udp 548
15:25:02.548223 workstation.cs.brown.edu.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:14 flags:0x8000 C:workstation.cs.brown.edu [|bootp]
15:25:04.524666 workstation.cs.brown.edu.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:15 flags:0x8000 C:workstation.cs.brown.edu [|bootp]
15:25:08.479225 workstation.cs.brown.edu.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:17 flags:0x8000 C:workstation.cs.brown.edu [|bootp]
15:25:14.411059 workstation.cs.brown.edu.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:20 flags:0x8000 C:workstation.cs.brown.edu [|bootp]

I've installed the tftp-hpa server and tested with a standalone client to
make sure it was serving up files properly. I'm not exactly sure what all is
going on in this step. The workstation appears to be searching for a server
via multicast and then broadcast???

I can feel we are close here, just a few missing pieces...

Any suggestions?

Thanks,

Mark

Josef,

Thanks for the suggestions, I'll start investigating these...

> > 11:17:23.787348 0.0.0.0.bootpc > 255.255.255.255.bootps: xid:0xb4284a46 secs:4 flags:0x8000 file ""[|bootp]
> > 11:17:23.790017 linuxserver.cs.brown.edu.bootps > 255.255.255.255.bootpc: xid:0xb4284a46 secs:4 flags:0x8000 Y:workstation.cs.brown.edu S:godzilla.cs.brown.edu G:router-33.cs.brown.edu ether 0:2:b3:28:4a:46 file ""[|bootp] (DF)
>
> The client is told its IP address. Are you sure that there's nothing
> in 'file'?

How do I determine if there is something in this "file"?

> Does it load pxelinux?

It does prompt for the user to hit F8 and comes up with the boot local or
install linux options. I blieve this is coming from pxelinux, right? If so,
then it is loading pxelinux.

> Perhaps you should try to put some additional parameters into your
> dhcpd.conf, I think of the bootp vs. dhcp options for the
> tftp server. Perhaps you should try to put the IP address of your dhcp
> server there.

I'm apologize for my ignorance here, but additional options like? I've got
"allow booting" and "allow bootp" listed above this group. In fact, here is
the top of my dhcp.conf file:

option domain-name "cs.brown.edu";
default-lease-time 36000;
max-lease-time 86400;
option nis-domain "sun";

# This is all for fai installs
option option-170 "nas:/sys0/Linux/fai";
option option-171 "install";
option option-172 "createvt sshd";

allow booting;
allow bootp;

group {
stuff I listed before ...
}

> I still don't like the pxe server, since this splits
> IP address assignment (dhcp) from the bootfile/bootserver assignment
> (pxe server). So you need to tell your pxe server to hand out this
> information. Perhaps you should try to get along completely without
> it.

If I don't have the PXEClient argument in the group declaration, then the
machine always goes to the RIS server. If I have the PXEClient argument
listed, then it at least boots from the pxelinux instead.

> BTW, it seems you are tracing with quite few packet length, try to
> increase it ('tcpdump -s 500' should do it). The standard length
> is 64 bytes, which is quite small for dhcp.

Thanks, I didn't realize this and will make use of it in the future.

Mark

Mark Dieterich wrote:
>
> It does prompt for the user to hit F8 and comes up with the boot local or
> install linux options. I blieve this is coming from pxelinux, right? If so,
> then it is loading pxelinux.
>

No, this is probably coming from the PXE server.
>
>>BTW, it seems you are tracing with quite few packet length, try to
>>increase it ('tcpdump -s 500' should do it). The standard length
>>is 64 bytes, which is quite small for dhcp.
>

Use -s 2000, get the whole packet.

-hpa

> No, this is probably coming from the PXE server.

Your absolutely correct, the boot menu goes away. If I get rid of the
PXEClient statement from the dhcp server, the machine goes back to getting
it's info from the RIS server.

I'd prefer not to use a PXE server, but it looks like I either need a PXE
server running or an internal firewall around the windows and linux servers.

Mark

Mark Dieterich wrote:
>>No, this is probably coming from the PXE server.
>
>
> Your absolutely correct, the boot menu goes away. If I get rid of the
> PXEClient statement from the dhcp server, the machine goes back to getting
> it's info from the RIS server.
>
> I'd prefer not to use a PXE server, but it looks like I either need a PXE
> server running or an internal firewall around the windows and linux servers.
>

Either that or you need to configure the RIS server to not try to claim
access to the rest of the universe.

This might be hard, however, because of how the PXE specification is done
(if no PXE boot server is received, it will try to obtain one via
broadcast on the normal DHCP port. The RIS server will claim this one,
and oh boy...)

-hpa