Mailing List Archive

Am Dienstag, den 12.04.2005, 14:29 +0200 schrieb Toens Bueker:
> the pdf on http://www.planet-lab.org/PDN/PDN-02-006/ has a very nice
> diagramm on the last page, which seems to describe networking in Xen.
>
> 1.) Can somebody confirm that the above ist true? If yes, I would suggest
> adding that diagram to the existing Xen docs.

Maybe the diagram is correct for the xenoserver project, but it's not
for xen-{2|unstable}. Virtual interfaces (pp* in the diagram, vif* on my
host) and eth0 appear in dom0. I do not know what the "virtual firewall
router" in the middle shall be if not dom0 (which is below).


> 2.) Has somebody on the list a working configuration with domUs on a private
> network, which is/are NATted to the internet via a public IP in dom0?

I did this with vmware and UML which is quiet similar. Should be easy.


> When I
> tried to assemble an iptables configuration for this setup, iptables seemed
> to not recognize xens vif-interfaces.

Maybe you still let xend setup bridging eth0? It's possible to do this
with bridging (then use iptables with "--physdev-{in|out}" instead of "-
i" and "-o"), but you probably want to do it with _routing_. That means
you may bridge all the virtual VM-devices together, but _not_ eth0. Try
this:

* Shutdown all domUs, stop xend

* Set up an empty bridge-device with private ip using you distro
sysconfig or by hand:

brctl addbr mybr0
ip addr add 192.168.1.1/24 dev mybr0
ip link set mybr0 up


* Configure xend (/etc/xen/xend-config.sxp) for not setting up
xen-br0 on startup, but nevertheless adding virtual interfaces
to your bridge:

(network-script network-route)
(vif-script vif-bridge)
(vif-bridge mybr0)


* start xend, boot your VMs, tell them to use IPs in 192.168.1.0/24,
default gw being 192.168.1.1. Now all doms should be able to ping
each other within 192.168.1.0/24.


* in dom0:

sysctl -w net.ipv4.ip_forward=1 # (if not allready done by xend)
iptables -t nat -A POSTROUTING -j MASQUERADE \
-o eth0 -s 192.168.1.0/24


What have i forgotten?


> Which Xen version/flavour is
> necessary to get such a setup up and running?

All you find on the xen download page.

/nils.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > 1.) Can somebody confirm that the above ist true? If yes, I would suggest
> > adding that diagram to the existing Xen docs.
>
> Maybe the diagram is correct for the xenoserver project, but it's not
> for xen-{2|unstable}. Virtual interfaces (pp* in the diagram, vif* on my
> host) and eth0 appear in dom0. I do not know what the "virtual firewall
> router" in the middle shall be if not dom0 (which is below).

It's a Xen 1.x-era diagram. If you imagine the whole Virtual Firewall Router
box being inside dom0 *as well* (i.e. Xen doesn't really have anything to do
with the network) then that's pretty close to what Xen 2.0 and above looks
like.

It'd be nice to have more figures for the documentation - it'd help make
things clearer. Nobody has got around to doing it yet, but any contributions
would be very welcome!

Cheers,
Mark

> > 2.) Has somebody on the list a working configuration with domUs on a
> > private network, which is/are NATted to the internet via a public IP in
> > dom0?
>
> I did this with vmware and UML which is quiet similar. Should be easy.
>
> > When I
> > tried to assemble an iptables configuration for this setup, iptables
> > seemed to not recognize xens vif-interfaces.
>
> Maybe you still let xend setup bridging eth0? It's possible to do this
> with bridging (then use iptables with "--physdev-{in|out}" instead of "-
> i" and "-o"), but you probably want to do it with _routing_. That means
> you may bridge all the virtual VM-devices together, but _not_ eth0. Try
> this:
>
> * Shutdown all domUs, stop xend
>
> * Set up an empty bridge-device with private ip using you distro
> sysconfig or by hand:
>
> brctl addbr mybr0
> ip addr add 192.168.1.1/24 dev mybr0
> ip link set mybr0 up
>
>
> * Configure xend (/etc/xen/xend-config.sxp) for not setting up
> xen-br0 on startup, but nevertheless adding virtual interfaces
> to your bridge:
>
> (network-script network-route)
> (vif-script vif-bridge)
> (vif-bridge mybr0)
>
>
> * start xend, boot your VMs, tell them to use IPs in 192.168.1.0/24,
> default gw being 192.168.1.1. Now all doms should be able to ping
> each other within 192.168.1.0/24.
>
>
> * in dom0:
>
> sysctl -w net.ipv4.ip_forward=1 # (if not allready done by xend)
> iptables -t nat -A POSTROUTING -j MASQUERADE \
> -o eth0 -s 192.168.1.0/24
>
>
> What have i forgotten?
>
> > Which Xen version/flavour is
> > necessary to get such a setup up and running?
>
> All you find on the xen download page.
>
> /nils.
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Toens Bueker <toens.bueker@lists0903.nurfuerspam.neuroserve.de> writes:

> 2.) Has somebody on the list a working configuration with domUs on a private
> network, which is/are NATted to the internet via a public IP in dom0?

Shouldn't be that hard. I had this running with UML some time ago.

In domain0 you'll have to create a bridge device with whatever name
you want ("brctl addbr xen0" for example) and tell the xen domU's to
connect to that bridge. The bridge will be your private network. Do
*not* add eth0 to that bridge. You can do the usual stuff with it in
domain 0, i.e. configure some rfc1918 IP address, setup routes, let a
dhcp server run on it and hand out IP addresses, whatever you want ;)

iptables setup can be done this way for example:

iptables -t filter -F
iptables -t mangle -F
iptables -t nat -F
iptables -t filter -P FORWARD DROP
iptables -t mangle -A FORWARD -i xen0 -o eth0 -j MARK --set-mark 1
iptables -t filter -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -t filter -A FORWARD -m mark --mark 1 -j ACCEPT
iptables -t nat -A POSTROUTING -m mark --mark 1 -j MASQUERADE

This will accept and NAT traffic which comes from interface "xen0" and
is routed to interface "eth0". Don't forget to enable IP forwarding
in domain 0.

HTH,

Gerd

--
#define printk(args...) fprintf(stderr, ## args)

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > 2.) Has somebody on the list a working configuration with
> domUs on a
> > private network, which is/are NATted to the internet via a
> public IP in dom0?
>
> Shouldn't be that hard. I had this running with UML some time ago.

I think there's even an example NAT setup script in the repo.

Ian



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Ian Pratt wrote:
>
>
>>>2.) Has somebody on the list a working configuration with
>>
>>domUs on a
>>
>>>private network, which is/are NATted to the internet via a
>>
>>public IP in dom0?
>>
>>Shouldn't be that hard. I had this running with UML some time ago.
>
>
> I think there's even an example NAT setup script in the repo.

if someone wants to help me get the right modifications into the ipcop
kernel (2.4) I will make a domU distribution.

--- eric


--
http://www.wired.com/wired/archive/13.03/view.html?pg=5

The result of the duopoly that currently defines "competition" is that
prices and service suck. We're the world's leader in Internet
technology - except that we're not.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Nils Toedtmann <xen-users@nils.toedtmann.net> wrote:

>> 2.) Has somebody on the list a working configuration with domUs on a private
>> network, which is/are NATted to the internet via a public IP in dom0?

[...]

> Try this:
>
> * Shutdown all domUs, stop xend
>
> * Set up an empty bridge-device with private ip using you distro
> sysconfig or by hand:
>
> brctl addbr mybr0
> ip addr add 192.168.1.1/24 dev mybr0
> ip link set mybr0 up

Check.


> * Configure xend (/etc/xen/xend-config.sxp) for not setting up
> xen-br0 on startup, but nevertheless adding virtual interfaces
> to your bridge:
>
> (network-script network-route)
> (vif-script vif-bridge)
> (vif-bridge mybr0)

Check.

> * start xend, boot your VMs, tell them to use IPs in 192.168.1.0/24,
> default gw being 192.168.1.1. Now all doms should be able to ping
> each other within 192.168.1.0/24.

I can ping 192.168.1.1 from each domU. None of the domUs
can ping the other one.

?!

by
Töns
--
There is no safe distance.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Am Mittwoch, den 13.04.2005, 18:40 +0200 schrieb Toens Bueker:
> Nils Toedtmann <xen-users@nils.toedtmann.net> wrote:
>
> >> 2.) Has somebody on the list a working configuration with domUs on a private
> >> network, which is/are NATted to the internet via a public IP in dom0?
>
> [...]
>
> > Try this:
> >
> > * Shutdown all domUs, stop xend
> >
> > * Set up an empty bridge-device with private ip using you distro
> > sysconfig or by hand:
> >
> > brctl addbr mybr0
> > ip addr add 192.168.1.1/24 dev mybr0
> > ip link set mybr0 up
>
> Check.
>
>
> > * Configure xend (/etc/xen/xend-config.sxp) for not setting up
> > xen-br0 on startup, but nevertheless adding virtual interfaces
> > to your bridge:
> >
> > (network-script network-route)
> > (vif-script vif-bridge)
> > (vif-bridge mybr0)
>
> Check.
>
> > * start xend, boot your VMs, tell them to use IPs in 192.168.1.0/24,
> > default gw being 192.168.1.1. Now all doms should be able to ping
> > each other within 192.168.1.0/24.
>
> I can ping 192.168.1.1 from each domU. None of the domUs
> can ping the other one.

Hmmm, the bridge does not bridge ... check this:

(1) "/usr/sbin/brctl show" should look like this:

bridge name bridge id STP enabled interfaces
mybr0 8000.000c7616d891 no vif1.0
vif2.0
vif3.0

and so on, this means that the virtual interfaces vif* belong to
the bridge "mybr0"

(2) "/sbin/iptables -nL ; /sbin/iptables -t nat -nL" should be empty
(just for testing) with policies "ACCEPT"

(3) The domUs use different MACs

If all is true and the domUs still cannot ping each other, ping all
domUs from dom0 and send me the resulting arptable:

for i in 2 3 4 ; do ping -c 1 192.168.1.$i ; done ; /sbin/arp -n

and the output of "/sbin/ip addr show up"

/nils.



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Nils Toedtmann <xen-users@nils.toedtmann.net> wrote:

>> I can ping 192.168.1.1 from each domU. None of the domUs
>> can ping the other one.
>
> Hmmm, the bridge does not bridge ... check this:

[...]

> (3) The domUs use different MACs

They didn't (I detected the error earlier today - now they do). I thought
Xen was taking care of that (but obviously one should not specify MACs in
the configuration, if one wants Xen to handle those) :-)

Thx.

by
Töns
--
There is no safe distance.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users