Mailing List Archive

OVS+iptables
Hello, netstack gurus!

I have problem with iptables filtering on XCP. We use two physical boxes:
XCP(with domU compute) and Ubuntu 12.04(controller). They connected by
patch-cord, so we could use internal vlans. We use quantum. nova-compute
creates expected iptables rules on dom0, but they have no effect . This
because traffic between VMs goes inside OVS and doesn't touch IP stack of
host system. Security groups not work at all:( Using OVS OpenFlow
impementation I think it is the best solution.


In this blueprint (http://wiki.openstack.org/xenapi-security-groups)
openflow security groups don't implemented.

R2B. XS/XCP uses Open vSwitch networking stack, security groups are
> configured through flow tables in Open vSwitch


and Security groups still configured thru iptables.

R2A. XS/XCP uses Open vSwitch networking stack, security groups still
> configured through iptables


Is it temporary and non-working solution or may be it works, but required
additional configuring?

Many thanks..

--
Regards, Roman Sokolkov
Re: OVS+iptables [ In reply to ]
Hi Roman!

It seems that iptables and open vswitch do not play nicely together, in particular when the rules need to be applied directly on the VIF ports.
As discussed in the "limitations" section of the plugin's documentation: http://openvswitch.org/openstack/documentation/

For the blueprint you reference [1], it was implemented using iptables only. I'd regard it as a mere porting of the existing iptables driver for KVM to XenServer.
The iptables drivers was never tested with any Quantum plugin. As far as I can remember, It was tested only in flat and flatdhcp modes, with OVS too (If I'm not wrong).
However, it is quite clear that this works only under specific circumstances, and certainly does not apply to the Quantum use case. I also suspect that the fact that the OVS plugin internally uses VLAN is not the root cause. [2] might somehow clarify why IPtables rules are not evaluated when OVS is enabled.

I reckon the way forward will be to implement security groups rules as OVS flow table entries. It is my opinion that OpenFlow might be not necessary.
One problem that needs to be solved is the explosion of the number of rules in the OVS flow table. Security groups often apply to port ranges, which, AFAIK, cannot be specified at the moment in OVS flow table entries.Given the limited size of the kernel-level flow table cache, a huge number of rules could potentially cause an high number of misses in this cache, and therefore a high number of kernel/user mode context switches.

This has been improved in recent versions of open vswitch, where bitmasks can be specified for most fields, including tp_src and tp_dst [3]. Assuming that the rule is then taken 'as it is' by the OVS module , ie: the bitmask is not transformed in the equivalent set of rules (I have never tested them), this will definitely allow to consistently reduce the number of required flow entries.


I hope this helps.
Salvatore

[1] http://wiki.openstack.org/xenapi-security-groups
[2] http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html#section3
[3] http://openvswitch.org/cgi-bin/ovsman.cgi?page=utilities%2Fovs-ofctl.8

________________________________________
From: netstack-bounces+salvatore.orlando=eu.citrix.com@lists.launchpad.net [netstack-bounces+salvatore.orlando=eu.citrix.com@lists.launchpad.net] On Behalf Of Roman Sokolkov [rsokolkov@gmail.com]
Sent: Monday, May 14, 2012 4:21 PM
To: netstack@lists.launchpad.net
Subject: [Netstack] OVS+iptables

Hello, netstack gurus!

I have problem with iptables filtering on XCP. We use two physical boxes: XCP(with domU compute) and Ubuntu 12.04(controller). They connected by patch-cord, so we could use internal vlans. We use quantum. nova-compute creates expected iptables rules on dom0, but they have no effect . This because traffic between VMs goes inside OVS and doesn't touch IP stack of host system. Security groups not work at all:( Using OVS OpenFlow impementation I think it is the best solution.


In this blueprint (http://wiki.openstack.org/xenapi-security-groups) openflow security groups don't implemented.

R2B. XS/XCP uses Open vSwitch networking stack, security groups are configured through flow tables in Open vSwitch

and Security groups still configured thru iptables.

R2A. XS/XCP uses Open vSwitch networking stack, security groups still configured through iptables

Is it temporary and non-working solution or may be it works, but required additional configuring?

Many thanks..

--
Regards, Roman Sokolkov
--
Mailing list: https://launchpad.net/~netstack
Post to : netstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~netstack
More help : https://help.launchpad.net/ListHelp
Re: OVS+iptables [ In reply to ]
Re-sending as it seems that for some reason the message was not delivered to the list.

Salvatore

> -----Original Message-----
> From: Salvatore Orlando
> Sent: 16 May 2012 16:46
> To: Roman Sokolkov; netstack@lists.launchpad.net
> Subject: RE: [Netstack] OVS+iptables
>
> Hi Roman!
>
> It seems that iptables and open vswitch do not play nicely together, in
> particular when the rules need to be applied directly on the VIF ports.
> As discussed in the "limitations" section of the plugin's documentation:
> http://openvswitch.org/openstack/documentation/
>
> For the blueprint you reference [1], it was implemented using iptables only.
> I'd regard it as a mere porting of the existing iptables driver for KVM to
> XenServer.
> The iptables drivers was never tested with any Quantum plugin. As far as I
> can remember, It was tested only in flat and flatdhcp modes, with OVS too (If
> I'm not wrong).
> However, it is quite clear that this works only under specific circumstances,
> and certainly does not apply to the Quantum use case. I also suspect that the
> fact that the OVS plugin internally uses VLAN is not the root cause. [2] might
> somehow clarify why IPtables rules are not evaluated when OVS is enabled.
>
> I reckon the way forward will be to implement security groups rules as OVS
> flow table entries. It is my opinion that OpenFlow might be not necessary.
> One problem that needs to be solved is the explosion of the number of rules
> in the OVS flow table. Security groups often apply to port ranges, which,
> AFAIK, cannot be specified at the moment in OVS flow table entries.Given
> the limited size of the kernel-level flow table cache, a huge number of rules
> could potentially cause an high number of misses in this cache, and therefore
> a high number of kernel/user mode context switches.
>
> This has been improved in recent versions of open vswitch, where bitmasks
> can be specified for most fields, including tp_src and tp_dst [3]. Assuming
> that the rule is then taken 'as it is' by the OVS module , ie: the bitmask is not
> transformed in the equivalent set of rules (I have never tested them), this
> will definitely allow to consistently reduce the number of required flow
> entries.
>
>
> I hope this helps.
> Salvatore
>
> [1] http://wiki.openstack.org/xenapi-security-groups
> [2] http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html#section3
> [3] http://openvswitch.org/cgi-bin/ovsman.cgi?page=utilities%2Fovs-ofctl.8
>
> ________________________________________
> From: netstack-
> bounces+salvatore.orlando=eu.citrix.com@lists.launchpad.net [netstack-
> bounces+salvatore.orlando=eu.citrix.com@lists.launchpad.net] On Behalf Of
> Roman Sokolkov [rsokolkov@gmail.com]
> Sent: Monday, May 14, 2012 4:21 PM
> To: netstack@lists.launchpad.net
> Subject: [Netstack] OVS+iptables
>
> Hello, netstack gurus!
>
> I have problem with iptables filtering on XCP. We use two physical boxes:
> XCP(with domU compute) and Ubuntu 12.04(controller). They connected by
> patch-cord, so we could use internal vlans. We use quantum. nova-compute
> creates expected iptables rules on dom0, but they have no effect . This
> because traffic between VMs goes inside OVS and doesn't touch IP stack of
> host system. Security groups not work at all:( Using OVS OpenFlow
> impementation I think it is the best solution.
>
>
> In this blueprint (http://wiki.openstack.org/xenapi-security-groups)
> openflow security groups don't implemented.
>
> R2B. XS/XCP uses Open vSwitch networking stack, security groups are
> configured through flow tables in Open vSwitch
>
> and Security groups still configured thru iptables.
>
> R2A. XS/XCP uses Open vSwitch networking stack, security groups still
> configured through iptables
>
> Is it temporary and non-working solution or may be it works, but required
> additional configuring?
>
> Many thanks..
>
> --
> Regards, Roman Sokolkov

--
Mailing list: https://launchpad.net/~netstack
Post to : netstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~netstack
More help : https://help.launchpad.net/ListHelp
Re: OVS+iptables [ In reply to ]
2012/5/16 Salvatore Orlando <Salvatore.Orlando@eu.citrix.com>

> Hi Roman!
>
> It seems that iptables and open vswitch do not play nicely together, in
> particular when the rules need to be applied directly on the VIF ports.
> As discussed in the "limitations" section of the plugin's documentation:
> http://openvswitch.org/openstack/documentation/
>
> For the blueprint you reference [1], it was implemented using iptables
> only. I'd regard it as a mere porting of the existing iptables driver for
> KVM to XenServer.
> The iptables drivers was never tested with any Quantum plugin. As far as I
> can remember, It was tested only in flat and flatdhcp modes, with OVS too
> (If I'm not wrong).
> However, it is quite clear that this works only under specific
> circumstances, and certainly does not apply to the Quantum use case. I also
> suspect that the fact that the OVS plugin internally uses VLAN is not the
> root cause. [2] might somehow clarify why IPtables rules are not evaluated
> when OVS is enabled.
>
> I reckon the way forward will be to implement security groups rules as OVS
> flow table entries. It is my opinion that OpenFlow might be not necessary.
> One problem that needs to be solved is the explosion of the number of
> rules in the OVS flow table. Security groups often apply to port ranges,
> which, AFAIK, cannot be specified at the moment in OVS flow table
> entries.Given the limited size of the kernel-level flow table cache, a huge
> number of rules could potentially cause an high number of misses in this
> cache, and therefore a high number of kernel/user mode context switches.
>
> This has been improved in recent versions of open vswitch, where bitmasks
> can be specified for most fields, including tp_src and tp_dst [3]. Assuming
> that the rule is then taken 'as it is' by the OVS module , ie: the bitmask
> is not transformed in the equivalent set of rules (I have never tested
> them), this will definitely allow to consistently reduce the number of
> required flow entries.
>
>
> I hope this helps.
> Salvatore
>
> [1] http://wiki.openstack.org/xenapi-security-groups
> [2] http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html#section3
> [3] http://openvswitch.org/cgi-bin/ovsman.cgi?page=utilities%2Fovs-ofctl.8
>
> ________________________________________
> From: netstack-bounces+salvatore.orlando=eu.citrix.com@lists.launchpad.net[netstack-bounces+salvatore.orlando=
> eu.citrix.com@lists.launchpad.net] On Behalf Of Roman Sokolkov [
> rsokolkov@gmail.com]
> Sent: Monday, May 14, 2012 4:21 PM
> To: netstack@lists.launchpad.net
> Subject: [Netstack] OVS+iptables
>
> Hello, netstack gurus!
>
> I have problem with iptables filtering on XCP. We use two physical boxes:
> XCP(with domU compute) and Ubuntu 12.04(controller). They connected by
> patch-cord, so we could use internal vlans. We use quantum. nova-compute
> creates expected iptables rules on dom0, but they have no effect . This
> because traffic between VMs goes inside OVS and doesn't touch IP stack of
> host system. Security groups not work at all:( Using OVS OpenFlow
> impementation I think it is the best solution.
>
>
> In this blueprint (http://wiki.openstack.org/xenapi-security-groups)
> openflow security groups don't implemented.
>
> R2B. XS/XCP uses Open vSwitch networking stack, security groups are
> configured through flow tables in Open vSwitch
>
> and Security groups still configured thru iptables.
>
> R2A. XS/XCP uses Open vSwitch networking stack, security groups still
> configured through iptables
>
> Is it temporary and non-working solution or may be it works, but required
> additional configuring?
>
> Many thanks..
>
> --
> Regards, Roman Sokolkov
>



--
Regards, Roman Sokolkov
Re: OVS+iptables [ In reply to ]
Salvatore is correct. Thankfully, there may be some even easier options
that allow us to still use iptables with OVS. I'm planning on exploring
them with the help of some OVS team members as part of my Folsom-2 work on
security groups for Quantum.

Dan

On Wed, May 16, 2012 at 8:45 AM, Salvatore Orlando <
Salvatore.Orlando@eu.citrix.com> wrote:

> Hi Roman!
>
> It seems that iptables and open vswitch do not play nicely together, in
> particular when the rules need to be applied directly on the VIF ports.
> As discussed in the "limitations" section of the plugin's documentation:
> http://openvswitch.org/openstack/documentation/
>
> For the blueprint you reference [1], it was implemented using iptables
> only. I'd regard it as a mere porting of the existing iptables driver for
> KVM to XenServer.
> The iptables drivers was never tested with any Quantum plugin. As far as I
> can remember, It was tested only in flat and flatdhcp modes, with OVS too
> (If I'm not wrong).
> However, it is quite clear that this works only under specific
> circumstances, and certainly does not apply to the Quantum use case. I also
> suspect that the fact that the OVS plugin internally uses VLAN is not the
> root cause. [2] might somehow clarify why IPtables rules are not evaluated
> when OVS is enabled.
>
> I reckon the way forward will be to implement security groups rules as OVS
> flow table entries. It is my opinion that OpenFlow might be not necessary.
> One problem that needs to be solved is the explosion of the number of
> rules in the OVS flow table. Security groups often apply to port ranges,
> which, AFAIK, cannot be specified at the moment in OVS flow table
> entries.Given the limited size of the kernel-level flow table cache, a huge
> number of rules could potentially cause an high number of misses in this
> cache, and therefore a high number of kernel/user mode context switches.
>
> This has been improved in recent versions of open vswitch, where bitmasks
> can be specified for most fields, including tp_src and tp_dst [3]. Assuming
> that the rule is then taken 'as it is' by the OVS module , ie: the bitmask
> is not transformed in the equivalent set of rules (I have never tested
> them), this will definitely allow to consistently reduce the number of
> required flow entries.
>
>
> I hope this helps.
> Salvatore
>
> [1] http://wiki.openstack.org/xenapi-security-groups
> [2] http://ebtables.sourceforge.net/br_fw_ia/br_fw_ia.html#section3
> [3] http://openvswitch.org/cgi-bin/ovsman.cgi?page=utilities%2Fovs-ofctl.8
>
> ________________________________________
> From: netstack-bounces+salvatore.orlando=eu.citrix.com@lists.launchpad.net[netstack-bounces+salvatore.orlando=
> eu.citrix.com@lists.launchpad.net] On Behalf Of Roman Sokolkov [
> rsokolkov@gmail.com]
> Sent: Monday, May 14, 2012 4:21 PM
> To: netstack@lists.launchpad.net
> Subject: [Netstack] OVS+iptables
>
> Hello, netstack gurus!
>
> I have problem with iptables filtering on XCP. We use two physical boxes:
> XCP(with domU compute) and Ubuntu 12.04(controller). They connected by
> patch-cord, so we could use internal vlans. We use quantum. nova-compute
> creates expected iptables rules on dom0, but they have no effect . This
> because traffic between VMs goes inside OVS and doesn't touch IP stack of
> host system. Security groups not work at all:( Using OVS OpenFlow
> impementation I think it is the best solution.
>
>
> In this blueprint (http://wiki.openstack.org/xenapi-security-groups)
> openflow security groups don't implemented.
>
> R2B. XS/XCP uses Open vSwitch networking stack, security groups are
> configured through flow tables in Open vSwitch
>
> and Security groups still configured thru iptables.
>
> R2A. XS/XCP uses Open vSwitch networking stack, security groups still
> configured through iptables
>
> Is it temporary and non-working solution or may be it works, but required
> additional configuring?
>
> Many thanks..
>
> --
> Regards, Roman Sokolkov
> --
> Mailing list: https://launchpad.net/~netstack
> Post to : netstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~netstack
> More help : https://help.launchpad.net/ListHelp
>



--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~~~~~~~~~~~~~~~~~~~~~~~~~