Mailing List Archive

Long email alert #2 :)

On Fri, Apr 6, 2012 at 10:29 PM, Sumit Naiksatam (snaiksat) <
snaiksat@cisco.com> wrote:

> Hi Troy, All,
>
> Thanks for this pointer. Yes I agree, there are constructs in here like
> the subnet which can be leveraged by a L3 (forwarding) component
> implementation.
>
> So just to build on this, and assuming that the IPAM component will be
> integrated into Quantum (btw, is that already decided?),


That's been the proposal, and I haven't heard any concerns voiced so far,
but if there are, we can talk through them. Going beyond Nova IPAM is very
important in my mind, and since Quantum is going to be core in Folsom,
we'll need a non-Nova IPAM implementation that is also "core".


> the L3 forwarding component's realization can make calls to the IPAM
> component to store/retrieve information pertaining to the subnets, and
> possibly routes etc. Effectively, this IPAM interface can be used
> internally by the Quantum service, and/or other privileged entities (say,
> the nova service).
>

This is similar to how I've been thinking about it as well, that the
L3-forwarding element feeds off IPAM info about Quantum networks that are
associated with the L3-forwarding element. That said, Troy made me realize
that from an API perspective, the lines between IPAM and defining the
behavior of the L3-forwarding element are somewhat blurry (even if in terms
of back-end implementation they are two separate things).



>
> However, we still need to have the L3 abstractions defined on the
> tenant/user side. For instance, take the example of the gateway IP address.
> Per the Melange API, a static route can be created, and requires a gateway
> IP address. This piece of information is more likely defined by the service
> provider.


Maybe I'm not following the logic correctly, but I think that is true in
some cases, and not others. If this is a private network behind a NAT
where a customer define the IP addressing, the service provider probably
doesn't care about the gateway IP (for example, it can just default to the
.1 of the subnet the customer choose to specify for that network).

However, if the gateway is part of a public network managed by the service
provider, the provider definitely does care about the next hop, as it is
allocated out of shared provider space. Conversely, as you point out, the
customer doesn't really care about this IP address, all he/she cares about
it getting to the "public" network.

One thing I really liked from the cisco proposal is the concept that you
can have abstracted services that a tenant can choose to connect to,
without having to deal with the details. This matches with my sense that in
logical space, as big reason you need L3 forwarding is to isolate between
"private" tenant space and "public" service provider space.

I can imagine a model with the following high-level flow:

1) service providers define "upstream services" using admin APIs (these
APIs could be plugin specific, or generic). For example, one upstream
service might provide NAT access to the public network. Another might
provide VPN connectivity to the customer's remote data center.

2) The tenant can list available upstream services offered by the provider.
This could include services that are available to all tenants, or services
that are specific to that tenant.

quantum list-l3-services
UUID name description
<UUID1> public-net "access to public network using NAT"
<UUID2> pubic-net-floating "access to public network using NAT,
automatic floating IPs"
<UUID3> tenantA-VPN1 "vpn access to tenantA data center"

3) Customer can create and list quantum networks with associated IP prefixes

quantum create-net net1 10.0.0.0/24
quantum create-net net2 10.0.1.0/24
quantum list-nets
UUID name IPv4 Prefix
<UUID4> net1 10.0.0.0/24
<UUID5> net2 10.0.1.0/24

VIFs provided on that network will be allocated an IP from the subnet,
given a local route to the subnet, and assigned the default gateway
associated with that subnet (for VMs with multiple VIFs, additional config
is needed to determine which network's gateway acts as the VM

4) define a logical l3-forwarding entity, which I will call a "router" for
now, though naming is flexible. This router can then be linked with both
"upstream" services and "downstream" networks. For example, the toy syntax
below would connect net1 and net2 via basic NAT to the public network .

quantum create-router r1
<UUID6>
quantum router-associate-service <UUID6> <UUID1>
quantum router-associate-network <UUID6> <UUID4>
quantum router-associate-network <UUID6> <UUID5>

quantum show-router
name: r1
interfaces:
UUID Attachment IP(s)
<UUID7> service <UUID1> 9.0.0.30
<UUID8> net <UUID4> 10.0.0.1
<UUID9> net <UUID5> 10.0.1.1
routes:
Destination Interface Next-hop
0.0.0.0/0 <UUID7> 9.0.0.1
9.0.0.0/24 <UUID7> direct
10.0.0.0/24 <UUID4> direct
10.0.1.0/24 <UUID5> direct

When attaching networks, the router essentially gets an "interface" on that
network, and by default that "interface" is assigned the gateway IP
address(es) associated with that Quantum network's subnet info.

In this example, the router would automatically add routes associated with
the service or network. For example, if you attached the router to a
network that uses 10.0.0.0/24, the router would automatically get a route
to that network out the 10.0.0.1 interface that is plugged into the L2
Quantum network. If a service would also define what subnets should be
routed to the corresponding router interface. For example, the public-net
would specify that it should be the default route.

There's a lot of scope for additional functionality (API extensions?)
around tenants controlling routing rules/ACLs, but I think a large number
of key use cases can be met with a model like the one described above.

Additionally, a service provider might pre-provision some parts of the
router on behalf of a new tenant to make the common case even simpler. For
example, automatically creating the router, the uplink to the public
network via NAT, and a single private network. That would map to logic
that is currently hard-coded in the VLANManager in nova.

Here's what I like about this type of model:

- pretty simple for basic things, yet is easy to extend using API
extensions, since you can associate additional attributes with either the
router as a whole (e.g., some complex routing table abstraction) or
individual interface (e.g., ACLs filtering traffic in and out of a subnet).


- L2, L3, gateway semantics are well defined and simple to grok: if I'm on
the same Quantum network, connectivity is L2. If I'm on two different
quantum networks connected via a router, that's L3 that I reach via a
gateway IP on my local L2 network. This is as opposed to the Amazon VPC
model, which gives you "subnets" that aren't really subnets from an L2 or a
next hop perspective.

- API model "degrades" well. Model still makes sense even if the router
connects private networks but does not connect to upstream, or if no router
is used at all (just a plain L2 network with no gateway). This second
model is also important if a gateway does exist, but is offered by a VM
appliance outside of Quantum. Different service providers will want to use
different models, so I see this as valuable.

- Allows for abstracted services, when the tenants doesn't care about the
details (this is basically stolen from Sumit's proposal).

Anyway, I didn't mean to end up writing a mini-proposal, but I guess that
happens some times :)

One other comment: Troy's email made me think that from an API perspective
we shouldn't automatically divide the world between IPAM and L3-forwarding.
We should just think about what the right abstractions are for tenants,
then as a second step figure out how those map to different pluggable
back-ends and chunks of existing code.

When thinking about logical models, I often go back to thinking about how I
would draw up a network topology on a whiteboard (where no physical world
constraints exist) and use that as my motivation. Hopefully we'll have
good access to whiteboards at the summit. I'll email ttx :)

Dan





>
> Thanks,
> ~Sumit.
>
> > -----Original Message-----
> > From: Troy Toman [mailto:troy.toman@rackspace.com]
> > Sent: Tuesday, April 03, 2012 3:28 PM
> > To: Sumit Naiksatam (snaiksat)
> > Cc: Jason Kölker; <netstack@lists.launchpad.net>
> > Subject: Re: [Netstack] L3 Forwarding
> >
> > Sumit,
> >
> > You can see the latest Melange documentation here:
> >
> > http://melange.readthedocs.org/en/latest/apidoc.html
> >
> > There is a section on static routes and a call for creating a tenant
> > subnet that I think are relevant. It does not fully implement a routing
> > mechanism. But, may be useful for implementing an L3 service.
> >
> > Troy
> >
> > On Apr 3, 2012, at 4:41 PM, Sumit Naiksatam (snaiksat) wrote:
> >
> > > Hi Troy,
> > >
> > > Thanks for your response. Would you be able to point us to more
> > information on the L3 constructs/abstractions you mention? That way we
> > can unify the ideas and arrive at a richer model. We have looked at the
> > Melange wiki and the pointers therein to the base API which is for
> > storing and retrieving network information. However, this API does not
> > seem to represent a L3 routing/forwarding model which can drive the
> > underlying implementation.
> > >
> > > Thanks,
> > > ~Sumit.
> > >
> > >> -----Original Message-----
> > >> From: Troy Toman [mailto:troy.toman@rackspace.com]
> > >> Sent: Tuesday, April 03, 2012 5:56 AM
> > >> To: Sumit Naiksatam (snaiksat)
> > >> Cc: Jason Kölker; <netstack@lists.launchpad.net>
> > >> Subject: Re: [Netstack] L3 Forwarding
> > >>
> > >>
> > >> On Apr 3, 2012, at 12:24 AM, Sumit Naiksatam (snaiksat) wrote:
> > >>
> > >>> Hi Jason,
> > >>>
> > >>> Per the information published, Melange is an IPAM service
> > >> (information repository for the network). Isn't that fundamentally
> > >> different in function from the notion of L3
> > >> abstractions/constructs/model required to drive routing/forwarding?
> > >> Reading the specification document referenced in the wiki will
> > clarify
> > >> this. Melange could be an IPAM service for the proposed L3 model
> > (these
> > >> are complementary).
> > >>
> > >> Melange was built as a network information repository that initially
> > >> focused on IPAM. One thing it does is manage IPs. But, it does have
> > >> some fundamental L3 constructs/abstractions as well. Melange
> > >> understands subnets, outside global/inside local relationships and
> > >> routes, for instance. So, it provides a foundation for many L3
> > >> services. I look forward to talking about how we can evolve this
> > >> thinking at the summit.
> > >>
> > >> Troy
> > >>
> > >>>
> > >>> Thanks,
> > >>> ~Sumit.
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: netstack-bounces+snaiksat=cisco.com@lists.launchpad.net
> > >>>> [mailto:netstack-bounces+snaiksat=cisco.com@lists.launchpad.net]
> > On
> > >>>> Behalf Of Jason Kölker
> > >>>> Sent: Monday, April 02, 2012 9:16 PM
> > >>>> To: netstack@lists.launchpad.net
> > >>>> Subject: Re: [Netstack] L3 Forwarding
> > >>>>
> > >>>> On Mon, 2012-04-02 at 20:59 -0700, Sumit Naiksatam (snaiksat)
> > wrote:
> > >>>>> Following is a summary of the additions:
> > >>>>> As you might have gleaned, the proposal until this point catered
> > >> more
> > >>>> to
> > >>>>> the tenant/user aspects of the L3 discussion, however it did not
> > >>>> address
> > >>>>> the service-provider/operator aspects. Some of the feedback we
> > have
> > >>>>> received until now also points to the missing SP constructs.
> > >>>>>
> > >>>>> The current thinking is that this missing SP aspect should be
> > >> handled
> > >>>> in
> > >>>>> a generic way so as to be able to handle different types of L3
> > >>>> entities
> > >>>>> like gateways, firewalls, etc. More specifically, in the context
> > of
> > >>>> the
> > >>>>> current proposal, it amounts to introducing a SP API to manage
> > >>>>> "targets". The key attribute when creating a "target" is the IP
> > >>>> address
> > >>>>> being assigned to this "target". Also note that the definition of
> > >> the
> > >>>>> "target" is in the context of a tenant (and also one or more
> > >>>> routetables
> > >>>>> for that tenant). Given this definition, the "target" has the
> > >>>> following
> > >>>>> attributes:
> > >>>>> ID: UUID
> > >>>>> Name: Public, Private, VPN, etc.
> > >>>>> IP Address
> > >>>>> Tenant_ID: The tenant for which this "target" is applicable
> > >>>>> Routetable IDs: (Optional parameter) One or more routetable IDs
> > >>>>> belonging to this tenant for which this target is valid. This
> > >>>> provides
> > >>>>> for more granular control of the "target " applicability (from a
> > >>>> service
> > >>>>> provider perspective, e.g. the "Private" target will map to
> > >> different
> > >>>>> end points with respect to different routetables for the same
> > >>>> tenant).
> > >>>>>
> > >>>>> To illustrate this better, let's take the example of the SP
> > having
> > >> to
> > >>>>> configure an internet gateway for a particular tenant. Let's say
> > >>>> that
> > >>>>> the tenant creates a subnet 10.0.0.0/24, and the SP uses the
> > >>>> convention
> > >>>>> of assigning the first IP in the subnet as a gateway (i.e.
> > >> 10.0.0.1).
> > >>>>>> From a logical model perspective, the SP would then create a
> > >>>> "target"
> > >>>>> with the following attributes:
> > >>>>> ID : UUID-XYZ-1
> > >>>>> Name: Public
> > >>>>> IP Address: 10.0.0.1
> > >>>>> Tenant_ID: UUID-Tenant-ABC
> > >>>>> Routetable IDs: [UUID-Routetable-UVW-1]
> > >>>>>
> > >>>>> The creation of this logical entity would result in the
> > underlying
> > >> SP
> > >>>>> implementation mapping it to the requirement for the creation of
> > a
> > >>>>> gateway entity such that on the tenant network side it would have
> > >> the
> > >>>>> 10.0.0.1 IP address, and it would have another interface to the
> > >>>> public
> > >>>>> network (with some PAT/NAT functionality).
> > >>>>>
> > >>>>> Subsequently, the tenant creates a route in the routetable
> > >>>>> UUID-Routetable-UVW-1,
> > >>>>> Source: 10.0.0.0/24, Destination: default, Target: Public
> > >>>>>
> > >>>>> When a VM is now instantiated with an interface on this subnet,
> > the
> > >>>>> default route for the packets going out of this interface will
> > lead
> > >>>> them
> > >>>>> to the gateway 10.0.0.1. Exactly how the L2/L3 networking is
> > setup
> > >> to
> > >>>>> handle this connectivity from the VM to the gateway is specific
> > to
> > >>>> that
> > >>>>> particular deployment.
> > >>>>>
> > >>>>> In the above case, the gateway could be a firewall, and which in
> > >> turn
> > >>>>> might be managed as a separate service. In that case, the SP
> > would
> > >>>>> correlate the above created logical "target" resource with a
> > >> resource
> > >>>>> created in that firewall service.
> > >>>>>
> > >>>>> We are actively trying to make this a better/simpler proposal,
> > and
> > >>>> any
> > >>>>> feedback/participation is very much welcome. Thanks in advance
> > and
> > >>>> stay
> > >>>>> tuned for further iterations on this!
> > >>>>
> > >>>> I'm not saying melange is the right way to manage this, but could
> > >> you
> > >>>> elaborate the differences between this proposal and allocating an
> > IP
> > >> in
> > >>>> melange for a device and creating a route entry for it?
> > >>>>
> > >>>> Happy Hacking!
> > >>>>
> > >>>> 7-11
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Mailing list: https://launchpad.net/~netstack
> > >>>> Post to : netstack@lists.launchpad.net
> > >>>> Unsubscribe : https://launchpad.net/~netstack
> > >>>> More help : https://help.launchpad.net/ListHelp
> > >>>
> > >>> --
> > >>> Mailing list: https://launchpad.net/~netstack
> > >>> Post to : netstack@lists.launchpad.net
> > >>> Unsubscribe : https://launchpad.net/~netstack
> > >>> More help : https://help.launchpad.net/ListHelp
> > >
>
>
> --
> Mailing list: https://launchpad.net/~netstack
> Post to : netstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~netstack
> More help : https://help.launchpad.net/ListHelp
>



--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~~~~~~~~~~~~~~~~~~~~~~~~~

On Mon, Apr 2, 2012 at 8:59 PM, Sumit Naiksatam (snaiksat) <
snaiksat@cisco.com> wrote:

> Hi All,
>
> Per the context setup by Dan in his email on the topic of Quantum L3
> Forwarding, here is an addition we are making to the earlier proposal -
> API/specification pertaining to creating and managing "targets". I have
> updated the API specification with this addition and posted the new
> version of the document online. As before, you can find it on the wiki:
> http://wiki.openstack.org/quantum-l3 (please refer to the pdf attached
> therein, version 5). The new additions are in Section 8 (in case you
> want to go straight to those).
>
> Following is a summary of the additions:
> As you might have gleaned, the proposal until this point catered more to
> the tenant/user aspects of the L3 discussion, however it did not address
> the service-provider/operator aspects. Some of the feedback we have
> received until now also points to the missing SP constructs.
>
> The current thinking is that this missing SP aspect should be handled in
> a generic way so as to be able to handle different types of L3 entities
> like gateways, firewalls, etc. More specifically, in the context of the
> current proposal, it amounts to introducing a SP API to manage
> "targets". The key attribute when creating a "target" is the IP address
> being assigned to this "target". Also note that the definition of the
> "target" is in the context of a tenant (and also one or more routetables
> for that tenant). Given this definition, the "target" has the following
> attributes:
> ID: UUID
> Name: Public, Private, VPN, etc.
> IP Address
> Tenant_ID: The tenant for which this "target" is applicable
> Routetable IDs: (Optional parameter) One or more routetable IDs
> belonging to this tenant for which this target is valid. This provides
> for more granular control of the "target " applicability (from a service
> provider perspective, e.g. the "Private" target will map to different
> end points with respect to different routetables for the same tenant).
>
> To illustrate this better, let's take the example of the SP having to
> configure an internet gateway for a particular tenant. Let's say that
> the tenant creates a subnet 10.0.0.0/24, and the SP uses the convention
> of assigning the first IP in the subnet as a gateway (i.e. 10.0.0.1).
> From a logical model perspective, the SP would then create a "target"
> with the following attributes:
> ID : UUID-XYZ-1
> Name: Public
> IP Address: 10.0.0.1
> Tenant_ID: UUID-Tenant-ABC
> Routetable IDs: [UUID-Routetable-UVW-1]
>
> The creation of this logical entity would result in the underlying SP
> implementation mapping it to the requirement for the creation of a
> gateway entity such that on the tenant network side it would have the
> 10.0.0.1 IP address, and it would have another interface to the public
> network (with some PAT/NAT functionality).
>
> Subsequently, the tenant creates a route in the routetable
> UUID-Routetable-UVW-1,
> Source: 10.0.0.0/24, Destination: default, Target: Public
>
> When a VM is now instantiated with an interface on this subnet, the
> default route for the packets going out of this interface will lead them
> to the gateway 10.0.0.1. Exactly how the L2/L3 networking is setup to
> handle this connectivity from the VM to the gateway is specific to that
> particular deployment.
>

I'm still trying to wrap my head around this exactly, so let me ask a
clarifying question: Is a "Routetable" correspond to both a forwarding
rule in the L3-forwarding element, and the description of what routing
should be configured within the VM (e.g., default gateway)?

These two things are definitely related, though one thing I'm struggling
with is what happens if a user changes a route table after a VM has been
booted. Is the assumption that the we can update such config on the VM?
Or that the state would be inconsistent?

Also, how to I describe the routing config that should go inside a VM in a
scenario where I don't want Quantum to be performing the L3 forwarding
(e.g., I have a load-balancer virtual server that is acting as my gateway).


Sorry, I think I need to re-read the proposal again before the summit, as
some of it is still fuzzy :)

Dan



>
> In the above case, the gateway could be a firewall, and which in turn
> might be managed as a separate service. In that case, the SP would
> correlate the above created logical "target" resource with a resource
> created in that firewall service.
>
> We are actively trying to make this a better/simpler proposal, and any
> feedback/participation is very much welcome. Thanks in advance and stay
> tuned for further iterations on this!
>
> ~Sumit.
>



--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt
Nicira, Inc: www.nicira.com
twitter: danwendlandt
~~~~~~~~~~~~~~~~~~~~~~~~~~~

A few (short) thoughts and questions:

* Might be useful to add a use case where a tenant brings their own
network service. As Dan points out, a popular case is a load-balancer
VM. Another interesting set of cases could be encountered in private
cloud environments, where specialized network resources can be
utilized as l3 services (e.g. physical firewalls), as long as quantum
can be made aware of them

* There's some interesting verbiage in the Targets section:
"
This operation returns the list of all the targets which are available
to this particular tenant. These will contain a list of system (i.e.
SP-published targets), and there might be
others which are published by other tenants.
"
Is the thought that the API to register and manage these targets is SP
specific? Or would it be included in this API?

* Authn ... in the face of multi-tenants. I realize that there are
other threads on this, but it ties nicely with the targets discussion
- how do tenants control which other tenants have access to the
services they might be willing to "publish" as targets?



a.

On Wed, Apr 11, 2012 at 5:09 PM, Dan Wendlandt <dan@nicira.com> wrote:
>
>
> On Mon, Apr 2, 2012 at 8:59 PM, Sumit Naiksatam (snaiksat)
> <snaiksat@cisco.com> wrote:
>>
>> Hi All,
>>
>> Per the context setup by Dan in his email on the topic of Quantum L3
>> Forwarding, here is an addition we are making to the earlier proposal -
>> API/specification pertaining to creating and managing "targets". I have
>> updated the API specification with this addition and posted the new
>> version of the document online. As before, you can find it on the wiki:
>> http://wiki.openstack.org/quantum-l3 (please refer to the pdf attached
>> therein, version 5). The new additions are in Section 8 (in case you
>> want to go straight to those).
>>
>> Following is a summary of the additions:
>> As you might have gleaned, the proposal until this point catered more to
>> the tenant/user aspects of the L3 discussion, however it did not address
>> the service-provider/operator aspects. Some of the feedback we have
>> received until now also points to the missing SP constructs.
>>
>> The current thinking is that this missing SP aspect should be handled in
>> a generic way so as to be able to handle different types of L3 entities
>> like gateways, firewalls, etc. More specifically, in the context of the
>> current proposal, it amounts to introducing a SP API to manage
>> "targets". The key attribute when creating a "target" is the IP address
>> being assigned to this "target". Also note that the definition of the
>> "target" is in the context of a tenant (and also one or more routetables
>> for that tenant). Given this definition, the "target" has the following
>> attributes:
>> ID: UUID
>> Name: Public, Private, VPN, etc.
>> IP Address
>> Tenant_ID: The tenant for which this "target" is applicable
>> Routetable IDs: (Optional parameter) One or more routetable IDs
>> belonging to this tenant for which this target is valid. This provides
>> for more granular control of the "target " applicability (from a service
>> provider perspective, e.g. the "Private" target will map to different
>> end points with respect to different routetables for the same tenant).
>>
>> To illustrate this better, let's take the example of the SP having to
>> configure an internet gateway for a particular tenant.  Let's say that
>> the tenant creates a subnet 10.0.0.0/24, and the SP uses the convention
>> of assigning the first IP in the subnet as a gateway (i.e. 10.0.0.1).
>> >From a logical model perspective, the SP would then create a "target"
>> with the following attributes:
>> ID : UUID-XYZ-1
>> Name: Public
>> IP Address: 10.0.0.1
>> Tenant_ID: UUID-Tenant-ABC
>> Routetable IDs: [UUID-Routetable-UVW-1]
>>
>> The creation of this logical entity would result in the underlying SP
>> implementation mapping it to the requirement for the creation of a
>> gateway entity such that on the tenant network side it would have the
>> 10.0.0.1 IP address, and it would have another interface to the public
>> network (with some PAT/NAT functionality).
>>
>> Subsequently, the tenant creates a route in the routetable
>> UUID-Routetable-UVW-1,
>> Source: 10.0.0.0/24, Destination: default, Target: Public
>>
>> When a VM is now instantiated with an interface on this subnet, the
>> default route for the packets going out of this interface will lead them
>> to the gateway 10.0.0.1. Exactly how the L2/L3 networking is setup to
>> handle this connectivity from the VM to the gateway is specific to that
>> particular deployment.
>
>
> I'm still trying to wrap my head around this exactly, so let me ask a
> clarifying question:  Is a "Routetable" correspond to both a forwarding rule
> in the L3-forwarding element, and the description of what routing should be
> configured within the VM (e.g., default gateway)?
>
> These two things are definitely related, though one thing I'm struggling
> with is what happens if a user changes a route table after a VM has been
> booted.  Is the assumption that the we can update such config on the VM?  Or
> that the state would be inconsistent?
>
> Also, how to I describe the routing config that should go inside a VM in a
> scenario where I don't want Quantum to be performing the L3 forwarding
> (e.g., I have a load-balancer virtual server that is acting as my gateway).
>
>
> Sorry, I think I need to re-read the proposal again before the summit, as
> some of it is still fuzzy :)
>
> Dan
>
>
>>
>>
>> In the above case, the gateway could be a firewall, and which in turn
>> might be managed as a separate service. In that case, the SP would
>> correlate the above created logical "target" resource with a resource
>> created in that firewall service.
>>
>> We are actively trying to make this a better/simpler proposal, and any
>> feedback/participation is very much welcome. Thanks in advance and stay
>> tuned for further iterations on this!
>>
>> ~Sumit.
>
>
>
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Dan Wendlandt
> Nicira, Inc: www.nicira.com
> twitter: danwendlandt
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>
> --
> Mailing list: https://launchpad.net/~netstack
> Post to     : netstack@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~netstack
> More help   : https://help.launchpad.net/ListHelp
>

--
Mailing list: https://launchpad.net/~netstack
Post to : netstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~netstack
More help : https://help.launchpad.net/ListHelp

Thanks Andi. Responses inline...

> -----Original Message-----
> From: andi abes [mailto:andi.abes@gmail.com]
> Sent: Wednesday, April 11, 2012 7:22 PM
> To: Dan Wendlandt
> Cc: Sumit Naiksatam (snaiksat); netstack@lists.launchpad.net
> Subject: Re: [Netstack] L3 Forwarding
>
> A few (short) thoughts and questions:
>
> * Might be useful to add a use case where a tenant brings their own
> network service. As Dan points out, a popular case is a load-balancer
> VM. Another interesting set of cases could be encountered in private
> cloud environments, where specialized network resources can be
> utilized as l3 services (e.g. physical firewalls), as long as quantum
> can be made aware of them
>

<Sumit> Yes good point. In general, there are probably at least three scenarios which need to be accounted for -
* A load balancer like service is provided by the Service Provider. E.g. Amazon provides Elastic Load Balancer service. In this case the service is being provided by the same entity which is also managing the other network infrastructure.
* A user wants to deploy his own service. E.g. User has a Load balancer VM and wants to use that to front end his server VMs.
* The service is offered by a 3rd party (it's neither the SP nor the user himself). This may not be relevant for all kinds of services, but this does introduce issues of access control.
</Sumit>

> * There's some interesting verbiage in the Targets section:
> "
> This operation returns the list of all the targets which are available
> to this particular tenant. These will contain a list of system (i.e.
> SP-published targets), and there might be
> others which are published by other tenants.
> "
> Is the thought that the API to register and manage these targets is SP
> specific? Or would it be included in this API?

<Sumit> The thinking on this has evolved from what is mentioned in this email thread. We want to make a distinction between target-types which are abstract, and targets which are realization of those target-types (actual entities). Target-types would be made available by a SP to the tenants (and would be specific to the tenant's context). Tenants would request creation of targets of a particular type. More details in the revised proposal here: http://wiki.openstack.org/quantum-l3?action=AttachFile&do=view&target=revised-quantum-l3-forwarding-model-sumitnaiksatam-2.pdf
</Sumit>

>
> * Authn ... in the face of multi-tenants. I realize that there are
> other threads on this, but it ties nicely with the targets discussion
> - how do tenants control which other tenants have access to the
> services they might be willing to "publish" as targets?

<Sumit> Very good point. I was checking the Keystone use cases page, and I think a similar use case has been listed where a one tenant needs to perform some operation on behalf another tenant. So yes, we will need the support from Keystone.
</Sumit>

>
>
>
> a.
>
> On Wed, Apr 11, 2012 at 5:09 PM, Dan Wendlandt <dan@nicira.com> wrote:
> >
> >
> > On Mon, Apr 2, 2012 at 8:59 PM, Sumit Naiksatam (snaiksat)
> > <snaiksat@cisco.com> wrote:
> >>
> >> Hi All,
> >>
> >> Per the context setup by Dan in his email on the topic of Quantum L3
> >> Forwarding, here is an addition we are making to the earlier
> proposal -
> >> API/specification pertaining to creating and managing "targets". I
> have
> >> updated the API specification with this addition and posted the new
> >> version of the document online. As before, you can find it on the
> wiki:
> >> http://wiki.openstack.org/quantum-l3 (please refer to the pdf
> attached
> >> therein, version 5). The new additions are in Section 8 (in case you
> >> want to go straight to those).
> >>
> >> Following is a summary of the additions:
> >> As you might have gleaned, the proposal until this point catered
> more to
> >> the tenant/user aspects of the L3 discussion, however it did not
> address
> >> the service-provider/operator aspects. Some of the feedback we have
> >> received until now also points to the missing SP constructs.
> >>
> >> The current thinking is that this missing SP aspect should be
> handled in
> >> a generic way so as to be able to handle different types of L3
> entities
> >> like gateways, firewalls, etc. More specifically, in the context of
> the
> >> current proposal, it amounts to introducing a SP API to manage
> >> "targets". The key attribute when creating a "target" is the IP
> address
> >> being assigned to this "target". Also note that the definition of
> the
> >> "target" is in the context of a tenant (and also one or more
> routetables
> >> for that tenant). Given this definition, the "target" has the
> following
> >> attributes:
> >> ID: UUID
> >> Name: Public, Private, VPN, etc.
> >> IP Address
> >> Tenant_ID: The tenant for which this "target" is applicable
> >> Routetable IDs: (Optional parameter) One or more routetable IDs
> >> belonging to this tenant for which this target is valid. This
> provides
> >> for more granular control of the "target " applicability (from a
> service
> >> provider perspective, e.g. the "Private" target will map to
> different
> >> end points with respect to different routetables for the same
> tenant).
> >>
> >> To illustrate this better, let's take the example of the SP having
> to
> >> configure an internet gateway for a particular tenant.  Let's say
> that
> >> the tenant creates a subnet 10.0.0.0/24, and the SP uses the
> convention
> >> of assigning the first IP in the subnet as a gateway (i.e.
> 10.0.0.1).
> >> >From a logical model perspective, the SP would then create a
> "target"
> >> with the following attributes:
> >> ID : UUID-XYZ-1
> >> Name: Public
> >> IP Address: 10.0.0.1
> >> Tenant_ID: UUID-Tenant-ABC
> >> Routetable IDs: [UUID-Routetable-UVW-1]
> >>
> >> The creation of this logical entity would result in the underlying
> SP
> >> implementation mapping it to the requirement for the creation of a
> >> gateway entity such that on the tenant network side it would have
> the
> >> 10.0.0.1 IP address, and it would have another interface to the
> public
> >> network (with some PAT/NAT functionality).
> >>
> >> Subsequently, the tenant creates a route in the routetable
> >> UUID-Routetable-UVW-1,
> >> Source: 10.0.0.0/24, Destination: default, Target: Public
> >>
> >> When a VM is now instantiated with an interface on this subnet, the
> >> default route for the packets going out of this interface will lead
> them
> >> to the gateway 10.0.0.1. Exactly how the L2/L3 networking is setup
> to
> >> handle this connectivity from the VM to the gateway is specific to
> that
> >> particular deployment.
> >
> >
> > I'm still trying to wrap my head around this exactly, so let me ask a
> > clarifying question:  Is a "Routetable" correspond to both a
> forwarding rule
> > in the L3-forwarding element, and the description of what routing
> should be
> > configured within the VM (e.g., default gateway)?
> >
> > These two things are definitely related, though one thing I'm
> struggling
> > with is what happens if a user changes a route table after a VM has
> been
> > booted.  Is the assumption that the we can update such config on the
> VM?  Or
> > that the state would be inconsistent?
> >
> > Also, how to I describe the routing config that should go inside a VM
> in a
> > scenario where I don't want Quantum to be performing the L3
> forwarding
> > (e.g., I have a load-balancer virtual server that is acting as my
> gateway).
> >
> >
> > Sorry, I think I need to re-read the proposal again before the
> summit, as
> > some of it is still fuzzy :)
> >
> > Dan
> >
> >
> >>
> >>
> >> In the above case, the gateway could be a firewall, and which in
> turn
> >> might be managed as a separate service. In that case, the SP would
> >> correlate the above created logical "target" resource with a
> resource
> >> created in that firewall service.
> >>
> >> We are actively trying to make this a better/simpler proposal, and
> any
> >> feedback/participation is very much welcome. Thanks in advance and
> stay
> >> tuned for further iterations on this!
> >>
> >> ~Sumit.
> >
> >
> >
> >
> > --
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > Dan Wendlandt
> > Nicira, Inc: www.nicira.com
> > twitter: danwendlandt
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~
> >
> >
> > --
> > Mailing list: https://launchpad.net/~netstack
> > Post to     : netstack@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~netstack
> > More help   : https://help.launchpad.net/ListHelp
> >

--
Mailing list: https://launchpad.net/~netstack
Post to : netstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~netstack
More help : https://help.launchpad.net/ListHelp

Hi Dan, All,



I missed this one...I think I responded to another email in which you
had similar questions, but I noticed that the netstack mailer was not
copied on that (thanks for the feedback!).



Some responses/thoughts inline. Discussion to be continued on the white
board! J



I believe this session is still targeted for 3 PM, Tuesday. Updated
proposal here:

http://wiki.openstack.org/quantum-l3?action=AttachFile&do=view&target=re
vised-quantum-l3-forwarding-model-sumitnaiksatam-2.pdf



Thanks,

~Sumit.



From: Dan Wendlandt [mailto:dan@nicira.com]
Sent: Wednesday, April 11, 2012 2:09 PM
To: Sumit Naiksatam (snaiksat)
Cc: netstack@lists.launchpad.net; Brad Hall; Carl Perry
Subject: Re: L3 Forwarding





On Mon, Apr 2, 2012 at 8:59 PM, Sumit Naiksatam (snaiksat)
<snaiksat@cisco.com> wrote:

Hi All,

Per the context setup by Dan in his email on the topic of Quantum L3
Forwarding, here is an addition we are making to the earlier proposal -
API/specification pertaining to creating and managing "targets". I have
updated the API specification with this addition and posted the new
version of the document online. As before, you can find it on the wiki:
http://wiki.openstack.org/quantum-l3 (please refer to the pdf attached
therein, version 5). The new additions are in Section 8 (in case you
want to go straight to those).

Following is a summary of the additions:
As you might have gleaned, the proposal until this point catered more to
the tenant/user aspects of the L3 discussion, however it did not address
the service-provider/operator aspects. Some of the feedback we have
received until now also points to the missing SP constructs.

The current thinking is that this missing SP aspect should be handled in
a generic way so as to be able to handle different types of L3 entities
like gateways, firewalls, etc. More specifically, in the context of the
current proposal, it amounts to introducing a SP API to manage
"targets". The key attribute when creating a "target" is the IP address
being assigned to this "target". Also note that the definition of the
"target" is in the context of a tenant (and also one or more routetables
for that tenant). Given this definition, the "target" has the following
attributes:
ID: UUID
Name: Public, Private, VPN, etc.
IP Address
Tenant_ID: The tenant for which this "target" is applicable
Routetable IDs: (Optional parameter) One or more routetable IDs
belonging to this tenant for which this target is valid. This provides
for more granular control of the "target " applicability (from a service
provider perspective, e.g. the "Private" target will map to different
end points with respect to different routetables for the same tenant).

To illustrate this better, let's take the example of the SP having to
configure an internet gateway for a particular tenant. Let's say that
the tenant creates a subnet 10.0.0.0/24, and the SP uses the convention
of assigning the first IP in the subnet as a gateway (i.e. 10.0.0.1).
>From a logical model perspective, the SP would then create a "target"
with the following attributes:
ID : UUID-XYZ-1
Name: Public
IP Address: 10.0.0.1
Tenant_ID: UUID-Tenant-ABC
Routetable IDs: [UUID-Routetable-UVW-1]

The creation of this logical entity would result in the underlying SP
implementation mapping it to the requirement for the creation of a
gateway entity such that on the tenant network side it would have the
10.0.0.1 IP address, and it would have another interface to the public
network (with some PAT/NAT functionality).

Subsequently, the tenant creates a route in the routetable
UUID-Routetable-UVW-1,
Source: 10.0.0.0/24, Destination: default, Target: Public

When a VM is now instantiated with an interface on this subnet, the
default route for the packets going out of this interface will lead them
to the gateway 10.0.0.1. Exactly how the L2/L3 networking is setup to
handle this connectivity from the VM to the gateway is specific to that
particular deployment.



I'm still trying to wrap my head around this exactly, so let me ask a
clarifying question: Is a "Routetable" correspond to both a forwarding
rule in the L3-forwarding element, and the description of what routing
should be configured within the VM (e.g., default gateway)?



<Sumit> Yes, that is thinking. </Sumit>



These two things are definitely related, though one thing I'm struggling
with is what happens if a user changes a route table after a VM has been
booted. Is the assumption that the we can update such config on the VM?
Or that the state would be inconsistent?



<Sumit> We could take either approach, but it should be possible to
update the config as well, right? May be using shorter DHCP leases?
</Sumit>



Also, how to I describe the routing config that should go inside a VM in
a scenario where I don't want Quantum to be performing the L3 forwarding
(e.g., I have a load-balancer virtual server that is acting as my
gateway).



<Sumit> I think you will find that this is covered in the cases where a
target can be created by pointing it to a resource owned by the tenant
(LB in a VM as you state here), or to a resource owned by a 3rd party.
</Sumit>



Sorry, I think I need to re-read the proposal again before the summit,
as some of it is still fuzzy :)



Dan






In the above case, the gateway could be a firewall, and which in
turn
might be managed as a separate service. In that case, the SP
would
correlate the above created logical "target" resource with a
resource
created in that firewall service.

We are actively trying to make this a better/simpler proposal,
and any
feedback/participation is very much welcome. Thanks in advance
and stay
tuned for further iterations on this!

~Sumit.







--
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dan Wendlandt

Nicira, Inc: www.nicira.com

twitter: danwendlandt
~~~~~~~~~~~~~~~~~~~~~~~~~~~