Mailing List Archive

I assume it is OK to share this with libvirt (aka Daniel) and Xen-CIM now?

- Gareth

Gareth S. Bestor, PhD.
IBM Linux Technology Center
M/S DES2-01
15300 SW Koll Parkway, Beaverton, OR 97006
503-578-3186, T/L 775-3186, Fax 503-578-3186

Ewan Mellor <ewan@xensource.com>@lists.xensource.com on 06/22/2006 10:01:30
AM

Sent by: xen-api-bounces@lists.xensource.com


To: Xen-API <xen-api@lists.xensource.com>
cc:
Subject: [Xen-API] Xen Management API draft


Attached is a draft Xen Management API. This document presents our ideas
in
terms of a data model, with an implied binding to XML-RPC calls. There are
a
few examples in there too -- hopefully it's clear how the mapping between
the
document and the wire protocol.

Hopefully, we can standardise the data model and wire protocol (one
implying
the other) and then the Xen project would guarantee that that wire protocol
would be supported for the long term. Behind that interface, we would then
be
free to improve Xend, and giving a solid foundation for Xen-CIM, and third
party GUIs and tools and so on. The API becomes effectively part of the
"guarantees" for Xen.

I expect that we will be asked for client-side bindings for a number of
languages. It is my intention that a binding to any particular language
would
be a very thin translation from the host language's values and types onto
the
fixed XML-RPC, and then we shouldn't have too much trouble maintaining
bindings in Python or Perl or C++ or whatever people want.

This document is totally open to discussion and modification, so please,
let's
get started!

In particular, we need to get this API into a state so that it is the right
API for Xen-CIM. We need to think whether libvirt is ready to rely upon
this
API too. That would mean moving the hypercalls and Xenstore reads and
writes
out of libvirt, and pushing it up the stack alongside the other client-side
bindings.

Like the OVA spec, this document is for narrow circulation, and once we are
getting happy with it, we'll circulate it more widely. I'd like to be able
to
circulate this before OLS, as that will be a good chance to discuss it with
people face-to-face once they've had a chance to read it.

Cheers,

Ewan.

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Thu, 2006-06-22 at 18:01 +0100, Ewan Mellor wrote:
>
> Like the OVA spec, this document is for narrow circulation, and once
> we are getting happy with it, we'll circulate it more widely. I'd
> like to be able to circulate this before OLS, as that will be a good
> chance to discuss it with people face-to-face once they've had a
> chance to read it.

I'm confused. You said this is an open mailing list, but this document
says "DO NOT CIRCULATE" on the title page?

--
Hollis Blanchard
IBM Linux Technology Center


_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Thu, 2006-06-22 at 18:01 +0100, Ewan Mellor wrote:
> Attached is a draft Xen Management API. This document presents our ideas in
> terms of a data model, with an implied binding to XML-RPC calls. There are a
> few examples in there too -- hopefully it's clear how the mapping between the
> document and the wire protocol.

Hi, I'm concerned about some of the x86 assumptions in this document. In
particular:

enum bios_boot_option:
Is this relevant outside of HVM? Perhaps both those "bios" fields in
class VM could be replaced with a "boot device" string? Many non-x86
systems specify boot devices with free text; they are not limited by
legacy x86 BIOS. I believe that's even true for Intel's EFI.

enum cpu_feature:
Very concerned about this one. How is this supposed to be used? If
higher-level code uses constants like "PAE", that software will need
modification for every non-x86 architecture, and I think that should be
an anti-goal. What makes sense to me is if Xen passes up an *opaque*
bitmask specifying required features (hcall: "what features does domain
7 require?"), and that bitmask could be passed back down to Xen e.g.
when migrating a VM (hcall: "do you support these features?").
I see that IA64 has its own "feature" bit. When thinking about managing
a heterogeneous cluster of Xen systems, it would probably make more
sense to have an "architecture" field in class VM. Then we could add a
"compatible architectures" field to class host_cpu, i.e. a list of
architectures that CPU can support. For an x86-64 CPU, that would be
"x86, x86-64". That way, when looking for a system on which to start an
x86 VM, the user could be presented with a list of all x86 and x86-64
systems being managed.
Each architecture needs its own "optional feature" values, but there's
no sense in putting them all into the same namespace. For example,
PowerPC would have an FPU bit, but also Altivec, and of course almost
none of the x86 bits listed would be relevant for us, so it's a waste of
bits. I'm sure IA64 has the same situation.
Of course, there are already a lot of bits in this list; it's
conceivable we could run out of bits in an int/long/whatever in the
future. Would it be better to communicate required features as strings
instead of an enum? Or am I misunderstanding, and the "enums" in this
document do not mean the same as "enum" in C, instead meaning "list of
acceptable string values"?

Those are the first issues that stand out for me. In general I'd like to
remind everybody that computer networks are not homogeneous, so let's
not box ourselves into a corner in the future by designing an
x86-specific API today... :)

--
Hollis Blanchard
IBM Linux Technology Center


_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

I also reviewed the API document this afternoon. I'll say that I'm a
bit less emphatic about this one (although I still appreciate the effort).

My basic problem is that this API is huge. I understand why most of
this stuff is included but at the same time, it greatly conflicts with
existing systems that already do this (for instance, IBM Director's CIM
interfaces).

What I'd like to see is a minimalistic API that only dealt with Xen
specific stuff. If you wished to have an accompanying API that also
provided other system info (via the same model) that would seem like a
rather good compromise.

We're interested in a minimalistic Xen stack. So being able to omit the
portions of this API that we don't need would be very useful.

Thoughts?

Regards,

Anthony Liguori

Ewan Mellor wrote:
> Attached is a draft Xen Management API. This document presents our ideas in
> terms of a data model, with an implied binding to XML-RPC calls. There are a
> few examples in there too -- hopefully it's clear how the mapping between the
> document and the wire protocol.
>
> Hopefully, we can standardise the data model and wire protocol (one implying
> the other) and then the Xen project would guarantee that that wire protocol
> would be supported for the long term. Behind that interface, we would then be
> free to improve Xend, and giving a solid foundation for Xen-CIM, and third
> party GUIs and tools and so on. The API becomes effectively part of the
> "guarantees" for Xen.
>
> I expect that we will be asked for client-side bindings for a number of
> languages. It is my intention that a binding to any particular language would
> be a very thin translation from the host language's values and types onto the
> fixed XML-RPC, and then we shouldn't have too much trouble maintaining
> bindings in Python or Perl or C++ or whatever people want.
>
> This document is totally open to discussion and modification, so please, let's
> get started!
>
> In particular, we need to get this API into a state so that it is the right
> API for Xen-CIM. We need to think whether libvirt is ready to rely upon this
> API too. That would mean moving the hypercalls and Xenstore reads and writes
> out of libvirt, and pushing it up the stack alongside the other client-side
> bindings.
>
> Like the OVA spec, this document is for narrow circulation, and once we are
> getting happy with it, we'll circulate it more widely. I'd like to be able to
> circulate this before OLS, as that will be a good chance to discuss it with
> people face-to-face once they've had a chance to read it.
>
> Cheers,
>
> Ewan.
> ------------------------------------------------------------------------
>
> _______________________________________________
> xen-api mailing list
> xen-api@lists.xensource.com
> http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api


_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

Ewan Mellor wrote:
> Attached is a draft Xen Management API. This document presents our ideas in
> terms of a data model, with an implied binding to XML-RPC calls. There are a
> few examples in there too -- hopefully it's clear how the mapping between the
> document and the wire protocol.

Ewan, I like the API a lot. The storage repository class needs some
additional thought. It really requires some dbms-like attributes. I will
try to get some more specific feedback on the storage repository class
to you in a different note.

here is some cursory feedback:

1.4.1 transport layer

We should also support bare xml which would enable file redirection
(stdin and stdout) and the use of ssh.

1.4.2

session -

We should allow optional delegation of authentication to the transport
layer. For example, use ssh public key authentication instead of a
userid/password login.

1.6 (to do)

Authorization schema that maps uids to operations on objects.

notification - need a notification class and a notification listener
on the client.

2.6.1 VM class


Need an addition field "profile" that allows the user to classify VMs
for particular purposes. For example, "web server," "sendmail switch,"
"dbms host," "devel workstation," etc.

This should be a list attribute. Might be good to supply some
enumerated values but also allow values that are not enumerated.

Clone method needs to have a customization parameter. Simply cloning a
VM is not sufficient - the clone at minimum needs a different host
name, network configuration, and changes to various config files.

One way to customize a cloned image is to patch the
filesystem. Another is to replace specific files with new versions
(usually works best for binary files).

VM needs a migrate method:

Bool migrate(session id s, vm ref vm, host ref current, host ref
destination, Bool test)

If test is true, then the migration method will not actually migrate
the VM, but it will qualify the destination.


2.8 Host class

Host needs additional attributes that will allow automated
determination that a given host is capable of hosting a given VM.

For example, a VM may need specific host characteristics beyond CPU: a
minimum amount of memory, TPM, network bandwidth, and so on.

If the host class has these attributes we can create an addition
method that automatically determines if a host can host a particular
vm.

Bool qualify(session id s, vm ref vm, host ref host)


2.9 Class Network

Network needs additional attributes that provide media
characteristics of the NIC.

RO bandwidth integer Bandwidth in mbps

RO latency integer time in ms for an icmp roundtrip to a host on the
same subnet.


_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

Mike D. Day wrote:
> Ewan Mellor wrote:
>> Attached is a draft Xen Management API. This document presents our

One item I forgot in the previous email:

The NIC field of the Network class should be a list (Set) so that we can
signify NIC teaming. (Combining physical NICs in a single host interface
to achieve greater bandwidth).

Mike

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

> I'm confused. You said this is an open mailing list, but this document
> says "DO NOT CIRCULATE" on the title page?

The document is free to circulate and entirely open. The "DO NOT
CIRCULATE" was boiler-plate text that should not have been included on
this particular titlepage. :)

Sorry for the confusion.

Best regards,
Richard
--
Richard Sharp
Senior Engineer
Xensource

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Fri, Jun 23, 2006 at 04:47:04PM +0000, Richard Sharp wrote:
>
> >I'm confused. You said this is an open mailing list, but this document
> >says "DO NOT CIRCULATE" on the title page?
>
> The document is free to circulate and entirely open. The "DO NOT
> CIRCULATE" was boiler-plate text that should not have been included on
> this particular titlepage. :)

So I assume the Copyright part in the first page stating among other
things:
"Access to this work is restricted to XenSource, Inc."
can be considered an error too, good that makes it easier. Would be
good to get a document with an updated Copyright too,

thanks in advance,

Daniel

--
Daniel Veillard | Red Hat http://redhat.com/
veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On the subject, the copyright notice of the front page is pretty scary too
- can it be dropped or at least toned down?

- Gareth

Dr. Gareth S. Bestor
IBM Linux Technology Center
M/S DES2-01
15300 SW Koll Parkway, Beaverton, OR 97006
503-578-3186, T/L 775-3186, Fax 503-578-3186




Richard Sharp <rsharp@xensource.com>
Sent by: xen-api-bounces@lists.xensource.com
06/23/2006 12:47 PM

To
hollisb@us.ltcfwd.linux.ibm.com
cc
Xen-API <xen-api@lists.xensource.com>
Subject
Re: [Xen-API] Xen Management API draft







> I'm confused. You said this is an open mailing list, but this document
> says "DO NOT CIRCULATE" on the title page?

The document is free to circulate and entirely open. The "DO NOT
CIRCULATE" was boiler-plate text that should not have been included on
this particular titlepage. :)

Sorry for the confusion.

Best regards,
Richard
--
Richard Sharp
Senior Engineer
Xensource

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

> We're interested in a minimalistic Xen stack. So being able to omit the
> portions of this API that we don't need would be very useful.

Thanks for this feedback. Please could you elaborate on which parts of the
API you don't need.

Best regards,
Richard
--
Richard Sharp
Senior Engineer
Xensource

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

Ewan Mellor wrote:
> Attached is a draft Xen Management API. This document presents our ideas in
> terms of a data model, with an implied binding to XML-RPC calls. There are a
> few examples in there too -- hopefully it's clear how the mapping between the
> document and the wire protocol.
>

Thanks Ewan.

> In particular, we need to get this API into a state so that it is the right
> API for Xen-CIM. We need to think whether libvirt is ready to rely upon this
> API too. That would mean moving the hypercalls and Xenstore reads and writes
> out of libvirt, and pushing it up the stack alongside the other client-side
> bindings.
>

Is it realistic to use XML-RPC for tasks such as resource monitoring?
Seems like there should be an optional, lower-latency, lighter-weight
mechanism to retrieve things such as host_cpu.utilisation,
VBD.IO_bandwidth/incoming_kbs, etc. (Makes me thing of Uncertainty
Principle or Observer Effect, i.e. we crank up cpu utilization by asking
for cpu utilization :-)). I guess the hypercalls and Xenstore
reads/writes mentioned above would still exist but not be part of the
"guarantees" for Xen.

Many of the RPCs return void, e.g. VM.start(), VM.pause(), etc. How are
failures indicated? Via the Status and ErrorDescription elements of
return value struct?

Regards,
Jim


_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

> Is it realistic to use XML-RPC for tasks such as resource monitoring?
> Seems like there should be an optional, lower-latency, lighter-weight
> mechanism to retrieve things such as host_cpu.utilisation,
> VBD.IO_bandwidth/incoming_kbs, etc.

The whole stats reporting interface certainly needs more thought.

However, I think it definitely should still use the xml-rpc interface,
we just need to 'cook' the raw data into something more directly useful
within xend so that there's no need to do high-rate polling on the API.
This might involve using something like rrdtool to generate averages
over different time scales etc.

Ian

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Thu, Jun 22, 2006 at 12:51:42PM -0500, Hollis Blanchard wrote:

> On Thu, 2006-06-22 at 18:01 +0100, Ewan Mellor wrote:
> >
> > Like the OVA spec, this document is for narrow circulation, and once
> > we are getting happy with it, we'll circulate it more widely. I'd
> > like to be able to circulate this before OLS, as that will be a good
> > chance to discuss it with people face-to-face once they've had a
> > chance to read it.
>
> I'm confused. You said this is an open mailing list, but this document
> says "DO NOT CIRCULATE" on the title page?

I would like to keep discussion and circulation of this document on this list,
for now. I'm happy for other people to join this list, and to see the
document, but we've already had problems with people seeing this work out of
context and getting confused. I don't think that the document has enough in
the way of explanation to stand on its own just yet, so I think that the best
way to ensure that everyone gets the right context is to keep the discussion
here on this list, just for a short while.

Once we're happy that the document can stand on its own feet, in good time
before OLS, then we can relax the circulation restrictions, change the
copyright statement, and so on.

Ewan.

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Sat, Jun 24, 2006 at 09:24:40AM +0100, Ian Pratt wrote:
> > Is it realistic to use XML-RPC for tasks such as resource monitoring?
> > Seems like there should be an optional, lower-latency, lighter-weight
> > mechanism to retrieve things such as host_cpu.utilisation,
> > VBD.IO_bandwidth/incoming_kbs, etc.
>
> The whole stats reporting interface certainly needs more thought.
>
> However, I think it definitely should still use the xml-rpc interface,

definitely ? because that's really where it should be done, or just because
you don't want any direct hypervisor calls ? A direct hypervisor call
don't even force a dom0 context switch to implement, in contrast the RPC
is, very very expensive.

> we just need to 'cook' the raw data into something more directly useful
> within xend so that there's no need to do high-rate polling on the API.
> This might involve using something like rrdtool to generate averages
> over different time scales etc.

I don't think trying to add more feature into xend is really a good
way to solve the real problem, in my opinion the hyprvisor calls for
getting performance data should be considered at least as standard as
any xend RPC api. I understand that we should avoid them for any call
with side effect, but for monitoring, I don't see how you will be able
to offer a flexible alternative. You don't need very high rate polling
to just eat all CPU when going though RPC and Xend as our experiments
have shown. In libvirt if I don't use the hypervisor calls a few refresh
per second of the state of the hypervisor takes a very significant fraction
of the overall CPU (for example if you use the fedora virtualization applet
for monitoring as an non-root user).

Daniel

--
Daniel Veillard | Red Hat http://redhat.com/
veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Sat, Jun 24, 2006 at 07:22:20PM -0400, Daniel Veillard wrote:

> On Sat, Jun 24, 2006 at 09:24:40AM +0100, Ian Pratt wrote:
> > > Is it realistic to use XML-RPC for tasks such as resource monitoring?
> > > Seems like there should be an optional, lower-latency, lighter-weight
> > > mechanism to retrieve things such as host_cpu.utilisation,
> > > VBD.IO_bandwidth/incoming_kbs, etc.
> >
> > The whole stats reporting interface certainly needs more thought.
> >
> > However, I think it definitely should still use the xml-rpc interface,
>
> definitely ? because that's really where it should be done, or just because
> you don't want any direct hypervisor calls ? A direct hypervisor call
> don't even force a dom0 context switch to implement, in contrast the RPC
> is, very very expensive.

Your answer here goes right to the heart of what you want libvirt to be. We
know that we will need a C binding to the XML-RPC, so that people can make RPC
calls from their C programs. The CIM guys need this, for example. I thought
that libvirt was trying to be this binding, and that the hypercalls in there
at the moment were just a stop-gap.

Above though, you seem to be talking of libvirt as something different to that
-- it's a library that sits on the managed node, that uses Xend when it needs
to, but that is happy to bypass Xend when it can, for performance.
Architecturally, that's a very different beast. Is that how you see libvirt
developing? Do you intend to continue to bypass Xend in the long-term, where
performance can be gained from doing so? That's an interesting idea.

Ewan.

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Sun, Jun 25, 2006 at 12:37:23AM +0100, Ewan Mellor wrote:
> On Sat, Jun 24, 2006 at 07:22:20PM -0400, Daniel Veillard wrote:
>
> > On Sat, Jun 24, 2006 at 09:24:40AM +0100, Ian Pratt wrote:
> > > > Is it realistic to use XML-RPC for tasks such as resource monitoring?
> > > > Seems like there should be an optional, lower-latency, lighter-weight
> > > > mechanism to retrieve things such as host_cpu.utilisation,
> > > > VBD.IO_bandwidth/incoming_kbs, etc.
> > >
> > > The whole stats reporting interface certainly needs more thought.
> > >
> > > However, I think it definitely should still use the xml-rpc interface,
> >
> > definitely ? because that's really where it should be done, or just because
> > you don't want any direct hypervisor calls ? A direct hypervisor call
> > don't even force a dom0 context switch to implement, in contrast the RPC
> > is, very very expensive.
>
> Your answer here goes right to the heart of what you want libvirt to be. We
> know that we will need a C binding to the XML-RPC, so that people can make RPC
> calls from their C programs. The CIM guys need this, for example. I thought
> that libvirt was trying to be this binding, and that the hypercalls in there
> at the moment were just a stop-gap.

They are there because for monitoring, just having an RPC based mechanism
could eats 15-20% of the CPU time of an otherwise idle machine when using the
monitoring applet based on libvirt.

> Above though, you seem to be talking of libvirt as something different to that
> -- it's a library that sits on the managed node, that uses Xend when it needs
> to, but that is happy to bypass Xend when it can, for performance.
> Architecturally, that's a very different beast. Is that how you see libvirt
> developing? Do you intend to continue to bypass Xend in the long-term, where
> performance can be gained from doing so? That's an interesting idea.

Well libvirt must be usable for at least minimal monitoring. It uses xend
access by default for all operations except in the case where 1/ the user
can actually use something else (i.e. root currently) and 2/ the operation
is better implemented by other means.
Architecturally, this is based on some driver plugin definition, and
there is a plugin for xend access, one for xenstore access and one for the
xen hypervisor access. The architecture is expected to remain that way,
for example there is a test plugin now for regression testing too.
From the API point of view it's the same whatever the mechanism used,
underneath the library will try to use the best available method. And for
example if xend is dead and the user is root libvirt will first contact
the xenstore if the call can be done that way, and then fallback to the
hypervisor if that's not possible or failed.

I hope this helps clarify the current design a bit :-)
I don't think a pure C binding over a very specific XML-RPC API would be
very attractive as a standalone library, maybe I'm wrong.

Daniel

--
Daniel Veillard | Red Hat http://redhat.com/
veillard@redhat.com | libxml GNOME XML XSLT toolkit http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Sun, Jun 25, 2006 at 03:52:23AM -0400, Daniel Veillard wrote:

> On Sun, Jun 25, 2006 at 12:37:23AM +0100, Ewan Mellor wrote:
> > On Sat, Jun 24, 2006 at 07:22:20PM -0400, Daniel Veillard wrote:
> >
> > > On Sat, Jun 24, 2006 at 09:24:40AM +0100, Ian Pratt wrote:
> > > > > Is it realistic to use XML-RPC for tasks such as resource monitoring?
> > > > > Seems like there should be an optional, lower-latency, lighter-weight
> > > > > mechanism to retrieve things such as host_cpu.utilisation,
> > > > > VBD.IO_bandwidth/incoming_kbs, etc.
> > > >
> > > > The whole stats reporting interface certainly needs more thought.
> > > >
> > > > However, I think it definitely should still use the xml-rpc interface,
> > >
> > > definitely ? because that's really where it should be done, or just because
> > > you don't want any direct hypervisor calls ? A direct hypervisor call
> > > don't even force a dom0 context switch to implement, in contrast the RPC
> > > is, very very expensive.
> >
> > Your answer here goes right to the heart of what you want libvirt to be. We
> > know that we will need a C binding to the XML-RPC, so that people can make RPC
> > calls from their C programs. The CIM guys need this, for example. I thought
> > that libvirt was trying to be this binding, and that the hypercalls in there
> > at the moment were just a stop-gap.
>
> They are there because for monitoring, just having an RPC based mechanism
> could eats 15-20% of the CPU time of an otherwise idle machine when using the
> monitoring applet based on libvirt.
>
> > Above though, you seem to be talking of libvirt as something different to that
> > -- it's a library that sits on the managed node, that uses Xend when it needs
> > to, but that is happy to bypass Xend when it can, for performance.
> > Architecturally, that's a very different beast. Is that how you see libvirt
> > developing? Do you intend to continue to bypass Xend in the long-term, where
> > performance can be gained from doing so? That's an interesting idea.
>
> Well libvirt must be usable for at least minimal monitoring. It uses xend
> access by default for all operations except in the case where 1/ the user
> can actually use something else (i.e. root currently) and 2/ the operation
> is better implemented by other means.
> Architecturally, this is based on some driver plugin definition, and
> there is a plugin for xend access, one for xenstore access and one for the
> xen hypervisor access. The architecture is expected to remain that way,
> for example there is a test plugin now for regression testing too.
> From the API point of view it's the same whatever the mechanism used,
> underneath the library will try to use the best available method. And for
> example if xend is dead and the user is root libvirt will first contact
> the xenstore if the call can be done that way, and then fallback to the
> hypervisor if that's not possible or failed.

I understand your arguments here, especially the point about the cost of using
Xend for second-by-second load monitoring. What's interesting here is that
this places libvirt in a curious place, architecturally.

I see the management stack looking like this:

Tools
-----
Binding to XML-RPC
-----
|
| XML-RPC (standard and fixed)
|
-----
Binding to XML-RPC
-----
Xend
---------------------
libxc libxs
----- -----
Hypervisor Xenstored


This has a number of notable features:

o The tools, once compiled, are not tied to a specific version of Xen.

o The existence of Xenstored is not implied at the interface -- it's an
implementation detail.

o The tools do not need to run on the managed node.

o The tools don't need to run as root to make modifications -- they can
authenticate over the XML-RPC to Xend, which is the only thing that needs to
run as root.


What you are saying is that libvirt is not intended to be near the top of this
stack, but instead will talk to Xend (over XML-RPC) or libxc, or libxs as it
sees fit. That's fine, but it's very different to what I _thought_ you were
trying to do!

Your design has a number of tradeoffs:

o Performance-critical stuff is much easier to do.

o Tools compiled against libvirt will be fixed to that version of the
hypervisor, will have to run as root to make modifications, and will need to
run on the managed node.

o libvirt will have to be kept in sync with changes in Xend that change the
store layout (this is currently an undocumented interface, though it's pretty
stable).


To address the load-monitoring concerns, we clearly have to do something. I
would say that remote access to the node is still important, so we need to
design a "cooked" interface to the load statistics, one that is suitable for
use across the wire. You clearly wouldn't poll the managed host for each stat
individually every second if you were talking to it remotely. We need to
instead design an API that looks something like "please give me the
thirty-second load average for the following list of stats", with the
monitoring, trending, and averaging being done on the managed host.

> I hope this helps clarify the current design a bit :-)
> I don't think a pure C binding over a very specific XML-RPC API would be
> very attractive as a standalone library, maybe I'm wrong.

That certainly clarifies things a lot, thank you! A C binding would still be
useful -- it could be the bottom end of your Xend interface plugin, for
example. It's obviously only a small portion of the problem that libvirt is
trying to solve, that's the difference.

Ewan.

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Thu, Jun 22, 2006 at 05:49:49PM -0500, Anthony Liguori wrote:

> I also reviewed the API document this afternoon. I'll say that I'm a
> bit less emphatic about this one (although I still appreciate the effort).
>
> My basic problem is that this API is huge. I understand why most of
> this stuff is included but at the same time, it greatly conflicts with
> existing systems that already do this (for instance, IBM Director's CIM
> interfaces).
>
> What I'd like to see is a minimalistic API that only dealt with Xen
> specific stuff. If you wished to have an accompanying API that also
> provided other system info (via the same model) that would seem like a
> rather good compromise.

The intention is for this API to be precisely what you would need to be able
to manage Xen hosts and guests, no more, no less. It's the API that we would
put underneath things like the Xen-CIM providers, so yes, I'm expecting the
services on offer to look similar!

What we want to provide is an object-oriented data model, much the same as
CIM, but lighter-weight where CIM models things that we don't care about, and
including those things that we _do_ care about that CIM hasn't got around to
yet.

> We're interested in a minimalistic Xen stack. So being able to omit the
> portions of this API that we don't need would be very useful.

I don't think that there's anything in this spec that one could omit, so could
you elaborate here?

Thanks,

Ewan.


_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Thu, Jun 22, 2006 at 01:34:12PM -0500, Hollis Blanchard wrote:

> On Thu, 2006-06-22 at 18:01 +0100, Ewan Mellor wrote:
> > Attached is a draft Xen Management API. This document presents our ideas in
> > terms of a data model, with an implied binding to XML-RPC calls. There are a
> > few examples in there too -- hopefully it's clear how the mapping between the
> > document and the wire protocol.
>
> Hi, I'm concerned about some of the x86 assumptions in this document. In
> particular:
>
> enum bios_boot_option:
> Is this relevant outside of HVM? Perhaps both those "bios" fields in
> class VM could be replaced with a "boot device" string? Many non-x86
> systems specify boot devices with free text; they are not limited by
> legacy x86 BIOS. I believe that's even true for Intel's EFI.

IIRC, people have spoke in the past about using a "BIOS" even with paravirt
guests, as opposed to using pygrub for example, so I think that this is
potentially relevant outside of HVM. I wouldn't have a problem with making
these free-form strings instead, or providing a different field, if you think
you need extra flexibility.

> enum cpu_feature:
> Very concerned about this one. How is this supposed to be used? If
> higher-level code uses constants like "PAE", that software will need
> modification for every non-x86 architecture, and I think that should be
> an anti-goal. What makes sense to me is if Xen passes up an *opaque*
> bitmask specifying required features (hcall: "what features does domain
> 7 require?"), and that bitmask could be passed back down to Xen e.g.
> when migrating a VM (hcall: "do you support these features?").
> I see that IA64 has its own "feature" bit. When thinking about managing
> a heterogeneous cluster of Xen systems, it would probably make more
> sense to have an "architecture" field in class VM. Then we could add a
> "compatible architectures" field to class host_cpu, i.e. a list of
> architectures that CPU can support. For an x86-64 CPU, that would be
> "x86, x86-64". That way, when looking for a system on which to start an
> x86 VM, the user could be presented with a list of all x86 and x86-64
> systems being managed.
> Each architecture needs its own "optional feature" values, but there's
> no sense in putting them all into the same namespace. For example,
> PowerPC would have an FPU bit, but also Altivec, and of course almost
> none of the x86 bits listed would be relevant for us, so it's a waste of
> bits. I'm sure IA64 has the same situation.
> Of course, there are already a lot of bits in this list; it's
> conceivable we could run out of bits in an int/long/whatever in the
> future. Would it be better to communicate required features as strings
> instead of an enum? Or am I misunderstanding, and the "enums" in this
> document do not mean the same as "enum" in C, instead meaning "list of
> acceptable string values"?
>
> Those are the first issues that stand out for me. In general I'd like to
> remind everybody that computer networks are not homogeneous, so let's
> not box ourselves into a corner in the future by designing an
> x86-specific API today... :)

The constants in cpu_feature are taken from Linux's
include/asm-i386/cpufeature.h. It's obviously x86-specific at the moment, and
if that's a problem, then we can work on it.

If there are fields that only make sense on x86, I don't have a problem with
that -- they can just be ignored on other, more sensible, platforms, as long
as we've got the capacity to manage x86's idiosyncrasies. On the other hand,
if the problem needs to be solved on other arches too, then we should put
together a platform-agnostic API.

I can see the appeal of using opaque bitstrings, but I'm worried that these
will be unmanagable by non-specialist users. We can't very well say to people
"look up the CPU features for your particular platforms, AND them together,
and then type the resultant bitmask here". We need to be able to provide a
meaningful mapping at the API level, which is why it would be nice to use an
enum. Perhaps a list of strings would be the best compromise -- we could
define the strings that make sense on any particular platform, and the API
would remain flexible for the future.

Ewan.

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Sat, Jun 24, 2006 at 09:26:10PM +0100, Ewan Mellor wrote:
> On Thu, Jun 22, 2006 at 12:51:42PM -0500, Hollis Blanchard wrote:
>
> > On Thu, 2006-06-22 at 18:01 +0100, Ewan Mellor wrote:
> > >
> > > Like the OVA spec, this document is for narrow circulation, and once
> > > we are getting happy with it, we'll circulate it more widely. I'd
> > > like to be able to circulate this before OLS, as that will be a good
> > > chance to discuss it with people face-to-face once they've had a
> > > chance to read it.
> >
> > I'm confused. You said this is an open mailing list, but this document
> > says "DO NOT CIRCULATE" on the title page?
>
> I would like to keep discussion and circulation of this document on this list,
> for now. I'm happy for other people to join this list, and to see the
> document, but we've already had problems with people seeing this work out of
> context and getting confused. I don't think that the document has enough in
> the way of explanation to stand on its own just yet, so I think that the best
> way to ensure that everyone gets the right context is to keep the discussion
> here on this list, just for a short while.
>
> Once we're happy that the document can stand on its own feet, in good time
> before OLS, then we can relax the circulation restrictions, change the
> copyright statement, and so on.

First, this statement is totally incompatible with Xen being an open source
project. You can not ask members of an open source project to read & review
documents with restrictive distribution clauses & threatening legalese on
them.

Second, the statement here is completely contradictory to the one posted
by your collague Richard Sharp[1]

"The document is free to circulate and entirely open. The "DO NOT
CIRCULATE" was boiler-plate text that should not have been included
on this particular titlepage. :)"

Given such confusion & contradictory statements, I request that the PDF document
be re-sent to this list with the restrictive text & legalese removed, thus
clarifying that the document is indeed open. I've have no issue with your desire
to keep discussion & circulation soley on tihs list for now, however, there is
no need for threatening legalese to achieve this - a simple polite request to
not distribute broadly when posting the document is sufficient.


Regards,
Dan.
[1] http://lists.xensource.com/archives/html/xen-api/2006-06/msg00012.html
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Sat, Jun 24, 2006 at 09:24:40AM +0100, Ian Pratt wrote:
> > Is it realistic to use XML-RPC for tasks such as resource monitoring?
> > Seems like there should be an optional, lower-latency, lighter-weight
> > mechanism to retrieve things such as host_cpu.utilisation,
> > VBD.IO_bandwidth/incoming_kbs, etc.
>
> The whole stats reporting interface certainly needs more thought.
>
> However, I think it definitely should still use the xml-rpc interface,
> we just need to 'cook' the raw data into something more directly useful
> within xend so that there's no need to do high-rate polling on the API.
> This might involve using something like rrdtool to generate averages
> over different time scales etc.

Cooking the data before making it available to applications is not very
desirable, because it is making an assumption that the way XenD cooks
it is compatible with the format in which monitoring applications need
the data for processing / analysis. It would be like saying you can only
access Linux kernel performance stats from the averaged SAR records - it
is fine if all you want to do with the data is render graphs, or perform
long term trend analysis, however, if you want to analyse the 'live' data
to do actively monitor system performance you really need access to the
raw data over a low-latency, high performance channel.

It may be that this is impossible over a protocol like XML-RPC, in which
case we need to consider an alternate channel enabling local applications
to bypass XML-RPC and collect the raw performance / utilization statitics
without incurring a performance hit. Perhaps this alternate channel is
simply to make direct HyperCalls, which is something libvirt already does
when running as root - calling virDomainGetInfo to collect a domain's
CPU utilization is about 350x faster when dones as a hypercall, compared
to using HTTP/SEXPR[1]

Regards,
Dan.

[1] https://www.redhat.com/archives/libvir-list/2006-April/msg00044.html
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Fri, Jun 23, 2006 at 05:48:08PM -0600, Jim Fehlig wrote:
> Ewan Mellor wrote:
> >In particular, we need to get this API into a state so that it is the right
> >API for Xen-CIM. We need to think whether libvirt is ready to rely upon
> >this
> >API too. That would mean moving the hypercalls and Xenstore reads and
> >writes
> >out of libvirt, and pushing it up the stack alongside the other client-side
> >bindings.
> >
>
> Is it realistic to use XML-RPC for tasks such as resource monitoring?
> Seems like there should be an optional, lower-latency, lighter-weight
> mechanism to retrieve things such as host_cpu.utilisation,
> VBD.IO_bandwidth/incoming_kbs, etc. (Makes me thing of Uncertainty
> Principle or Observer Effect, i.e. we crank up cpu utilization by asking
> for cpu utilization :-)). I guess the hypercalls and Xenstore
> reads/writes mentioned above would still exist but not be part of the
> "guarantees" for Xen.

Even if there were a optional high peformance channel for these stats,
it would still be desirable to adjust the APIs to make it possible to
retrieve all the statistics associated with a guest (CPU, network,
disk utilization, etc) in a single XML-RPC call - with the current API
you'd have to make 3 separate calls - one per guest device you want
stats from which compounds the performance hit.

> Many of the RPCs return void, e.g. VM.start(), VM.pause(), etc. How are
> failures indicated? Via the Status and ErrorDescription elements of
> return value struct?

I would like to see an explicit set of error/exception conditions documented
against each API. When writing apps against the API it is important to know
what failure scenarios one has to be aware of. I realize some of the failures
scenarios won't become clear until the server side implementation is actually
written, but it ought to be possible to identify an initial set of errors
for each API, and add others later as they become known. If the errors can
be enumerated then it ought to be possible to associate an explicit "exception"
name or number against each, to allow them to be handled without resorting
to string comparisons on the description field.

Moving on to a different topic, I've a couple of questions around the topic
of authentication:

* Section 1.4.2 states that all clients must login & initiate a session
before making any of the other XML-RPC calls. Is there any provision
made for anonymous access. For an application which merely wants to read
information about guest VM state it would preferrable to have access
without needing to prompt the user for a login/password.

Could the requirement to always login be relaxed, and instead annotate
each API to indicate whether anonymous (read-only) access is allowed

* What is the motivation for implementing an explicit login_with_password
method rather than utilizing the existing HTTP authentication protocols ?
The proposed login API utilizing a simple username/password pair is quite
limiting, preventing the use of any of the more advanced authentication
protocols such as challenge/response, public / private key, kerberos
ticket passing.

The latter would be particuarly important if the apps using this API want
to integrate with any kind of single sign on system. Perhaps it would be
possible to define a more advanced login process which could be backed by
something like SASL

http://www.ietf.org/rfc/rfc2222.txt
http://asg.web.cmu.edu/sasl/

* What is the lifetime of sessions ? I assume they persist across distinct
HTTP connections for the case where the client isn't using keep-alive.
Will there, however, be any form of timeout, after which the client will
implicitly be logged out.


Minor question on the 'bios_boot_option' enumeration - rather than hardcoding
a list of floppy/hd/cdrom, could the bootable device field instead simply be
a UUID refering to one of the VDB devices defined for the domain ?

Finally, a higher level question - the API is still basically unidirectional,
poll based model. By this I mean, if I want to detect changes in a guest VMs
lifecycle state (eg, from running -> paused), then I have no choice but to
poll the 'power_state' field on every VM i'm interested in. If the app using
the API is a GUI app, then this polling is going to have to be done pretty
frequently (once per second or more) to provide an acceptable refresh in the
UI. I'm concerned that polling so frequently over an HTTP / XML-RPC protocol
is going to impose a very significant performance hit on the host machine.
It would be very desriable if there was a formal API for getting asynchronous
callbacks notifying the client of changes in a VM's lifecycle state. I know
this is not really possible with XML-RPC, but I wanted to bring it up as a
use-case none-the-less in case there are alternate ways to provide such a
capability.

The same issue is true of the proposed mechanism for asynchronous invocation
invocation of APIs - it also requires the client to make requests polling
for completion of the API which is not really buying any performance benefit.
Unless the client actually wanted the 'estimated time of completion' data,
they might as well just send a regular synchronous request & use select()
or poll() system calls on the HTTP socket to do a non-blocking wait for
completion client side.

Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Sun, Jun 25, 2006 at 04:49:03PM +0100, Daniel P. Berrange wrote:

> On Fri, Jun 23, 2006 at 05:48:08PM -0600, Jim Fehlig wrote:
> > Ewan Mellor wrote:
> > >In particular, we need to get this API into a state so that it is the right
> > >API for Xen-CIM. We need to think whether libvirt is ready to rely upon
> > >this
> > >API too. That would mean moving the hypercalls and Xenstore reads and
> > >writes
> > >out of libvirt, and pushing it up the stack alongside the other client-side
> > >bindings.
> > >
> >
> > Is it realistic to use XML-RPC for tasks such as resource monitoring?
> > Seems like there should be an optional, lower-latency, lighter-weight
> > mechanism to retrieve things such as host_cpu.utilisation,
> > VBD.IO_bandwidth/incoming_kbs, etc. (Makes me thing of Uncertainty
> > Principle or Observer Effect, i.e. we crank up cpu utilization by asking
> > for cpu utilization :-)). I guess the hypercalls and Xenstore
> > reads/writes mentioned above would still exist but not be part of the
> > "guarantees" for Xen.
>
> Even if there were a optional high peformance channel for these stats,
> it would still be desirable to adjust the APIs to make it possible to
> retrieve all the statistics associated with a guest (CPU, network,
> disk utilization, etc) in a single XML-RPC call - with the current API
> you'd have to make 3 separate calls - one per guest device you want
> stats from which compounds the performance hit.

That's a good idea -- we'll certainly want to include that.

> > Many of the RPCs return void, e.g. VM.start(), VM.pause(), etc. How are
> > failures indicated? Via the Status and ErrorDescription elements of
> > return value struct?

Yes, that's right. Status may be "Failure", implying the presence of the
ErrorDescription array, regardless of the fact that the function returns
void. It's effectively an exception "thrown" over the wire.

> I would like to see an explicit set of error/exception conditions documented
> against each API. When writing apps against the API it is important to know
> what failure scenarios one has to be aware of. I realize some of the failures
> scenarios won't become clear until the server side implementation is actually
> written, but it ought to be possible to identify an initial set of errors
> for each API, and add others later as they become known. If the errors can
> be enumerated then it ought to be possible to associate an explicit "exception"
> name or number against each, to allow them to be handled without resorting
> to string comparisons on the description field.

Certainly, I'd like to see a list of expected failures. We need to be able to
standardise that list so that people can internationalise their clients, in
particular.

> Moving on to a different topic, I've a couple of questions around the topic
> of authentication:
>
> * Section 1.4.2 states that all clients must login & initiate a session
> before making any of the other XML-RPC calls. Is there any provision
> made for anonymous access. For an application which merely wants to read
> information about guest VM state it would preferrable to have access
> without needing to prompt the user for a login/password.
>
> Could the requirement to always login be relaxed, and instead annotate
> each API to indicate whether anonymous (read-only) access is allowed
>
> * What is the motivation for implementing an explicit login_with_password
> method rather than utilizing the existing HTTP authentication protocols ?

We discussed this on xen-devel last week -- HTTP auth doesn't seem to be
widely supported, so we didn't want to rely upon it. Also, this way we can
use the XML-RPC over something other than HTTP (such as a raw unix domain
socket).

> The proposed login API utilizing a simple username/password pair is quite
> limiting, preventing the use of any of the more advanced authentication
> protocols such as challenge/response, public / private key, kerberos
> ticket passing.
>
> The latter would be particuarly important if the apps using this API want
> to integrate with any kind of single sign on system. Perhaps it would be
> possible to define a more advanced login process which could be backed by
> something like SASL
>
> http://www.ietf.org/rfc/rfc2222.txt
> http://asg.web.cmu.edu/sasl/

What would be involved in making this work? The username / password is
already a step up for Xen -- how complicated is SASL or similar?

> * What is the lifetime of sessions ? I assume they persist across distinct
> HTTP connections for the case where the client isn't using keep-alive.
> Will there, however, be any form of timeout, after which the client will
> implicitly be logged out.

No timeout is specified at the moment -- do we want one? This could be a
configuration option to Xend with little effort.

> Minor question on the 'bios_boot_option' enumeration - rather than hardcoding
> a list of floppy/hd/cdrom, could the bootable device field instead simply be
> a UUID refering to one of the VDB devices defined for the domain ?

That's a nice idea. We'll certainly roll that into the API.

Cheers,

Ewan.

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Fri, Jun 23, 2006 at 09:42:05AM -0400, Mike D. Day wrote:

> Mike D. Day wrote:
> >Ewan Mellor wrote:
> >>Attached is a draft Xen Management API. This document presents our
>
> One item I forgot in the previous email:
>
> The NIC field of the Network class should be a list (Set) so that we can
> signify NIC teaming. (Combining physical NICs in a single host interface
> to achieve greater bandwidth).

Is that better than specifying the NIC as "bond0" or whatever and relying on
out-of-band mechanisms to get the device bonding set up? I'm worried that
simply specifying the NIC list is not going far enough, so actually what we
need to do is a) throw our hands up and say that we don't handle it or b)
put in an awful lot of network-related modelling.

I don't really know how much effort and complexity is involved in b). Perhaps
someone who understands networking better than me can tell us how much
complexity you need before things start to be useful.

Ewan.

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api