Mailing List Archive

On Tue, Dec 19, 2006 at 09:54:10AM -0800, jdsw wrote:
> Do you guys have any suggestion ?
> (I did not get satisfactory answers to each of the questions on dev forum)
>
> In addition is there a plan for a streamlined access to things happening
> in Domu information through Dom0. For example: Top process information in a
> DomU available through Dom0 ?

The Xen-API work is really about management of Dom0 and DomU VMs as opaque
entities viewed from Dom0. Collecting OS specific details about work going on
inside the DomU (eg processes running, detailed breakdown of CPU & memory
usage) is best handled by a dedicated monitoring API. It has very different
requirements of that of Xen-API, in particular from a performance / overhead
POV. There are a wide variety of existing OS monitoring tools which can be
deployed inside DomUs, Big-Sister, Nagios, Ganglia, to name but three such
tools.

Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

I understand that mechanism for collecting data might be different and specific. But the transport and API to get that information can be standardised. For example, a way to push some structured data on xenbus and make it accessible through Xend api.

Also, I am not sure, if the email that I fwd got other questions about, how to get certain basic metrics through Xend instead of using combination of Xend API, Memory Maps, parsing files for VBD and n/w information.
/Jd

"Daniel P. Berrange" <berrange@redhat.com> wrote: On Tue, Dec 19, 2006 at 09:54:10AM -0800, jdsw wrote:
> Do you guys have any suggestion ?
> (I did not get satisfactory answers to each of the questions on dev forum)
>
> In addition is there a plan for a streamlined access to things happening
> in Domu information through Dom0. For example: Top process information in a
> DomU available through Dom0 ?

The Xen-API work is really about management of Dom0 and DomU VMs as opaque
entities viewed from Dom0. Collecting OS specific details about work going on
inside the DomU (eg processes running, detailed breakdown of CPU & memory
usage) is best handled by a dedicated monitoring API. It has very different
requirements of that of Xen-API, in particular from a performance / overhead
POV. There are a wide variety of existing OS monitoring tools which can be
deployed inside DomUs, Big-Sister, Nagios, Ganglia, to name but three such
tools.

Regards,
Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|


__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

On Tue, Dec 19, 2006 at 09:54:10AM -0800, jdsw wrote:

> Do you guys have any suggestion ?
> (I did not get satisfactory answers to each of the questions on dev forum)

Sorry I didn't reply before -- I thought that you were happy with the
answer you got from Tim Wood re XenMon.

> In addition is there a plan for a streamlined access to things happening
> in Domu information through Dom0. For example: Top process information in
> a DomU available through Dom0 ?

There isn't a plan to work on a protocol for transport of arbitrary stats out of
domUs at the moment. There are things that we can collect in dom0 --
CPU and I/O load for instance -- that we intend to package up into the
Xen-API, to enable basic load monitoring. This is pretty near the top
of my list for the Xen-API bindings, so if those stats are enough for
you, then that will be the best way to get them. By the end of this
year or early next, you should be able to get these basic stats with a
15 line Python script.

More advanced statistics from in the guest, are best served with a
different tool, I think. Everyone wants something different with this
regard, and there are plenty of good tools out there (Daniel mentioned a
few), that the best thing to do is run Nagios or whatever inside your
guest, and ship the collated stats out over the network, just as you
would with a native machine.

If what you're looking for is the One True Way to monitor your cluster,
whether you're monitoring dom0 or a guest, or the applications running
in the guest, then what you want is DMTF's CIM. This is the long-term
answer to hetrogeneous monitoring and management, and we have a project
running at the moment (driven by IBM and Novell, with help from others)
to develop a CIM provider for Xen.

The answer to your question "why isn't there a standard transport and
format for all this information?" is that there is (or will be), and it is CIM.

Cheers,

Ewan.

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

On Tue, Dec 19, 2006 at 11:08:42AM -0800, jdsw wrote:
> I understand that mechanism for collecting data might be different and
> specific. But the transport and API to get that information can be
> standardised. For example, a way to push some structured data on xenbus
> and make it accessible through Xend api.

Any monitoring standard though, is not going to standardize on a Xen-API
transport, because it'll obviously want to be equally appicable to non-Xen
environments. I imagine that Xen-API will be used as one data source for
collecting such monitoring data, but the monitoring data transport & arch
will be at a layer above - it would merely have a Xen-API plugin for getting
HV / VBD / VIF stats.

Wrt to pushing stats from DomU back to Dom0 via XenBus - there's not really
any clear benefit to doing that for general perf monitoring. Monitoring systems
will typically send data off-host to a remote / centralized collection point,
so sending it from DomU -> Dom0 -> <monitor-host>, does seem to really
give any benefit over just having DomU -> <monitor-host>. It could even be
detrimental when you think of the complication of migration which means the
undering Dom0 you're on can change at any time.

> Also, I am not sure, if the email that I fwd got other questions about,
> how to get certain basic metrics through Xend instead of using combination
> of Xend API, Memory Maps, parsing files for VBD and n/w information.

The Xen-API is intended to give access to stats on CPU timeslice allocated
to VMs, memory allocation, VBD & VIF I/O stats - basically anything that is
related to the Dom0 management side of VMs. So for Xen specific stats the
Xen-API should be the primary access point, while OS specific stats the Dom0
or DomU OS can be used as with baremetal. Xen-API isn't intending to provide
general stats for Dom0 OS, since they can already be obtained with a variety
of other tools.

> "Daniel P. Berrange" <berrange@redhat.com> wrote: On Tue, Dec 19, 2006 at 09:54:10AM -0800, jdsw wrote:
> > Do you guys have any suggestion ?
> > (I did not get satisfactory answers to each of the questions on dev forum)
> >
> > In addition is there a plan for a streamlined access to things happening
> > in Domu information through Dom0. For example: Top process information in a
> > DomU available through Dom0 ?
>
> The Xen-API work is really about management of Dom0 and DomU VMs as opaque
> entities viewed from Dom0. Collecting OS specific details about work going on
> inside the DomU (eg processes running, detailed breakdown of CPU & memory
> usage) is best handled by a dedicated monitoring API. It has very different
> requirements of that of Xen-API, in particular from a performance / overhead
> POV. There are a wide variety of existing OS monitoring tools which can be
> deployed inside DomUs, Big-Sister, Nagios, Ganglia, to name but three such
> tools.


Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496 -=|
|=- Perl modules: http://search.cpan.org/~danberr/ -=|
|=- Projects: http://freshmeat.net/~danielpb/ -=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 -=|

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api

Thanks to Evan and Daniel both for sharing their views and clarifying the scope of xen-api etc.

/Jd

Ewan Mellor <ewan@xensource.com> wrote: On Tue, Dec 19, 2006 at 09:54:10AM -0800, jdsw wrote:

> Do you guys have any suggestion ?
> (I did not get satisfactory answers to each of the questions on dev forum)

Sorry I didn't reply before -- I thought that you were happy with the
answer you got from Tim Wood re XenMon.

> In addition is there a plan for a streamlined access to things happening
> in Domu information through Dom0. For example: Top process information in
> a DomU available through Dom0 ?

There isn't a plan to work on a protocol for transport of arbitrary stats out of
domUs at the moment. There are things that we can collect in dom0 --
CPU and I/O load for instance -- that we intend to package up into the
Xen-API, to enable basic load monitoring. This is pretty near the top
of my list for the Xen-API bindings, so if those stats are enough for
you, then that will be the best way to get them. By the end of this
year or early next, you should be able to get these basic stats with a
15 line Python script.

More advanced statistics from in the guest, are best served with a
different tool, I think. Everyone wants something different with this
regard, and there are plenty of good tools out there (Daniel mentioned a
few), that the best thing to do is run Nagios or whatever inside your
guest, and ship the collated stats out over the network, just as you
would with a native machine.

If what you're looking for is the One True Way to monitor your cluster,
whether you're monitoring dom0 or a guest, or the applications running
in the guest, then what you want is DMTF's CIM. This is the long-term
answer to hetrogeneous monitoring and management, and we have a project
running at the moment (driven by IBM and Novell, with help from others)
to develop a CIM provider for Xen.

The answer to your question "why isn't there a standard transport and
format for all this information?" is that there is (or will be), and it is CIM.

Cheers,

Ewan.


__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

I agree with Ewan, Dan, et al. Xen/Xen-API/xend/xenbus should *not* attempt
to be the "One True Ring" management infrastructure for all host,
virtualization and guest management, but rather it should focus
specifically on managing what it knows and controls best - allocation of
certain host resources to virtualized guests. There is still going to be
*lots* of other important system management tasks, including performance
and resource utilization monitoring, that are outside of the scope of Xen
itself - eg inband guest OS memory usage - and some that it should get
involved with - eg exposing metric info from the front/back vbd & vif
drivers.

I dont think going down the path of try to exploiting xenbus and/or Xen-API
as a general purpose conduit for all guest/host mgmt is a good idea. As
stated, there are lots of other existing infrastructure/tooling for
obtaining this info. In terms of *exposing* this data to consumers in a
standardized form, when this is desired, we have CIM (note CIM does not
dictate how/where you *get* the raw data, just how to present it).

- Gareth





"Daniel P.
Berrange"
<berrange@redhat. To
Sent by: cc
xen-api-bounces@l xen-api@lists.xensource.com
ists.xensource.co Subject
m Re: [Xen-API] Fwd: Getting VM stats


12/20/06 06:12 AM


Please respond to
"Daniel P.
Berrange"
<berrange@redhat.






On Tue, Dec 19, 2006 at 11:08:42AM -0800, jdsw wrote:
> I understand that mechanism for collecting data might be different and
> specific. But the transport and API to get that information can be
> standardised. For example, a way to push some structured data on xenbus
> and make it accessible through Xend api.

Any monitoring standard though, is not going to standardize on a Xen-API
transport, because it'll obviously want to be equally appicable to non-Xen
environments. I imagine that Xen-API will be used as one data source for
collecting such monitoring data, but the monitoring data transport & arch
will be at a layer above - it would merely have a Xen-API plugin for
getting
HV / VBD / VIF stats.

Wrt to pushing stats from DomU back to Dom0 via XenBus - there's not really
any clear benefit to doing that for general perf monitoring. Monitoring
systems
will typically send data off-host to a remote / centralized collection
point,
so sending it from DomU -> Dom0 -> <monitor-host>, does seem to really
give any benefit over just having DomU -> <monitor-host>. It could even be
detrimental when you think of the complication of migration which means the
undering Dom0 you're on can change at any time.

> Also, I am not sure, if the email that I fwd got other questions about,
> how to get certain basic metrics through Xend instead of using
combination
> of Xend API, Memory Maps, parsing files for VBD and n/w information.

The Xen-API is intended to give access to stats on CPU timeslice allocated
to VMs, memory allocation, VBD & VIF I/O stats - basically anything that
is
related to the Dom0 management side of VMs. So for Xen specific stats the
Xen-API should be the primary access point, while OS specific stats the
Dom0
or DomU OS can be used as with baremetal. Xen-API isn't intending to
provide
general stats for Dom0 OS, since they can already be obtained with a
variety
of other tools.

> "Daniel P. Berrange" <berrange@redhat.com> wrote: On Tue, Dec 19, 2006 at
09:54:10AM -0800, jdsw wrote:
> > Do you guys have any suggestion ?
> > (I did not get satisfactory answers to each of the questions on dev
forum)
> >
> > In addition is there a plan for a streamlined access to things
happening
> > in Domu information through Dom0. For example: Top process information
in a
> > DomU available through Dom0 ?
>
> The Xen-API work is really about management of Dom0 and DomU VMs as
opaque
> entities viewed from Dom0. Collecting OS specific details about work
going on
> inside the DomU (eg processes running, detailed breakdown of CPU & memory
> usage) is best handled by a dedicated monitoring API. It has very
different
> requirements of that of Xen-API, in particular from a performance /
overhead
> POV. There are a wide variety of existing OS monitoring tools which can
be
> deployed inside DomUs, Big-Sister, Nagios, Ganglia, to name but three
such
> tools.


Dan.
--
|=- Red Hat, Engineering, Emerging Technologies, Boston. +1 978 392 2496
-=|
|=- Perl modules: http://search.cpan.org/~danberr/
-=|
|=- Projects: http://freshmeat.net/~danielpb/
-=|
|=- GnuPG: 7D3B9505 F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505
-=|

_______________________________________________
xen-api mailing list
xen-api@lists.xensource.com
http://lists.xensource.com/cgi-bin/mailman/listinfo/xen-api