Mailing List Archive

Re: Xen as a kernel module
I'm obviously partisan, but I think that would largely defeat the
privilege de-coupling that they're trying to achieve. Not to mention
nullifying the notion of a driver domain.

-Kip



On Tue, 25 Jan 2005, Jacob Gorm Hansen wrote:

> hi,
>
> with Xen increasingly depending on Linux for bootstrap, drivers, packet
> filtering etc., would it make sense to have the option of compiling Xen
> as a Linux kernel module, like in VMWare or coLinux?
>
> It seems this would give similar performance to Xen 1.2, while retaining
> most of the benefits of the NGIO model (i.e. not having to port
> drivers). The only downside would be the lack of driver isolation, but
> most people would be willing to live with that is my guess (plus as long
> as there is no IO-MMU a bad driver is still able to take down the
> complete system anyhow).
>
> I imagine this could be done in a way that would also work under other
> host-OSes, like *BSD or Windows.
>
> Any comments?
>
> Jacob
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel
>


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Re: Xen as a kernel module [ In reply to ]
Kip Macy wrote:
> I'm obviously partisan, but I think that would largely defeat the
> privilege de-coupling that they're trying to achieve. Not to mention
> nullifying the notion of a driver domain.

Is anyone actually using driver domains, other than dom0? It would be
really cool if hardware vendors started shipping drivers in Xen-VM
format, but until that happens I do not see much motivation for having
multiple full Linuxes around just to achieve a limited amount of isolation.

It is no secret that I much preferred the old and simpler model, but I
understand that noone likes having to port drivers, and I guess this is
one reason for the current state of things. With Xen inside the host OS
kernel, you would get the performance of the old model, and the ease of
development of the new model, plus potentially the option of running Xen
on millions of Windows-desktops (I am sure the Silicon Valley VCs would
love to hear that ;-)).

With all the OSes currently ported to Xen, the Xen hypercall interface
could become a de-facto standard for paravirtualized OSes, especially if
Xen were able to run on top of Windows as well as on top of Linux.

Jacob


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
RE: Xen as a kernel module [ In reply to ]
> -----Original Message-----
> From: xen-devel-admin@lists.sourceforge.net [mailto:xen-devel-
> admin@lists.sourceforge.net] On Behalf Of Jacob Gorm Hansen
> Sent: 26 January 2005 02:00
> To: Kip Macy
> Cc: Xen-devel@lists.sourceforge.net
> Subject: Re: [Xen-devel] Xen as a kernel module
>
> Kip Macy wrote:
> > I'm obviously partisan, but I think that would largely defeat the
> > privilege de-coupling that they're trying to achieve. Not to mention
> > nullifying the notion of a driver domain.
>
> Is anyone actually using driver domains, other than dom0? It would be
> really cool if hardware vendors started shipping drivers in Xen-VM
> format, but until that happens I do not see much motivation for having
> multiple full Linuxes around just to achieve a limited amount of
> isolation.

I don't think many people currently use separate driver domains and most
people run all their device drivers in dom0 and export them from there.

However, I think it would be foolish to remove the ability to run
separate driver domains from Xen as it doesn't seem to add significant
overhead (over running them in Dom0) and it has the potential of being
extremely useful. You already mentioned the shipping of device drivers,
there might also some interesting security applications. We can
currently already provide some degree of isolation against device driver
bugs and for the DMA issue there are a number of known and existing
solutions (see eg the L4 OSDI device driver paper). The grant-table
design (unfortunately not fully implemented yet) with some form of
IO-MMU should provide the rest of the isolation.

The interfaces exported by Xen (for PCI config space, IO register
access, and interrupts) should be generic enough to support other OSes
as driver domains and the virtual device interfaces should be easily
implementable in other OSes too. Again, I think it would be foolish to
limit ourselves to Linux in this regard as your original post seems to
suggest. I think it would be great to see other OSes being used as
driver domains.

> It is no secret that I much preferred the old and simpler model, but I
> understand that noone likes having to port drivers, and I guess this
is
> one reason for the current state of things. With Xen inside the host
OS
> kernel, you would get the performance of the old model, and the ease
of
> development of the new model, plus potentially the option of running
Xen
> on millions of Windows-desktops (I am sure the Silicon Valley VCs
would
> love to hear that ;-)).
>
> With all the OSes currently ported to Xen, the Xen hypercall interface
> could become a de-facto standard for paravirtualized OSes, especially
if
> Xen were able to run on top of Windows as well as on top of Linux.

I think it would be harder to come up with OS agnostic/generic
interfaces for Xen to be used inside a host OS (essentially sliding Xen
underneath a OS) than with generic abstractions which a OS can
use...plus there is the privilege decoupling issue Kip already raised.

The current plan to move some of the platform init code out of Xen into
a VM will certainly make this slightly more complicated (eg having the
IOAPIC initialized by a VM and then handing control back to Xen) but I
hope we will come up with a generic interface for Xen to deal with
possibly different OSes to perform this task. At least that's something
I am advocating.

Rolf



> Jacob
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive
Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Re: Xen as a kernel module [ In reply to ]
Neugebauer, Rolf wrote:

> However, I think it would be foolish to remove the ability to run
> separate driver domains from Xen as it doesn't seem to add significant
> overhead (over running them in Dom0) and it has the potential of being
> extremely useful. You already mentioned the shipping of device drivers,
> there might also some interesting security applications. We can
> currently already provide some degree of isolation against device driver
> bugs and for the DMA issue there are a number of known and existing
> solutions (see eg the L4 OSDI device driver paper). The grant-table
> design (unfortunately not fully implemented yet) with some form of
> IO-MMU should provide the rest of the isolation.

First of all, my suggestion was to have this as a configuration option,
or even a separate implementation, providing just the ability to host
domUs. I am sure this is doable, as it seems to be what coLinux is doing
with some success. I was not suggestion throwing away existing
functionality, but to optimize for the common case, i.e. a single Linux
providing all drivers.

Secondly, there have been repeated reports on this list of people having
problems with lower performance in domU than in dom0, perhaps due to
cheap hardware, perhaps just due to misconfiguration, and the figures on
the Xen website have not been updated to reflect what the actual
situation is, so I guess nobody knows what the overhead will look like
for a specific type of application. I would worry about MPI-style jobs,
where you need both low latency and high bandwidth networking, and where
you are likely to fully utilize the TLBs as well. I cannot see how
performance does not get hurt in this situation, when Xen needs to flush
the TLB for every interrupt, or alternatively needs to bundle
interrupts, thus increasing latency.

Jacob


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
RE: Xen as a kernel module [ In reply to ]
well, I vote for Xen 2.0, i.e. keep dom0 and xen separate. My laptop runs
xen as the default and I don't see a huge hit performance wise.

Plus, I get to dream that someday plan9 will be dom0.

ron


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Re: Xen as a kernel module [ In reply to ]
> I would worry about MPI-style jobs,
> where you need both low latency and high bandwidth networking, and where
> you are likely to fully utilize the TLBs as well. I cannot see how
> performance does not get hurt in this situation, when Xen needs to flush
> the TLB for every interrupt, or alternatively needs to bundle
> interrupts, thus increasing latency.

This kind of workload needs measuring to find out just how bad things
are -- it's interesting if only because it will certainly represent a
worst case scenario for us. Ultimately this should be fixed with
smarter NICs. I am optimistic suitable support may end up in high-end
NICs: manufacturers are always looking at how to differentiate their
product, and we have a few contacts we would like to follow up when we
have more time. :-)

-- Keir


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Re: Xen as a kernel module [ In reply to ]
> for a specific type of application. I would worry about MPI-style jobs,
> where you need both low latency and high bandwidth networking, and where

Won't many people running this sort of workload be using specialised network
hardware anyhow? A modern cluster interconnect (Infiniband, for instance)
will likely support direct IO to user level applications - this could still
be done under Xen.

Cheers,
Mark

> you are likely to fully utilize the TLBs as well. I cannot see how
> performance does not get hurt in this situation, when Xen needs to flush
> the TLB for every interrupt, or alternatively needs to bundle
> interrupts, thus increasing latency.
>
> Jacob
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
> Tool for open source databases. Create drag-&-drop reports. Save time
> by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
> Download a FREE copy at http://www.intelliview.com/go/osdn_nl
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
Re: Xen as a kernel module [ In reply to ]
On Thu, 27 Jan 2005, Mark Williamson wrote:

> Won't many people running this sort of workload be using specialised
> network hardware anyhow? A modern cluster interconnect (Infiniband, for
> instance) will likely support direct IO to user level applications -
> this could still be done under Xen.

that's been our assumption.

ron


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel