Mailing List Archive

On Wed, 2005-01-26 at 12:01, Anthony Liguori wrote:
> On Wed, 2005-01-26 at 13:11, Andrew Warfield wrote:
> > > In reality, what's the different between a domain that's suspended to
> > > disk and a clone domain that just hasn't been started yet.
> > >
> > > Does the current persistent store architecture attempt to solve this
> > > problem? Does it attempt to solve both problems?
> >
> > No. There has been no discussion of storing suspended domains. For
> > the moment, i think it's probably best to design the store to contain
> > runtime state for the currently executing set of VMs on the physical
> > host. So records in the store are persistent across daemon restarts,
> > and dom0 restarts.
>
> Excellent! Then I have no worries at all about it.
>
> You mentioned the current design looks a lot like a file system, any
> thoughts on actually using the file system? Either within the normal
> file system (within something like /var/xen/<domid>) or as a pid-like
> /proc entry (like /proc/xen/<domid>).


Do you really want the store in Kernel Space? Wouldn't it be better if
it were stored in User Space? Could you restart Dom0 if it's in Dom0?
Wouldn't it be better to store out on disk that can be recovered? You
could use the store for more than the above, domain configuration could
be stored and accessed there.

Thanks,
Dan



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 14:01 -0600, Anthony Liguori wrote:
> On Wed, 2005-01-26 at 13:11, Andrew Warfield wrote:
> > > In reality, what's the different between a domain that's suspended to
> > > disk and a clone domain that just hasn't been started yet.
> > >
> > > Does the current persistent store architecture attempt to solve this
> > > problem? Does it attempt to solve both problems?
> >
> > No. There has been no discussion of storing suspended domains. For
> > the moment, i think it's probably best to design the store to contain
> > runtime state for the currently executing set of VMs on the physical
> > host. So records in the store are persistent across daemon restarts,
> > and dom0 restarts.
>
> Excellent! Then I have no worries at all about it.
>
> You mentioned the current design looks a lot like a file system, any
> thoughts on actually using the file system? Either within the normal
> file system (within something like /var/xen/<domid>) or as a pid-like
> /proc entry (like /proc/xen/<domid>).

Wouldn't want to do this in the kernel (i.e., /proc or /sysfs) as it
would not be persistent across dom0 restarts. In a regular filesystem
as {directories, subdirectories, files} would be reasonable. In either
case, persistent store would need to determine whether the entire
system (Xen) has restarted, dom0 has restarted, or just the xend has
restarted and clean up residual info appropriately.

While beyond the current focus, persistent store could feasibly
be used to hold domain definitions for non-existent domains and
suspended domains. One could envision adding a state field into
the domain configuration/definition. Valid states for current
capabilities would be {active, suspended, migrated, inactive}. On
the other hand, keeping persistent store info on only active domains
in this mechanism and moving info to higher level tools about predefined
domains, suspended domains, etc. would work. It's a tradeoff between
similar info being maintained in multiple places, or added complexity
in a single place.

Michael
>
> Regards,
--
Michael Hohnbaum 503 578 5486
hohnbaum@us.ibm.com t/l 775 5486



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 16:28 -0600, Anthony Liguori wrote:
> On Wed, 2005-01-26 at 15:21, Daniel Stekloff wrote:
> > On Wed, 2005-01-26 at 12:01, Anthony Liguori wrote:
> > > You mentioned the current design looks a lot like a file system, any
> > > thoughts on actually using the file system? Either within the normal
> > > file system (within something like /var/xen/<domid>) or as a pid-like
> > > /proc entry (like /proc/xen/<domid>).
> >
> >
> > Do you really want the store in Kernel Space? Wouldn't it be better if
> > it were stored in User Space? Could you restart Dom0 if it's in Dom0?
> > Wouldn't it be better to store out on disk that can be recovered? You
> > could use the store for more than the above, domain configuration could
> > be stored and accessed there.
>
> Yup. Automatic relaunching of domains wouldn't work if it was in kernel
> space.
>
> A downside of using disk also would be that you couldn't enforce any
> sort of schema.


Sorry, I think I misunderstood what you were stating. I like the idea of
storing it in a file system-like architecture that's easily traversed
and you can leverage existing file system security, calls, etc. I
thought for a minute that you wanted to implement it as a virtual file
system in Kernel Space.

My bad... sorry...

Dan



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 15:21, Daniel Stekloff wrote:
> On Wed, 2005-01-26 at 12:01, Anthony Liguori wrote:
> > You mentioned the current design looks a lot like a file system, any
> > thoughts on actually using the file system? Either within the normal
> > file system (within something like /var/xen/<domid>) or as a pid-like
> > /proc entry (like /proc/xen/<domid>).
>
>
> Do you really want the store in Kernel Space? Wouldn't it be better if
> it were stored in User Space? Could you restart Dom0 if it's in Dom0?
> Wouldn't it be better to store out on disk that can be recovered? You
> could use the store for more than the above, domain configuration could
> be stored and accessed there.

Yup. Automatic relaunching of domains wouldn't work if it was in kernel
space.

A downside of using disk also would be that you couldn't enforce any
sort of schema.

> Thanks,
> Dan
--
Anthony Liguori
Linux Technology Center (LTC) - IBM Austin
E-mail: aliguori@us.ibm.com
Phone: (512) 838-1208




-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 15:49, Michael Hohnbaum wrote:

> While beyond the current focus, persistent store could feasibly
> be used to hold domain definitions for non-existent domains and
> suspended domains. One could envision adding a state field into
> the domain configuration/definition. Valid states for current
> capabilities would be {active, suspended, migrated, inactive}. On

Yes, but the problem that occurs is uniquely identifying a domain. In
other words, what's the key used within the persistent store?

If it's domain id (which is what I assume it's going to be), you cannot
tag it as having an "inactive" state because there's nothing that
prevents a domain from being created with the same domain id.

Also, if you try to assign domains UUIDs or something, what do you do
for cloning/checkpointing? Do you assign a new UUID on a clone but not
on a checkpoint? Does assigning new UUIDs propagate to things like MAC
addresses or other things that are supposed to be unique?

There's a lot to be thought about. I think punting the problem (as Andy
suggests) is the right approach for now.

Regards,

--
Anthony Liguori
Linux Technology Center (LTC) - IBM Austin
E-mail: aliguori@us.ibm.com
Phone: (512) 838-1208




-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 06:33, Andrew Warfield wrote:
> Hi Michael,
>
> > Is it reasonable to expect that a port to another machine architecture,
> > assuming the libxc/xutil are provided, then xcs and higher-level tools
> > should just follow?
>
> This seems pretty reasonable to me. Most of what happens in xend
> right now is managing the state of the vms, much of the real mechanism
> (domain building, suspension, migration, etc.) is really inside the
> underlying libraries (xc, xutil). The only vaguely dependent thing i
> can think of that isn't masked by these libraries is the shared memory
> interface ot the control rings accessed through xcs. This isn't a
> very big deal though.
>
> > What form do you see this persistent store taking? Is it just for
> > currently present VM's, or can it be used also to pre-configure
> > VM's and/or have pre-defined VM's? I assume some sort of filesystem
> > backed entity is to be used for the persistent store, correct?
> > Format, content, etc.?
>
> We have had some discussion about this, and Mike Wray has written some
> design documentation on the new store plan. I think the principle
> goal of the store is to encapsulate all of the state relating to the
> currently running set of VMs in a single place, that has clear API for
> access. Our basic view of this at the moment is a hierarchical set of
> key-value pairs that looks remarkably like a file system. ;)


Could we see Mike Wray's design documentation for the new store?


> Having the store worked in properly should 1. make rebooting dom0 with
> active VMs unaffected more possible, 2. make adding new functionality
> (e.g. new split device types) more straightforward, and 3. make
> writing tools that need state information, and 4. scaling control up
> to a cluster all a lot more tractable.
>
> Including more long-lived stuff like config files seems like a
> potentially reasonable thing to do as well.
>
> > Yes. This is a good overview. More details especially in regards to
> > the persistent store, and the set of tools being built would add to
> > this discussion. Also, it is likely that I (and others) can provide
> > development and testing assistance if we understand what is being
> > developed and where our contributions can be used.
>
> Fantastic! We would be happy to have the help. We are still in the
> process of hashing out exactly what the new interfaces should look
> like and how to go about moving to a new set of tools, potentially for
> the 3.0 release. Probably the best way forward is for us to post
> design sketches to the list as they become available, and iteratively
> move towards bits of work that people can bite off. Any specific
> ideas that you guys want to throw in now would be more than welcome as
> well.


Do you have an outline of what the new set of tools will be? It's been
mentioned that xcs will make way for multiple management apps rather
than just xm/xend. I'm curious to know what you've outlined so far for
management apps.

Thanks,

Dan



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 16:57 -0600, Anthony Liguori wrote:
> On Wed, 2005-01-26 at 15:49, Michael Hohnbaum wrote:
>
> > While beyond the current focus, persistent store could feasibly
> > be used to hold domain definitions for non-existent domains and
> > suspended domains. One could envision adding a state field into
> > the domain configuration/definition. Valid states for current
> > capabilities would be {active, suspended, migrated, inactive}. On
>
> Yes, but the problem that occurs is uniquely identifying a domain. In
> other words, what's the key used within the persistent store?
>
> If it's domain id (which is what I assume it's going to be), you cannot
> tag it as having an "inactive" state because there's nothing that
> prevents a domain from being created with the same domain id.

Active domains would have a domain ID; inactive domains could be
labeled.
>
> Also, if you try to assign domains UUIDs or something, what do you do
> for cloning/checkpointing? Do you assign a new UUID on a clone but not
> on a checkpoint? Does assigning new UUIDs propagate to things like MAC
> addresses or other things that are supposed to be unique?

Now you are making my head hurt.
>
> There's a lot to be thought about. I think punting the problem (as Andy
> suggests) is the right approach for now.

Punting is reasonable. The immediate need is persistent store for
active domains. Other needs can be pushed up the management tool
stack to reside with the consumer of the information. Thus, the
specific tool used to start a domain can define its own configuration
file (but please not a python script). Whatever suspends/restarts
a domain can provide its own file for tracking suspended domains, etc.

Different usage scenarios could result in different tools providing
similar functions each which could choose to use a common config file
or define their own, based on needs.

While it is attractive to not have multiple similar config files,
trying to define a common one to contain all anticipated needed
information could easily become unwieldly.

Michael

>
> Regards,
>
--
Michael Hohnbaum 503 578 5486
hohnbaum@us.ibm.com t/l 775 5486



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 14:57, Anthony Liguori wrote:
> On Wed, 2005-01-26 at 15:49, Michael Hohnbaum wrote:
>
> > While beyond the current focus, persistent store could feasibly
> > be used to hold domain definitions for non-existent domains and
> > suspended domains. One could envision adding a state field into
> > the domain configuration/definition. Valid states for current
> > capabilities would be {active, suspended, migrated, inactive}. On
>
> Yes, but the problem that occurs is uniquely identifying a domain. In
> other words, what's the key used within the persistent store?
>
> If it's domain id (which is what I assume it's going to be), you cannot
> tag it as having an "inactive" state because there's nothing that
> prevents a domain from being created with the same domain id.


Can't we add functionality in Xen to make domain id's unique? When
creating a new domain, the management tools can query the running (or
even suspended) domains and find a unique domain id to use. I think you
can tag a domains with their state.


> Also, if you try to assign domains UUIDs or something, what do you do
> for cloning/checkpointing? Do you assign a new UUID on a clone but not
> on a checkpoint? Does assigning new UUIDs propagate to things like MAC
> addresses or other things that are supposed to be unique?


If you clone a domain, it isn't the same domain anymore and would get a
new id. As for checkpointing - you could use the time/date of the
checkpoint and add that to the domain's id.

MAC addresses for specific domains that are in the configuration file or
generated could be stored with the domain configuration and available
for queries.



> There's a lot to be thought about. I think punting the problem (as Andy
> suggests) is the right approach for now.


I disagree. If we're to consider the larger management world, we need to
lay the groundwork for managing domains now. I think the questions
aren't easily answered, but I believe they should be. If we don't
implement everything to start, we should at least have an idea where
we're going.

Thanks,

Dan



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 14:57, Anthony Liguori wrote:
> On Wed, 2005-01-26 at 15:49, Michael Hohnbaum wrote:
>
> > While beyond the current focus, persistent store could feasibly
> > be used to hold domain definitions for non-existent domains and
> > suspended domains. One could envision adding a state field into
> > the domain configuration/definition. Valid states for current
> > capabilities would be {active, suspended, migrated, inactive}. On
>
> Yes, but the problem that occurs is uniquely identifying a domain. In
> other words, what's the key used within the persistent store?
>
> If it's domain id (which is what I assume it's going to be), you cannot
> tag it as having an "inactive" state because there's nothing that
> prevents a domain from being created with the same domain id.
>
> Also, if you try to assign domains UUIDs or something, what do you do
> for cloning/checkpointing? Do you assign a new UUID on a clone but not
> on a checkpoint? Does assigning new UUIDs propagate to things like MAC
> addresses or other things that are supposed to be unique?
>
> There's a lot to be thought about. I think punting the problem (as Andy
> suggests) is the right approach for now.


I believe we should create a single store and API. We can start with the
current domain configuration files. We create a single store and a set
of functions to access it. Then atop that, we can create multiple single
function management apps if you want. You could have an app that just
manages the active domains. You could have an app that does suspend,
etc. Yet you have one store with a common set of commands to access the
information.

Having this architecture, starting it small even, will help with
answering the harder questions down the road.

Thanks,

Dan



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 2005-01-26 at 14:57, Anthony Liguori wrote:
> On Wed, 2005-01-26 at 15:49, Michael Hohnbaum wrote:
>
> > While beyond the current focus, persistent store could feasibly
> > be used to hold domain definitions for non-existent domains and
> > suspended domains. One could envision adding a state field into
> > the domain configuration/definition. Valid states for current
> > capabilities would be {active, suspended, migrated, inactive}. On
>
> Yes, but the problem that occurs is uniquely identifying a domain. In
> other words, what's the key used within the persistent store?
>
> If it's domain id (which is what I assume it's going to be), you cannot
> tag it as having an "inactive" state because there's nothing that
> prevents a domain from being created with the same domain id.
>
> Also, if you try to assign domains UUIDs or something, what do you do
> for cloning/checkpointing? Do you assign a new UUID on a clone but not
> on a checkpoint? Does assigning new UUIDs propagate to things like MAC
> addresses or other things that are supposed to be unique?


I think checkpoints should be stored by the domain id/key for the domain
they are from, using date as an identifier. If you're going to run that
checkpoint image, it then gets a new id to run under and a new entry as
a domain.

Sorry about blasting out messages, just thinking about the questions
everyone has raised.

Thanks,

Dan



> There's a lot to be thought about. I think punting the problem (as Andy
> suggests) is the right approach for now.
>
> Regards,



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Thursday 27 January 2005 00:59, Daniel Stekloff wrote:
> Can't we add functionality in Xen to make domain id's unique? When
> creating a new domain, the management tools can query the running (or
> even suspended) domains and find a unique domain id to use. I think you
> can tag a domains with their state.

Hey, that is exactly what I was thinking when reading that mail:-) Some new
states like 'unconfigured' etc. might need to get added.

> If you clone a domain, it isn't the same domain anymore and would get a
> new id.

... so the ID needs to be unique on a per machine basis only: If you clone and
then migrate a domain to another machine the copy will get a new ID there.

> As for checkpointing - you could use the time/date of the
> checkpoint and add that to the domain's id.

Or a a simple counter that gets incremented, that is easier to work with.
Creation date (and maybe creator) could get stored as some attribute along
with the configuration, independent of whether the set of attributes is a
checkpoint or a "normal" domain.

> MAC addresses for specific domains that are in the configuration file or
> generated could be stored with the domain configuration and available
> for queries.

Yeap. Management tools can then query the state and migrate those settings as
necessary. This approach reduces the amount of policy coded into the core of
xen. I like it.

> I disagree. If we're to consider the larger management world, we need to
> lay the groundwork for managing domains now. I think the questions
> aren't easily answered, but I believe they should be. If we don't
> implement everything to start, we should at least have an idea where
> we're going.

I have to agree with Dan again:-) Xen can proof that its concepts hold and can
be implemented. But all the flexibility xen adds is only worth as much as the
tools provided to manage it.

In my opinion unique DomIds are a good idea anyway: They make configuration
easier since you can just hardcode DomIds into the config files. Currently
you have to use domain names in some places and those have to be parsed at
setup time to figure out which domains are actually referred to. I even have
to edit some of my config files all the time since the name-hack is not
supported in all cases (namely backend in the vif-setup). As an additional
complication there is no requirement on the DomNames to be unique.

PS: This proposed filesystem-based config storage sounds like you could
leverage the elektra project for data storage. That is a simple lib for
registry-like storage of key/value pairs. It features few dependencies and
python bindings:-) The URL is http://elektra.sf.net/.

--
Gruss,
Tobias

------------------------------------------------------------
Tobias Hunger The box said: 'Windows 95 or better'
tobias@aquazul.com So I installed Linux.
------------------------------------------------------------

Daniel Stekloff wrote:

>Can't we add functionality in Xen to make domain id's unique? When
>creating a new domain, the management tools can query the running (or
>even suspended) domains and find a unique domain id to use. I think you
>can tag a domains with their state.
>
>
>
Yes, I think eventually domains have to have some sort of UUID. I'm not
convinced that those can't be mapped onto the current domain ids by
configuration tools though.

Here's a few scenarios to consider:

1) Someone suspends domain A to disk. Domain A continues to run.
Without stopping the original instance of Domain A, they start up the
suspended image.

2) Someone takes the suspended form of Domain A and transfers it to
another machine. They start it up while Domain A is still running on
another machine (creating a resource conflict). If you're using DHCP,
VMware can handle this scenario gracefully btw.

Are these errors? I think it can be argued either way.

>I disagree. If we're to consider the larger management world, we need to
>lay the groundwork for managing domains now. I think the questions
>aren't easily answered, but I believe they should be. If we don't
>implement everything to start, we should at least have an idea where
>we're going.
>
>
I completely agree that we need to lay a groundwork. I think
large-scale domain management tools our outside the scope of the current
focus. What I'd like to see is an architecture that's good enough that
when these large-scale domain management tools are finally worked out
they can just be plugged in without rewriting the Xen management
infrastructure.

I think the key to this is to keep things as simple as possible. Don't
require configuration files, don't rely on any sort of internal database
for persistent state.

BTW, this discussion is great. I'm very interested to see what others
think of the broader management picture.

Regards,

--
Anthony Liguori
anthony@codemonkey.ws



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Thu, 2005-01-27 at 08:19 -0600, Anthony Liguori wrote:
> Daniel Stekloff wrote:
>
> >Can't we add functionality in Xen to make domain id's unique? When
> >creating a new domain, the management tools can query the running (or
> >even suspended) domains and find a unique domain id to use. I think you
> >can tag a domains with their state.
> >
> >
> >
> Yes, I think eventually domains have to have some sort of UUID. I'm not
> convinced that those can't be mapped onto the current domain ids by
> configuration tools though.
>
> Here's a few scenarios to consider:
>
> 1) Someone suspends domain A to disk. Domain A continues to run.
> Without stopping the original instance of Domain A, they start up the
> suspended image.


At some point you need to make rules for operating, deciding based on
how you plan to use it what is the right way and the wrong way. A domain
needs to be unique. It can move through phases like being active or
being suspended. In the case you mention, I don't think Domain A can be
suspended and active at the same time. It's one or the other. If you
need to restart from a checkpoint, you clone the Domain A to Domain B.
We could think of it like process management.


> 2) Someone takes the suspended form of Domain A and transfers it to
> another machine. They start it up while Domain A is still running on
> another machine (creating a resource conflict). If you're using DHCP,
> VMware can handle this scenario gracefully btw.
>
> Are these errors? I think it can be argued either way.


You've raised a good question: are Domain ID's unique only to the system
they are running on? Do we let the cluster manager or virtual farm
manager handle Domains per virtual host? How is the process going to be
managed? Can you have multiple Xen systems in the same network without
having a cluster manager? What are the policies for migrating from one
system to another? What if I wish to start a checkpoint of a particular
domain on another system, which could mean resource conflicts?

I'd like to make sure we can easily reference resources to domains to
hosts, to enable better management - especially on error.

These are the questions we need to be answering now, even if we don't
implement it right away.


> >I disagree. If we're to consider the larger management world, we need to
> >lay the groundwork for managing domains now. I think the questions
> >aren't easily answered, but I believe they should be. If we don't
> >implement everything to start, we should at least have an idea where
> >we're going.
> >
> >
> I completely agree that we need to lay a groundwork. I think
> large-scale domain management tools our outside the scope of the current
> focus. What I'd like to see is an architecture that's good enough that
> when these large-scale domain management tools are finally worked out
> they can just be plugged in without rewriting the Xen management
> infrastructure.
>
> I think the key to this is to keep things as simple as possible. Don't
> require configuration files, don't rely on any sort of internal database
> for persistent state.


I agree that we can't implement the world now, we must do it in small
steps. But we should start working on an overall architecture with at
least a stab at some of these questions. We don't want to be surprised a
year down the road with a situation that our current architecture cannot
handle.

We can start small with incremental releases. The first release (from a
management standpoint only) could include the xcs layer with xend using
it. Another incremental release could put the domain configuration file
into a small store that xend uses through a limited API. This second
release could include the logic for unique domain ids. The third release
targets replacing the current Python tools xm/xend with other management
applications that build upon the initial store. As we add functionality
and management routines, we add to our store and what it can do.
Eventually, we enter the cluster management world.

Overall, we have a strategy for where we want to go and the uses we want
it to have. We just split it up into manageable chunks.


> BTW, this discussion is great. I'm very interested to see what others
> think of the broader management picture.


Me too.

Thanks,

Dan



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

> On Thu, 2005-01-27 at 08:19 -0600, Anthony Liguori wrote:
> > Daniel Stekloff wrote:
> >
> > >Can't we add functionality in Xen to make domain id's unique? When
> > >creating a new domain, the management tools can query the running (or
> > >even suspended) domains and find a unique domain id to use. I think you
> > >can tag a domains with their state.
> > >
> > >
> > >
> > Yes, I think eventually domains have to have some sort of UUID. I'm not
> > convinced that those can't be mapped onto the current domain ids by
> > configuration tools though.
> >
> > Here's a few scenarios to consider:
> >
> > 1) Someone suspends domain A to disk. Domain A continues to run.
> > Without stopping the original instance of Domain A, they start up the
> > suspended image.
>
> At some point you need to make rules for operating, deciding based on
> how you plan to use it what is the right way and the wrong way. A domain
> needs to be unique. It can move through phases like being active or
> being suspended. In the case you mention, I don't think Domain A can be
> suspended and active at the same time. It's one or the other. If you
> need to restart from a checkpoint, you clone the Domain A to Domain B.
> We could think of it like process management.

So it may be useful to revisit one of the original distinctions that
we made, that between "domain" and "virtual machine":

- A domain is a Xen concept that has an associated domain struct,
allocation of physical memory, CPU time, event channels, etc. A
domain has an ID which is unique to all the domains currently
existing on a given physical machine. In general, think of "domain"
as the analog of "process".

- A virtual machine is the thing that inhabits a domain. It may
inhabit different domains (and hence have different domain IDs)
at various points in its life-cycle (e.g. due to suspend/resume,
live migration or even reboot).

Most management tools will want to manage virtual machines as first
class entities, and worry less about domains (all that is required
is a dynamic mapping from VM to (machine id, domain id)). In the
current tools (xend etc), VMs are named by system-wide ASCII names.
Xen knows (and cares) nothing about these. The tools make a basic
effort to ensure that VM names are unique but currently this is
only guaranteed per physical machine rather than per cluster; furhermore
there are some potential race conditions in the current management,
and some inconveniences (e.g. VM name is part of the saved state
rather than being outside it) which makes e.g. VM fork a bit tricky.



> > 2) Someone takes the suspended form of Domain A and transfers it to
> > another machine. They start it up while Domain A is still running on
> > another machine (creating a resource conflict). If you're using DHCP,
> > VMware can handle this scenario gracefully btw.
> >
> > Are these errors? I think it can be argued either way.
>
>
> You've raised a good question: are Domain ID's unique only to the system
> they are running on? Do we let the cluster manager or virtual farm
> manager handle Domains per virtual host? How is the process going to be
> managed? Can you have multiple Xen systems in the same network without
> having a cluster manager? What are the policies for migrating from one
> system to another? What if I wish to start a checkpoint of a particular
> domain on another system, which could mean resource conflicts?
>
> I'd like to make sure we can easily reference resources to domains to
> hosts, to enable better management - especially on error.
>
> These are the questions we need to be answering now, even if we don't
> implement it right away.

See above: domain IDs are per machine (per Xen instance) things. They
are low level handles which Xen uses. They are not site wide unique,
nor are they VM instance unique.

A sensible management tool strategy would be to allow a set of physical
nodes to be managed in a coordinated fashion (particularly due to
live migration, but also due to other forms of location independence,
e.g. "creating" a VM from a stored (booted) template image). This
also implies that VM names should be (a) opaque and (b) site-wide unique.


> > >I disagree. If we're to consider the larger management world, we need to
> > >lay the groundwork for managing domains now. I think the questions
> > >aren't easily answered, but I believe they should be. If we don't
> > >implement everything to start, we should at least have an idea where
> > >we're going.
> > >
> > >
> > I completely agree that we need to lay a groundwork. I think
> > large-scale domain management tools our outside the scope of the current
> > focus. What I'd like to see is an architecture that's good enough that
> > when these large-scale domain management tools are finally worked out
> > they can just be plugged in without rewriting the Xen management
> > infrastructure.
> >
> > I think the key to this is to keep things as simple as possible. Don't
> > require configuration files, don't rely on any sort of internal database
> > for persistent state.
>
>
> I agree that we can't implement the world now, we must do it in small
> steps. But we should start working on an overall architecture with at
> least a stab at some of these questions. We don't want to be surprised a
> year down the road with a situation that our current architecture cannot
> handle.
>
> We can start small with incremental releases. The first release (from a
> management standpoint only) could include the xcs layer with xend using
> it. Another incremental release could put the domain configuration file
> into a small store that xend uses through a limited API. This second
> release could include the logic for unique domain ids. The third release
> targets replacing the current Python tools xm/xend with other management
> applications that build upon the initial store. As we add functionality
> and management routines, we add to our store and what it can do.
> Eventually, we enter the cluster management world.
>
> Overall, we have a strategy for where we want to go and the uses we want
> it to have. We just split it up into manageable chunks.

Right - and one part of that split is separating out "per Xen instance"
functionality and "per site" functionality. Most of the xcs and libxc
and other low level bits and pieces are in the former camp; the new
(in unstable) xend is logically in the latter.



cheers,

S.


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Friday 28 January 2005 01:17, Steven Hand wrote:
> > On Thu, 2005-01-27 at 08:19 -0600, Anthony Liguori wrote:
> > > Daniel Stekloff wrote:
> > > >Can't we add functionality in Xen to make domain id's unique? When
> > > >creating a new domain, the management tools can query the running (or
> > > >even suspended) domains and find a unique domain id to use. I think
> > > > you can tag a domains with their state.
> > >
> > > Yes, I think eventually domains have to have some sort of UUID. I'm
> > > not convinced that those can't be mapped onto the current domain ids by
> > > configuration tools though.
> > >
> > > Here's a few scenarios to consider:
> > >
> > > 1) Someone suspends domain A to disk. Domain A continues to run.
> > > Without stopping the original instance of Domain A, they start up the
> > > suspended image.
> >
> > At some point you need to make rules for operating, deciding based on
> > how you plan to use it what is the right way and the wrong way. A domain
> > needs to be unique. It can move through phases like being active or
> > being suspended. In the case you mention, I don't think Domain A can be
> > suspended and active at the same time. It's one or the other. If you
> > need to restart from a checkpoint, you clone the Domain A to Domain B.
> > We could think of it like process management.
>
> So it may be useful to revisit one of the original distinctions that
> we made, that between "domain" and "virtual machine":
>
> - A domain is a Xen concept that has an associated domain struct,
> allocation of physical memory, CPU time, event channels, etc. A
> domain has an ID which is unique to all the domains currently
> existing on a given physical machine. In general, think of "domain"
> as the analog of "process".
>
> - A virtual machine is the thing that inhabits a domain. It may
> inhabit different domains (and hence have different domain IDs)
> at various points in its life-cycle (e.g. due to suspend/resume,
> live migration or even reboot).


Ok, I misunderstood. I thought you'd tie one to the other.

Is there a need for and how easy would it be to reference a VM to domains
over time or virtual resources to physical resources? What happens when an
error occurs with a bit of hardware? Does Xen and Xen tools report it? What
about the VM that's running on the system? What if a VM reports an error
while running in one Domain and then the VM is moved to another Domain with
different resources? Would we need to store history of VM to machine id and
domain id?

Or am I missing something?

Thanks,

Dan


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel