Mailing List Archive

Hi Anthony,

Thanks for taking the time to read, and thanks for the feedback: The spec clearly has a way to go, and will benefit a lot from a wider reading and comments like yours. My comments inline:


> Overall, I'm rather keen on this concept. A format for describing a
> virtual machine such that it's easy to deploy as an appliance. I think
> what you've come up with is quite good as a first cut but I'm slightly
> concerned that it's a tad over specific. In general, I think this is
> bound to happen anytime you try to specify first before implementing and
> having users.

Well put. There is a bit of a chicken-egg problem with specifying this sort of thing, and I'm sure that it shows through in the current draft. We've done the best we can with the knowledge that we have and the set of appliances that we can anticipate.

I'm sure that there are aspects of the spec that are over-specific (disk names, for instance), in that they target xen/linux/x86 too much. We are very keen to iron these out. I think there are other aspects, like multi-vm network config, that may benefit from a more specific coverage.

> So before I get into specifics, what are the plans moving forward? I
> think if we approached this as a guideline and attempted to build a
> system for Xen that worked with these OVA packages, it would be a good
> way to validate the spec (which could then be finalized). Thoughts?

I think this is exactly what we imagine as the next step.

> I'm a little confused as to whether the intention is to build images
> that will run on any hypervisor or that run on a specific hypervisor.
> If it is the later (which it seems to be), then what is the value in
> standardizing since there is no compatibility?

The intention is the former. Our motivation was that "open" specs like VMDK are not sufficient as a transport vehicle: they don't let you configure/specialize the appliance, etc etc... hopefully the document does a reasonable job of articulating this. So the intention is certainly that the format accommodate other VMMs -- so the Xen-isms that are there all need a bit of development. We're hoping that the end product of the OVA process is a well-designed transport format that anyone can use; what they do in terms of juggling the bits into a specific run-time format is completely up to them.

> This brings me to the vbd description for Xen. device="sda1" is a
> concerning example as we shouldn't be using things other than xvd for
> paravirt domains. This brings up a practical question of whether these
> sort of things should be insulted from the developer to begin with?
> Thoughts?

Hopefully we can come up with a reasonable way of naming devices that everyone can be happy with. Suggestions are more than welcome. ;)

> I think the property system is a noble attempt at solving a pretty hard
> problem. However, I don't think it will be sufficient for very long.
> Users are going to want GUIs and I suspect that means we need to think a
> little harder about this problem.

Ewan will have more to say on this -- the property approach was intended to support GUIs while still being reasonably scriptable/automatable. If we want to support greater forms of prettiness/graphic design then this will need to be built out -- maybe that's not for the first rev of the spec though. ;)

> The script stuff is a little odd. The script gets it's own VM? With
> Perl? Seriously? ;-)

The bottom-line thing that we are desperately trying to avoid is the VMM equivalent of dll hell. OVA tries to represent an installable package that can be rigorously isolated, and that imposes a _minimum_ of shared config state on the VMM/tools. A one-off VM for running customization scripts seems perfectly reasonable to me, if we've got these VMs, we may as well use them, no?

> The security stuff is interesting but I don't know enough about whether
> that will be acceptable to vendors. Have you gotten an initial reaction
> yet?

The (very limited) initial reaction that I have seen has been good, but I agree that there are details to add. More specific comments here would be excellent.

> The rendevous section was a little odd too. I think it presupposes that
> each VM is going to have one network device (which is a bad
> assumption). Also, putting IPs in XenStore seems like a bad idea.
> There are so many ways to do network identification, why reinvent the wheel?

I agree that the rendezvous stuff is kind of strange. Having a more complete way to specify network topologies within appliances would be very good and if anyone can suggest established approaches we'd be happy to look at them. The spec certainly isn't meant to assume a limit of one NIC per VM.

Bear in mind that the use of xenstore described here is just to pass a bunch of config-time data to the VM for its scripts to use for customization. Nobody is suggesting that these be incorporated into split driver configs (for instance). Also, in many cases it's quite possible that these would just specify hostnames (or less) and let you defer to DHCP -- however, it seemed to make quite a bit of sense to allow an appliance the configuration option of sharing a local virtual subnet and assigning static non-forwarded IPs. Among other things this sort of config is nice for migration.

Thanks a lot for the feedback -- if you want to start working through some specific solutions to point-by-point concerns, that would be excellent. We can also start to talk in more detail about building a couple of more real prototypes and associated tool support.

best,
a.

On Thu, Jun 22, 2006 at 02:30:52PM +0100, Ewan Mellor wrote:
> Attached is a preview of the Open Virtual Appliance specification. We've been
> working on this for a little while now, and it's ready to open up to wider
> discussion and collaboration.
>
> The intention here is to produce a transport format for a Virtual Appliance --
> a collection of virtual machines that together, from the customer's point of
> view, provide a single useful facility.
>
> This document has a restrictive licence and disclaimer, which we've put on
> whilst it is being reviewed. When released, the document and the
> specification will have some form of free document licence, possibly the GNU
> FDL, with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
> Texts.
>
> Your comments are more than welcome.
>
> Ewan.

Ewan,

I spent some time reading the spec yesterday, and have some comments /
questions on it, mostly on the customization side. Before I get started,
I'd have to commend you and Andrew at taking a very reasonable stab at this
problem, and I like where this is headed. :) Most of the document is
straight up sensible to me, so I'll only focus on the areas that I think
need some more thought or other approaches.


I'm not very keen on the <prompt> model that is laid out. What I think
that leads to is an overuse of trying to get interactive user input at
deploy time. This causes the solution to be less useful on large scale roll
out, as interactive user input doesn't scale.

Instead I'd suggest moving most of the prompt attributes into the
<properties> tag. Properties would specify all the meta information around
them, and if someone really wanted to create an interactive element to them,
they could still do it, but it wouldn't imply that was the way to address
the problem.


On the scripts side, while it is good to have arbitrary script running to
accomplish anything, one of RPMs failings is the fact that a lot of what
happens in there is pretty repetitive, and it would have made sense to make
some common operations.

For instance, on the ntp-server prompt-values it would seem to have
something like:

<replace>
<file name="/etc/ntp.conf" fs="crm-root" property="ntp-server"/>
</replace>

This would tell the processor that crm-root's /etc/ntp.conf may be tagged as
so:

...
# server clock.via.net
server %!% ntp-server %!%

...

And that the ntp-server value should be replaced. (All syntax is mutable at
this point, but it seemed like a concrete example would be helpful.)

As some large percent of scripts are probably doing file modifications,
explicitly creating a call out for that would probably be good.


From the script perspective it seems that only a very small amount of the
root ends up writable (unless I am reading it wrong), which means
modifications to /etc don't seem like an option. Did I miss something, or
was that intentional.

Also, is there an edict that scripts are non interactive? That has been RPM
policy for a long time, and it makes automating processes much easier. I'd
really suggest that it also end up as policy in this spec.

Discussion welcomed. I'm very keen on helping this proposal mature and
become an open standard.

-Sean

--
Sean Dague
IBM Linux Technology Center email: japh@us.ibm.com
Open Hypervisor Team alt: sldague@us.ibm.com

On Wed, Jun 28, 2006 at 11:24:53AM -0400, Sean Dague wrote:

> On Thu, Jun 22, 2006 at 02:30:52PM +0100, Ewan Mellor wrote:
> > Attached is a preview of the Open Virtual Appliance specification. We've been
> > working on this for a little while now, and it's ready to open up to wider
> > discussion and collaboration.
> >
> > The intention here is to produce a transport format for a Virtual Appliance --
> > a collection of virtual machines that together, from the customer's point of
> > view, provide a single useful facility.
> >
> > This document has a restrictive licence and disclaimer, which we've put on
> > whilst it is being reviewed. When released, the document and the
> > specification will have some form of free document licence, possibly the GNU
> > FDL, with no Invariant Sections, no Front-Cover Texts, and no Back-Cover
> > Texts.
> >
> > Your comments are more than welcome.
> >
> > Ewan.
>
> Ewan,
>
> I spent some time reading the spec yesterday, and have some comments /
> questions on it, mostly on the customization side. Before I get started,
> I'd have to commend you and Andrew at taking a very reasonable stab at this
> problem, and I like where this is headed. :) Most of the document is
> straight up sensible to me, so I'll only focus on the areas that I think
> need some more thought or other approaches.

I'm glad ;-) We look forward to getting this out as an open specification so
that guys like you can get cracking on it!

> I'm not very keen on the <prompt> model that is laid out. What I think
> that leads to is an overuse of trying to get interactive user input at
> deploy time. This causes the solution to be less useful on large scale roll
> out, as interactive user input doesn't scale.
>
> Instead I'd suggest moving most of the prompt attributes into the
> <properties> tag. Properties would specify all the meta information around
> them, and if someone really wanted to create an interactive element to them,
> they could still do it, but it wouldn't imply that was the way to address
> the problem.

I'll have a think about this. To address the large-scale issue, the intention
is that OVAs can include answers to interactive questions, which the sysadmin
would specify once before rollout. A "preinstaller" tool would take the OVA
and tweak it as specified by the sysadmin to match the cluster. Of course,
OVA authors ought to be made aware that customers are always going to welcome
a non-interactive install than an interactive one.

> On the scripts side, while it is good to have arbitrary script running to
> accomplish anything, one of RPMs failings is the fact that a lot of what
> happens in there is pretty repetitive, and it would have made sense to make
> some common operations.
>
> For instance, on the ntp-server prompt-values it would seem to have
> something like:
>
> <replace>
> <file name="/etc/ntp.conf" fs="crm-root" property="ntp-server"/>
> </replace>
>
> This would tell the processor that crm-root's /etc/ntp.conf may be tagged as
> so:
>
> ...
> # server clock.via.net
> server %!% ntp-server %!%
>
> ...
>
> And that the ntp-server value should be replaced. (All syntax is mutable at
> this point, but it seemed like a concrete example would be helpful.)
>
> As some large percent of scripts are probably doing file modifications,
> explicitly creating a call out for that would probably be good.

Rather than adding more XML tags, and turning into Apache Ant, I'd like to
specify a script library, required to be present in all installers, and
available to all OVAs. If we get these scripts right, then the effect should
be the same, as far as OVA authors are concerned, but the DTD for the ova.xml
isn't enormous. Your example would turn into:

<script name="fix-ntp" script="library/property-replacer">
<param>/etc/ntp.conf</param>
</script>

> From the script perspective it seems that only a very small amount of the
> root ends up writable (unless I am reading it wrong), which means
> modifications to /etc don't seem like an option. Did I miss something, or
> was that intentional.

The script will have the whole filesystem available as (for example)
/var/lib/ova/fs/crm-root -- the install script can mount and edit those
filesystems however they wish.

> Also, is there an edict that scripts are non interactive? That has been RPM
> policy for a long time, and it makes automating processes much easier. I'd
> really suggest that it also end up as policy in this spec.

Yes, they are non-interactive -- I'll put that in the spec, thanks. The
scripts themselves are running in a separate VM from the installer, so there's
not a lot to interact with ;-)

> Discussion welcomed. I'm very keen on helping this proposal mature and
> become an open standard.

Looking forward to it!

Ewan.

On Wed, Jun 28, 2006 at 04:49:13PM +0100, Ewan Mellor wrote:
<snip>
> > I spent some time reading the spec yesterday, and have some comments /
> > questions on it, mostly on the customization side. Before I get started,
> > I'd have to commend you and Andrew at taking a very reasonable stab at this
> > problem, and I like where this is headed. :) Most of the document is
> > straight up sensible to me, so I'll only focus on the areas that I think
> > need some more thought or other approaches.
>
> I'm glad ;-) We look forward to getting this out as an open specification so
> that guys like you can get cracking on it!

Any ETAs on that? ;)

> > I'm not very keen on the <prompt> model that is laid out. What I think
> > that leads to is an overuse of trying to get interactive user input at
> > deploy time. This causes the solution to be less useful on large scale roll
> > out, as interactive user input doesn't scale.
> >
> > Instead I'd suggest moving most of the prompt attributes into the
> > <properties> tag. Properties would specify all the meta information around
> > them, and if someone really wanted to create an interactive element to them,
> > they could still do it, but it wouldn't imply that was the way to address
> > the problem.
>
> I'll have a think about this. To address the large-scale issue, the intention
> is that OVAs can include answers to interactive questions, which the sysadmin
> would specify once before rollout. A "preinstaller" tool would take the OVA
> and tweak it as specified by the sysadmin to match the cluster. Of course,
> OVA authors ought to be made aware that customers are always going to welcome
> a non-interactive install than an interactive one.

One of the things I think that will make OVA successful is if there is a
seperation between the What and the How, and leave the How up to someone
else.

In the current form you specify both "what" as properties, and "how" to fill
them in as prompts. This already starts to dictate implementation, which
builds more assumptions into the model, and less flexibility. If you just
specify "what" needs to be filled in, and leave the rest as an exercise to
the user, I think it is a better approach.

> > On the scripts side, while it is good to have arbitrary script running to
> > accomplish anything, one of RPMs failings is the fact that a lot of what
> > happens in there is pretty repetitive, and it would have made sense to make
> > some common operations.
> >
> > For instance, on the ntp-server prompt-values it would seem to have
> > something like:
> >
> > <replace>
> > <file name="/etc/ntp.conf" fs="crm-root" property="ntp-server"/>
> > </replace>
> >
> > This would tell the processor that crm-root's /etc/ntp.conf may be tagged as
> > so:
> >
> > ...
> > # server clock.via.net
> > server %!% ntp-server %!%
> >
> > ...
> >
> > And that the ntp-server value should be replaced. (All syntax is mutable at
> > this point, but it seemed like a concrete example would be helpful.)
> >
> > As some large percent of scripts are probably doing file modifications,
> > explicitly creating a call out for that would probably be good.
>
> Rather than adding more XML tags, and turning into Apache Ant, I'd like to
> specify a script library, required to be present in all installers, and
> available to all OVAs. If we get these scripts right, then the effect should
> be the same, as far as OVA authors are concerned, but the DTD for the ova.xml
> isn't enormous. Your example would turn into:
>
> <script name="fix-ntp" script="library/property-replacer">
> <param>/etc/ntp.conf</param>
> </script>

So, again, it is "what" vs. "how". If we know there needs to be file level
replaces with a <replace> stanza, many people can write different ways that
fulfill that. Which means you could have a perl solution on Linux/UNIX, and
a C# solution on Win32, and a Objective C solution on Mac OSX.

If you go down a standard script route, then you'll be limitting platforms
that this supports, as how is presuposed, and you'd really need to also add
exact behavior of those scripts into the spec.

I think that anything which happens often enough to get a "standard script",
should really get a set of standard tags instead, and an example standard
script could be provided. This will allow complete replacement of the
standard script if there were specific needs by the end user (like small
memory contraints, or some platform specific features).

> > From the script perspective it seems that only a very small amount of the
> > root ends up writable (unless I am reading it wrong), which means
> > modifications to /etc don't seem like an option. Did I miss something, or
> > was that intentional.
>
> The script will have the whole filesystem available as (for example)
> /var/lib/ova/fs/crm-root -- the install script can mount and edit those
> filesystems however they wish.

Ok, I saw that only /mnt, /tmp, and some /var dirs were mounted. Perhaps
the wording could be cleaned up there to make ti more clear.

> > Also, is there an edict that scripts are non interactive? That has been RPM
> > policy for a long time, and it makes automating processes much easier. I'd
> > really suggest that it also end up as policy in this spec.
>
> Yes, they are non-interactive -- I'll put that in the spec, thanks. The
> scripts themselves are running in a separate VM from the installer, so there's
> not a lot to interact with ;-)

While true, you be surprised what people will do without guidelines. ;)

> > Discussion welcomed. I'm very keen on helping this proposal mature and
> > become an open standard.
>
> Looking forward to it!
>
> Ewan.

-Sean

--
Sean Dague
IBM Linux Technology Center email: japh@us.ibm.com
Open Hypervisor Team alt: sldague@us.ibm.com