Mailing List Archive

pvhgrub docs
Out of curiosity, could someone point me to some documentation for pvhgrub?
Searching for

"pvhgrub" xen

returns very few results.

Thanks!

On Mon, Oct 4, 2021, 9:09 PM Chris Myers <chrismyers81@gmail.com> wrote:

> Thanks! I'll check that out.
>
> Up to this point I've always just done in-place upgrades of everything
> (been running Xen PV domu's for about 7 years now) and have just followed
> with that old way because it generally just worked (with a few nuances.)
>
> On Mon, Oct 4, 2021 at 8:38 PM Andy Smith <andy@strugglers.net> wrote:
>
>> On Mon, Oct 04, 2021 at 08:35:35PM -0500, Chris Myers wrote:
>> > Out of curiosity is there a better/preferred way of handling it so that
>> I
>> > don't need to remember to keep them in sync every time I do a kernel
>> > upgrade?
>>
>> > > Once you've got to the bottom of this I recommend switching to
>> > > pvhgrub booting to do away with that class of problems…
>>
>>
>
> --
> People use duct tape to fix everything....God used nails.
>
> http://www.myerscountry.net
>
Re: pvhgrub docs [ In reply to ]
is pvhgrub a part of xen or grub2 project? (dont see it at xen 4.15.1 in centos8 RPM's) Am 2021-10-07 14:09, Chris Myers <chrismyers81@gmail.com> schrieb: > > Out of curiosity, could someone point me to some documentation for pvhgrub? Searching for > > > "pvhgrub" xen > > > > returns very few results. > > > > Thanks! > > > > > On Mon, Oct 4, 2021, 9:09 PM Chris Myers <chrismyers81@gmail.com (mailto:chrismyers81@gmail.com)> wrote: > > > > > > > > Thanks! I'll check that out. > > > > > > > > Up to this point I've always just done in-place upgrades of everything (been running Xen PV domu's for about 7 years now) and have just followed with that old way because it generally just worked (with a few nuances.) > > > > > > > > > > On Mon, Oct 4, 2021 at 8:38 PM Andy Smith <andy@strugglers.net (mailto:andy@strugglers.net)> wrote: > > > > > On Mon, Oct 04, 2021 at 08:35:35PM -0500, Chris Myers wrote: > > > > Out of curiosity is there a better/preferred way of handling it so that I > > > > don't need to remember to keep them in sync every time I do a kernel > > > > upgrade? > > > > > > > > Once you've got to the bottom of this I recommend switching to > > > > > pvhgrub booting to do away with that class of problems… > > > > -- > > > > People use duct tape to fix everything....God used nails. > > > > http://www.myerscountry.net
Re: pvhgrub docs [ In reply to ]
On 07.10.21 14:30, perun wrote:
> is pvhgrub a part of xen or grub2 project? (dont see it at xen 4.15.1 in
> centos8 RPM's)

It is part of grub2.


Juergen
Re: pvhgrub docs [ In reply to ]
Hi Chris,

FWIW I found a combination of the Xen PVH wiki [1] and [2] useful when
evaluating PVH mode for a use case.

On a Debian host (from at least Buster) pvhgrub's install location is
/usr/lib/grub-xen/grub-i386-xen_pvh.bin (grub-xen-host) and
/usr/lib/grub/* (grub-xen-bin).

[1] https://wiki.xenproject.org/wiki/Linux_PVH
[2] https://tools.bitfolk.com/wiki/PVH

HTH?
C

On Thu, 7 Oct 2021 at 13:10, Chris Myers <chrismyers81@gmail.com> wrote:
>
> Out of curiosity, could someone point me to some documentation for pvhgrub? Searching for
>
> "pvhgrub" xen
>
> returns very few results.
>
> Thanks!
>
> On Mon, Oct 4, 2021, 9:09 PM Chris Myers <chrismyers81@gmail.com> wrote:
>>
>> Thanks! I'll check that out.
>>
>> Up to this point I've always just done in-place upgrades of everything (been running Xen PV domu's for about 7 years now) and have just followed with that old way because it generally just worked (with a few nuances.)
>>
>> On Mon, Oct 4, 2021 at 8:38 PM Andy Smith <andy@strugglers.net> wrote:
>>>
>>> On Mon, Oct 04, 2021 at 08:35:35PM -0500, Chris Myers wrote:
>>> > Out of curiosity is there a better/preferred way of handling it so that I
>>> > don't need to remember to keep them in sync every time I do a kernel
>>> > upgrade?
>>>
>>> > > Once you've got to the bottom of this I recommend switching to
>>> > > pvhgrub booting to do away with that class of problems…
>>>
>>
>>
>> --
>> People use duct tape to fix everything....God used nails.
>>
>> http://www.myerscountry.net
Re: pvhgrub docs [ In reply to ]
On 2021-10-07 5:09 a.m., Chris Myers wrote:
> Out of curiosity, could someone point me to some documentation for
> pvhgrub? Searching for
>
> "pvhgrub" xen
>
> returns very few results.
>
> Thanks!
>
> On Mon, Oct 4, 2021, 9:09 PM Chris Myers <chrismyers81@gmail.com
> <mailto:chrismyers81@gmail.com>> wrote:
>
> Thanks! I'll check that out.
>
> Up to this point I've always just done in-place upgrades of
> everything (been running Xen PV domu's for about 7 years now) and
> have just followed with that old way because it generally just
> worked (with a few nuances.)
>
> On Mon, Oct 4, 2021 at 8:38 PM Andy Smith <andy@strugglers.net
> <mailto:andy@strugglers.net>> wrote:
>
> On Mon, Oct 04, 2021 at 08:35:35PM -0500, Chris Myers wrote:
> > Out of curiosity is there a better/preferred way of handling
> it so that I
> > don't need to remember to keep them in sync every time I do
> a kernel
> > upgrade?
>
> > > Once you've got to the bottom of this I recommend switching to
> > > pvhgrub booting to do away with that class of problems…
>
>
>
> --
> People use duct tape to fix everything....God used nails.
>
> http://www.myerscountry.net <http://www.myerscountry.net>
>
Hi Chris,

pvhgrub is new to me, but to understand the transition away from pygrub,
I found pvgrub2 documentation helpful (although some of it is a bit dated):

https://wiki.xenproject.org/wiki/PvGrub2
Re: pvhgrub docs [ In reply to ]
Hi Chris,

On Thu, Oct 07, 2021 at 07:09:35AM -0500, Chris Myers wrote:
> Out of curiosity, could someone point me to some documentation for pvhgrub?

I haven't found much, I've had to teach myself from other people's
examples and some out of date articles on the Xen wiki. Here is what I
learned so far and have had working for a year or so.

If you're currently running a PV mode guest and it has a kernel of
4.19 or greater, you can/should change it to a PVH mode guest. You
can also boot it using pvhgrub (instead of pvgrub, pygrub or direct
kernel booting).

Inside the domU all you need is a grub.cfg file generated that grub
understands. The easiest way to do this in my experience is to
actually install grub inside the guest, even though only its config
file is needed. So inside a Debian guest you'd install grub-pc and
make sure that the install process leaves you with a correct
/boot/grub/grub.cfg. I didn't have to do anything special on
Debian/Ubuntu to make it generate a correct config, and keep it up
to date.

On the dom0 the kernel file you use for PVH guests is a grub2 image
built as a PVH binary. On Debian it can be found in the
grub-xen-host package and once installed that lives at
/usr/lib/grub-xen/grub-i386-xen_pvh.bin. I'll refer to that as "the
pvhgrub image/binary".

So in your domU config:

[…]
type = pvh
kernel = /usr/lib/grub-xen/grub-i386-xen_pvh.bin
[…]

I recommend just comment out your existing "kernel", "ramdisk" and
"extra" lines so you can put them back if this doesn't work.

The default behaviour of that pvhgrub binary is to run grub and
search all available block devices for the following file paths:

/boot/grub/grub.cfg
/grub/grub.cfg

If it finds any such file then it assumes that is a grub config and
loads it. The second path is for the case where /boot is its own
block device. The path on that device would be /grub/grub.cfg.

Thus, the domU admin can install their own kernels, initramfs and
kernel command line arguments and boot them without dom0 admin
assistance. The domU admin can do anything that grub can do, because
it really is grub. The domU config file in dom0 doesn't have to be
changed every time there is a kernel upgrade inside the guest.

I personally wanted a bit more complex initial grub config because I
also wanted it to search for /boot/grub2/grub.cfg (Red Hat / Fedora
path) and /boot/grub/menu.lst (legacy grub config file), so I
compile my own pvhgrub binary. Instructions for that are here:

https://wiki.xenproject.org/wiki/Xen_4.12_RC_test_instructions#PVH_domU_boot_via_grub2

Except I seem to have modified my process a little:

$ git clone https://git.savannah.gnu.org/git/grub.git
$ cd grub
$ ./configure --prefix=/opt/grub \
--target=x86-64 \
--with-platform=xenpvh && \
make && sudo make install
$ edit /opt/grub/share/memdisk/grub.cfg # YOUR BOOTSTRAP CONFIG
$ echo "normal (memdisk)/grub.cfg" | \
sudo tee /opt/grub/etc/grub-bootstrap.cfg
$ sudo tar -C /opt/grub/share/memdisk \
-cvf /opt/grub/share/memdisk.tar grub.cfg && \
sudo /opt/grub/bin/grub-mkimage -O i386-xenpvh \
-c /opt/grub/etc/grub-bootstrap.cfg \
-m /opt/grub/share/memdisk.tar \
-o /opt/grub/lib/pvhgrub.bin \
-p /boot/grub \
-v /opt/grub/lib/grub/i386-xenpvh/*.mod

So, that works for me, but you don't need to do any of that if
you're happy with the Debian default behaviour of just looking for
boot/grub.cfg on the domU's block devices.

Even if you do want a different bootstrap behaviour maybe you can
even get away without completely compiling your own pvhgrub binary —
as I do there — by just coming up with your own config and using
grub-mkimage from the package, I am doing this because I started
before pvhgrub was packaged and only just committed in upstream
grub.

If you run into any trouble put back the "kernel", "ramdisk" and
"extra" lines to return to a booting configuration for that domU.

Perhaps some or all of this should go in an article on the Xen wiki.
I do worry about it becoming yet another out of date reference
though.

Cheers,
Andy
Re: pvhgrub docs [ In reply to ]
Hello,

On Thu, Oct 07, 2021 at 02:47:08PM -0700, David Hill wrote:
> pvhgrub is new to me, but to understand the transition away from pygrub, I
> found pvgrub2 documentation helpful (although some of it is a bit dated):
>
> https://wiki.xenproject.org/wiki/PvGrub2

Ah yes, that's another one I used to get the idea of the basic
concept, and if you still need to run PV domains that would be the
way to do it still.

But if the guest kernel is new enough I recommend just go straight
to PVH mode now, which for this style of booting does need grub
compiled as --with-platform=xenpvh

On Debian dom0 if you install grub-xen-host you get pvgrub images
at:

/usr/lib/grub-xen/grub-x86_64-xen.bin
/usr/lib/grub-xen/grub-i386-xen.bin

and pvhgrub image at:

/usr/lib/grub-xen/grub-i386-xen_pvh.bin

The PVH one should work for both 64- and 32-bit.

If needing the specific one for 32-bit PV then bear in mind that:

- there's no security support from Xen for that

- 32-bit PV guest support is removed in 4.15

- support for booting as 32-bit PV guest was already removed in
Linux kernel at v5.9.

Cheers,
Andy
Re: pvhgrub docs [ In reply to ]
Awesome, thanks very much! I'll start playing around with it this weekend
after I finish getting things upgraded.

Have a great Friday!

On Thu, Oct 7, 2021, 5:26 PM Andy Smith <andy@strugglers.net> wrote:

> Hello,
>
> On Thu, Oct 07, 2021 at 02:47:08PM -0700, David Hill wrote:
> > pvhgrub is new to me, but to understand the transition away from pygrub,
> I
> > found pvgrub2 documentation helpful (although some of it is a bit dated):
> >
> > https://wiki.xenproject.org/wiki/PvGrub2
>
> Ah yes, that's another one I used to get the idea of the basic
> concept, and if you still need to run PV domains that would be the
> way to do it still.
>
> But if the guest kernel is new enough I recommend just go straight
> to PVH mode now, which for this style of booting does need grub
> compiled as --with-platform=xenpvh
>
> On Debian dom0 if you install grub-xen-host you get pvgrub images
> at:
>
> /usr/lib/grub-xen/grub-x86_64-xen.bin
> /usr/lib/grub-xen/grub-i386-xen.bin
>
> and pvhgrub image at:
>
> /usr/lib/grub-xen/grub-i386-xen_pvh.bin
>
> The PVH one should work for both 64- and 32-bit.
>
> If needing the specific one for 32-bit PV then bear in mind that:
>
> - there's no security support from Xen for that
>
> - 32-bit PV guest support is removed in 4.15
>
> - support for booting as 32-bit PV guest was already removed in
> Linux kernel at v5.9.
>
> Cheers,
> Andy
>
>
Re: pvhgrub docs [ In reply to ]
On Fri, 8 Oct 2021 at 22:53, Chris Myers <chrismyers81@gmail.com> wrote:

> Awesome, thanks very much! I'll start playing around with it this weekend
> after I finish getting things upgraded.
>
> Have a great Friday!
>
> On Thu, Oct 7, 2021, 5:26 PM Andy Smith <andy@strugglers.net> wrote:
>
>> Hello,
>> and pvhgrub image at:
>>
>> /usr/lib/grub-xen/grub-i386-xen_pvh.bin
>>
>> The PVH one should work for both 64- and 32-bit.
>>
>>
it does work very well indeed

Cheers,
Tomasz
Re: pvhgrub docs [ In reply to ]
Hi,

On 10/8/21 12:14 AM, Andy Smith wrote:
>
> On Thu, Oct 07, 2021 at 07:09:35AM -0500, Chris Myers wrote:
>> Out of curiosity, could someone point me to some documentation for pvhgrub?
>
> I haven't found much, I've had to teach myself from other people's
> examples and some out of date articles on the Xen wiki. Here is what I
> learned so far and have had working for a year or so.
>
> If you're currently running a PV mode guest and it has a kernel of
> 4.19 or greater, you can/should change it to a PVH mode guest.
> [...]

Actually, it's since Linux 4.20, officially, but the few remaining bits
to make it possible were also added [0] to the Debian 4.19 kernel so
that we had everything working for Buster.

Hans

[0]
https://salsa.debian.org/kernel-team/linux/commit/4d63e6ccbbd6081068633b1147e0f77a59379795