Mailing List Archive: Grub xen_pvh platform does not seem to support uncompressed kernels

Grub xen_pvh platform does not seem to support uncompressed kernels

Nov 1, 2020, 10:22 PM

Post #1 of 7 (547 views)

Hi,

[. xen-users in CC as I know a few there are using a decompressed
kernel workaround for LZ4 and might be hit by this too ]

I'm currently evaluating switching from booting Xen guests with grub
Xen PV (grub-mkimage -O x86_64-xen) to Xen PVH (grub-mkimage -O
i386-xen_pvh). I've noticed that a decompressed guest kernel refuses
to boot under PVH with the following error:

Loading Linux 5.4.0-52-generic ...
error: invalid magic number.

That same kernel boots with PV.

If I use the LZ4-compressed version of that kernel as supplied by
the distribution (Ubuntu 20.04), it works PVH. I therefore can only
conclude that PVH grub doesn't like kernels with no compression. Is
that expected?

I suppose it is possible that the method I use to decompress the
kernel could be wrong, though as I say, it does work with PV grub. I
basically use the extract-vmlinux script from the upstream kernel
source?.

When decompressed:

$ sudo file /boot/vmlinuz-5.4.0-52-generic
/boot/vmlinuz-5.4.0-52-generic: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=d435b2b7b13b7e1904dddeee22bff30225d2bd04, stripped
$ ls -lah /boot/vmlinuz-5.4.0-52-generic
-rw------- 1 root root 43M Nov 1 20:00 /boot/vmlinuz-5.4.0-52-generic

When leaving it LZ4 compressed as it comes from the distributor:

$ sudo file /boot/vmlinuz-5.4.0-52-generic
/boot/vmlinuz-5.4.0-52-generic: Linux kernel x86 boot executable bzImage, version 5.4.0-52-generic (buildd@lgw01-amd64-060) #57-Ubuntu SMP Thu Oct 15 10:57:00 UTC 2020, RO-rootFS, swap_dev 0xB, Normal VGA
$ ls -lah /boot/vmlinuz-5.4.0-52-generic
-rw------- 1 root root 12M Oct 15 11:33 /boot/vmlinuz-5.4.0-52-generic

The reason why I am decompressing these guest kernels is because
PV grub does not understand LZ4 compression and as I currently do
boot guests this way, they would be unbootable otherwise.

For new installs I can skip installing my hack that decompresses the
kernel image, but if PVH grub won't support uncompressed kernel
images then this is inconvenient for the installed base who will
need to make extra changes.

Is not supporting decompressed kernel images a bug in PVH grub?

This was tested with a git clone just now, following these
instructions:

https://wiki.xenproject.org/wiki/Xen_4.12_RC_test_instructions#PVH_domU_boot_via_grub2

Thanks,
Andy

? https://raw.githubusercontent.com/torvalds/linux/master/scripts/extract-vmlinux

Re: Grub xen_pvh platform does not seem to support uncompressed kernels [ In reply to ]

jgross at suse

Nov 1, 2020, 11:43 PM

Post #2 of 7 (547 views)

Permalink

On 02.11.20 07:22, Andy Smith wrote:
> Hi,
>
> [. xen-users in CC as I know a few there are using a decompressed
> kernel workaround for LZ4 and might be hit by this too ]
>
> I'm currently evaluating switching from booting Xen guests with grub
> Xen PV (grub-mkimage -O x86_64-xen) to Xen PVH (grub-mkimage -O
> i386-xen_pvh). I've noticed that a decompressed guest kernel refuses
> to boot under PVH with the following error:
>
> Loading Linux 5.4.0-52-generic ...
> error: invalid magic number.
>
> That same kernel boots with PV.
>
> If I use the LZ4-compressed version of that kernel as supplied by
> the distribution (Ubuntu 20.04), it works PVH. I therefore can only
> conclude that PVH grub doesn't like kernels with no compression. Is
> that expected?

Can you boot the decompressed kernel on bare metal or in an HVM
domain using grub?

I'm asking because PVH grub is using the standard i386 kernel loading
functions, so I wouldn't expect any differences here compared to bare
metal.

It might still be that PVH grub is missing some code due to wrong
configure parameters, of course.

Juergen

Re: Grub xen_pvh platform does not seem to support uncompressed kernels [ In reply to ]

andy at strugglers

Nov 2, 2020, 12:27 AM

Post #3 of 7 (547 views)

Permalink

Hi J?rgen,

On Mon, Nov 02, 2020 at 08:43:24AM +0100, J?rgen Gro? wrote:
> On 02.11.20 07:22, Andy Smith wrote:
> >I therefore can only conclude that PVH grub doesn't like kernels
> >with no compression. Is that expected?
>
> Can you boot the decompressed kernel on bare metal or in an HVM
> domain using grub?

Oh that's interesting. The decompressed kernel doesn't boot bare
metal either (same message).

The decompressed kernel is generated like this:

# extract-vmlinux /boot/vmlinuz-5.4.0-52-generic > /boot/vmlinuz-5.4.0-52-generic-decompressed

The decompressed kernel does boot when presented to the hypervisor
directly, i.e. no grub, directly listed in guest config file.

I don't have an HVM test setup right now so am unable to test that,
but I suppose we can assume it will fail as the problem is in the
standard grub loading functions.

So, is this a bug? Grub should be able to boot uncompressed kernels,
shouldn't it?

Thanks,
Andy

Re: Grub xen_pvh platform does not seem to support uncompressed kernels [ In reply to ]

jgross at suse

Nov 2, 2020, 12:43 AM

Post #4 of 7 (547 views)

Permalink

On 02.11.20 09:27, Andy Smith wrote:
> Hi Jürgen,
>
> On Mon, Nov 02, 2020 at 08:43:24AM +0100, Jürgen Groß wrote:
>> On 02.11.20 07:22, Andy Smith wrote:
>>> I therefore can only conclude that PVH grub doesn't like kernels
>>> with no compression. Is that expected?
>>
>> Can you boot the decompressed kernel on bare metal or in an HVM
>> domain using grub?
>
> Oh that's interesting. The decompressed kernel doesn't boot bare
> metal either (same message).
>
> The decompressed kernel is generated like this:
>
> # extract-vmlinux /boot/vmlinuz-5.4.0-52-generic > /boot/vmlinuz-5.4.0-52-generic-decompressed
>
> The decompressed kernel does boot when presented to the hypervisor
> directly, i.e. no grub, directly listed in guest config file.
>
> I don't have an HVM test setup right now so am unable to test that,
> but I suppose we can assume it will fail as the problem is in the
> standard grub loading functions.
>
> So, is this a bug? Grub should be able to boot uncompressed kernels,
> shouldn't it?

As long as the boot entry code is included in this kernel, yes.

Juergen

Re: Grub xen_pvh platform does not seem to support uncompressed kernels [ In reply to ]

bblank at thinkmo

Nov 2, 2020, 1:08 AM

Post #5 of 7 (547 views)

Permalink

On Mon, Nov 02, 2020 at 08:27:23AM +0000, Andy Smith wrote:
> The decompressed kernel is generated like this:
> # extract-vmlinux /boot/vmlinuz-5.4.0-52-generic > /boot/vmlinuz-5.4.0-52-generic-decompressed

vmlinuz contains two parts:
- the x86 boot code and
- the compressed ELF file.

extract-vmlinux extracts and decompresses the second part. But it can't
restore the boot code, which is required to make this file actually
bootable in normal conditions.

> The decompressed kernel does boot when presented to the hypervisor
> directly, i.e. no grub, directly listed in guest config file.

Because Xen extracts and boots the ELF file embedded in the file without
any help of the x86 boot code.

> So, is this a bug? Grub should be able to boot uncompressed kernels,
> shouldn't it?

No, at least not the one you gave it on x86.

Regards,
Bastian

--
It is undignified for a woman to play servant to a man who is not hers.
-- Spock, "Amok Time", stardate 3372.7

Re: Grub xen_pvh platform does not seem to support uncompressed kernels [ In reply to ]

andy at strugglers

Nov 2, 2020, 1:17 AM

Post #6 of 7 (547 views)

Permalink

Hi,

On Mon, Nov 02, 2020 at 09:43:06AM +0100, J?rgen Gro? wrote:
> On 02.11.20 09:27, Andy Smith wrote:
> >The decompressed kernel is generated like this:
> >
> ># extract-vmlinux /boot/vmlinuz-5.4.0-52-generic > /boot/vmlinuz-5.4.0-52-generic-decompressed
> >
> >The decompressed kernel does boot when presented to the hypervisor
> >directly, i.e. no grub, directly listed in guest config file.
> >
> >I don't have an HVM test setup right now so am unable to test that,
> >but I suppose we can assume it will fail as the problem is in the
> >standard grub loading functions.
> >
> >So, is this a bug? Grub should be able to boot uncompressed kernels,
> >shouldn't it?
>
> As long as the boot entry code is included in this kernel, yes.

I actually think now that the output of extract-vmlinux lacks
something and I further suspect that if I would use a vmlinux file
from within a kernel build tree it would be fine.

In which case this is not an issue for grub or xen, and only for
those of us employing hacks based around extract-vmlinux.

Apologies for the distraction!

Thanks,
Andy

Re: Grub xen_pvh platform does not seem to support uncompressed kernels [ In reply to ]

jgross at suse

Nov 2, 2020, 1:39 AM

Post #7 of 7 (547 views)

Permalink

On 02.11.20 10:17, Andy Smith wrote:
> Hi,
>
> On Mon, Nov 02, 2020 at 09:43:06AM +0100, Jürgen Groß wrote:
>> On 02.11.20 09:27, Andy Smith wrote:
>>> The decompressed kernel is generated like this:
>>>
>>> # extract-vmlinux /boot/vmlinuz-5.4.0-52-generic > /boot/vmlinuz-5.4.0-52-generic-decompressed
>>>
>>> The decompressed kernel does boot when presented to the hypervisor
>>> directly, i.e. no grub, directly listed in guest config file.
>>>
>>> I don't have an HVM test setup right now so am unable to test that,
>>> but I suppose we can assume it will fail as the problem is in the
>>> standard grub loading functions.
>>>
>>> So, is this a bug? Grub should be able to boot uncompressed kernels,
>>> shouldn't it?
>>
>> As long as the boot entry code is included in this kernel, yes.
>
> I actually think now that the output of extract-vmlinux lacks
> something and I further suspect that if I would use a vmlinux file
> from within a kernel build tree it would be fine.
>
> In which case this is not an issue for grub or xen, and only for
> those of us employing hacks based around extract-vmlinux.
>
> Apologies for the distraction!

No need to apologize.

We should really think about adding lz4 decompression support to grub
for being able to use lz4 compressed kernels with pvgrub.

Juergen