Mailing List Archive

Running Xen on RaspberryPi-3-B
Hello Xen,


- I am trying to run Xen-4.6.5 on RaspberryPi-3-B
- Can someone please guide if this is possible straight away or some
porting needs to be done?
- In case porting is required, it would be great if some direction is
provided on the same like what is the issue, what needs to be done et al

Thanks in advance and looking forward to the reply.

Regards,
Rajagopal
Re: Running Xen on RaspberryPi-3-B [ In reply to ]
On 18/07/2019 19:48, Rajagopal Aravindan wrote:
> Hello Xen,

Hi,

>
> * I am trying to run Xen-4.6.5 on RaspberryPi-3-B

Xen 4.6 was released 3 and half years ago. The Arm port mature quite a lot since
then. So you should at least move to Xen 4.12 if not unstable for a new port.

> * Can someone please guide if this is possible straight away or some porting
> needs to be done?
> * In case porting is required, it would be great if some direction is provided
> on the same like what is the issue, what needs to be done et al

The RPI 3 is still using the Broadcom interrupt controller. It does
not have virtualization extension provided and not based on the GIC
specification. So porting Xen on the RPI 3 is probably going to be quite consequent.

Instead I would recommend to look at the new RPI4. It contains a GIC and the
port should be easier.

Cheers,

>
> Thanks in advance and looking forward to the reply.
>
> Regards,
> Rajagopal
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-users
>

--
Julien Grall

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users
Re: Running Xen on RaspberryPi-3-B [ In reply to ]
Hi,

Please keep xen-users in CC.

On 7/20/19 4:39 AM, Rajagopal Aravindan wrote:
> Hello Julien,
>
> Appreciate your reply, thanks.
>
> > Instead I would recommend to look at the new RPI4.
> Sure, will give it a look-in.
>
> >It contains a GIC and the port should be easier.
> Does that mean Xen won't run straight away and some porting needs to be
> done ?

I bought one recently but I haven't yet had the chance to see how much
effort is required to getting Xen running on it.

For some general guidance on porting a new board to Xen, please see [1].

> Again, any directions would be helpful, on where/what to start with.
>
> > So porting Xen on the RPI 3 is probably going to be quite consequent.
> Sounds like Xen & its guests can be made to run on RPi3, assuming
> information on BRCM GIC is available on the net.

The RPI 3 does not use the GIC so I am not sure what you found online...

> If yes,
>
> * Getting Xen to run on RPi3 would be a very good case study for
> understanding its internals, for Xen beginner like me.

I don't think trying to port Xen on the RPI 3 is a good way to
understand the internals. It would be best to look at platform that are
closer to Xen requirements...

> * Should anyone manage to get it running on RPi3  ...
> o Would Xen be willing to take it upstream ?

Not really. See below...

> o Or are there reservations about it, since it may NOT be useful
> on other hardware and hence not worth maintaining ?

AFAIK, the interrupt controller used by the RPI 3 does not support
virtualization. So this is quite departure from the minimal requirement
used by Xen.

There are various way to make it working on Xen:
1) Emulating the interrupt controller
2) Use a PV interface as used by x86

In both cases, the work is going to be consequent with very limited
benefits to other platforms.

Cheers,

[1]
https://www.slideshare.net/xen_com_mgr/xpds16-porting-xen-on-arm-to-a-new-soc-julien-grall-arm

--
Julien Grall

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users
Re: Running Xen on RaspberryPi-3-B [ In reply to ]
On 20/07/2019 17:39, Julien Grall wrote:

Hi,

> On 7/20/19 4:39 AM, Rajagopal Aravindan wrote:
>> Hello Julien,
>>
>> Appreciate your reply, thanks.
>>
>>  > Instead I would recommend to look at the new RPI4.
>> Sure, will give it a look-in.
>>
>>  >It contains a GIC and the port should be easier.
>> Does that mean Xen won't run straight away and some porting needs to
>> be done ?

Not much needed on the Xen side, I got away with a small patch for the
UART. Using CONFIG_EARLY_PRINTK=8250,0xfe215040,2 should give you some
output even without it:
https://pastebin.com/uZqQ2XPm

You need to have some more decent firmware than what is shipped, though.

> I bought one recently but I haven't yet had the chance to see how much
> effort is required to getting Xen running on it.
>
> For some general guidance on porting a new board to Xen, please see [1].
>
>> Again, any directions would be helpful, on where/what to start with.
>>
>>  > So porting Xen on the RPI 3 is probably going to be quite consequent.
>> Sounds like Xen & its guests can be made to run on RPi3, assuming
>> information on BRCM GIC is available on the net.
>
> The RPI 3 does not use the GIC so I am not sure what you found online...
>
>> If yes,
>>
>>   * Getting Xen to run on RPi3 would be a very good case study for
>>     understanding its internals, for Xen beginner like me.
>
> I don't think trying to port Xen on the RPI 3 is a good way to
> understand the internals. It would be best to look at platform that are
> closer to Xen requirements...
>
>>   * Should anyone manage to get it running on RPi3  ...
>>       o Would Xen be willing to take it upstream ?
>
> Not really. See below...
>
>>       o Or are there reservations about it, since it may NOT be useful
>>         on other hardware and hence not worth maintaining ?
>
> AFAIK, the interrupt controller used by the RPI 3 does not support
> virtualization. So this is quite departure from the minimal requirement
> used by Xen.
>
> There are various way to make it working on Xen:
>    1) Emulating the interrupt controller
>    2) Use a PV interface as used by x86
>
> In both cases, the work is going to be consequent with very limited
> benefits to other platforms.

I agree on this, adding support for the RPi3 interrupt controller would
be quite painful, and also needs to be integrated with the existing
(V)GIC support, which has the potential to introduce regressions.
Plus you need to amend the arch timer code, as this is integrated with
the GIC for virtualisation purposes.
AFAIK the RPi3 is one of the few (if not the only one) ARM64 platform
without a GIC, and given that the RPi4 fixes this, I don't see much hope
for upstreaming this kind of change.

If you are desperate for (hardware) virtualisation, KVM supports guests
without a (hardware) GIC (via the QEMU emulation), although this is
slower and less tested. Or you just go for some container solution.

Cheers,
Andre.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users
Re: Running Xen on RaspberryPi-3-B [ In reply to ]
Hi,

On 7/21/19 1:14 AM, André Przywara wrote:
> On 20/07/2019 17:39, Julien Grall wrote:
>
> Hi,
>
>> On 7/20/19 4:39 AM, Rajagopal Aravindan wrote:
>>> Hello Julien,
>>>
>>> Appreciate your reply, thanks.
>>>
>>>  > Instead I would recommend to look at the new RPI4.
>>> Sure, will give it a look-in.
>>>
>>>  >It contains a GIC and the port should be easier.
>>> Does that mean Xen won't run straight away and some porting needs to
>>> be done ?
>
> Not much needed on the Xen side, I got away with a small patch for the
> UART. Using CONFIG_EARLY_PRINTK=8250,0xfe215040,2 should give you some
> output even without it:
> https://pastebin.com/uZqQ2XPm

I was about to say that we don't deal with page-aligned base address at
the moment. But it looks like it works, however I think I need to
rethink my "head.S rework" series as I assumed the earlyprink base
address is always page-aligned.

Also, because the address is not page-aligned, I guess this means a page
may hold multiple devices. So I think we would need to blacklist
anything in that page or proxy it in Xen.

>
> You need to have some more decent firmware than what is shipped, though.

Do you have a link to a more decent one? :)

Cheers,

--
Julien Grall

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users
Re: Running Xen on RaspberryPi-3-B [ In reply to ]
Hello Andre,

On Sun 21 Jul, 2019, 5:45 AM André Przywara, <andre.przywara@arm.com> wrote:

> On 20/07/2019 17:39, Julien Grall wrote:
>
> Hi,
>
> > On 7/20/19 4:39 AM, Rajagopal Aravindan wrote:
> >> Hello Julien,
> >>
> >> Appreciate your reply, thanks.
> >>
> >> > Instead I would recommend to look at the new RPI4.
> >> Sure, will give it a look-in.
> >>
> >> >It contains a GIC and the port should be easier.
> >> Does that mean Xen won't run straight away and some porting needs to
> >> be done ?
>
> Not much needed on the Xen side, I got away with a small patch for the
> UART. Using CONFIG_EARLY_PRINTK=8250,0xfe215040,2 should give you some
> output even without it:
> https://pastebin.com/uZqQ2XPm
>
> You need to have some more decent firmware than what is shipped, though.
>
> > I bought one recently but I haven't yet had the chance to see how much
> > effort is required to getting Xen running on it.
> >
> > For some general guidance on porting a new board to Xen, please see [1].
> >
> >> Again, any directions would be helpful, on where/what to start with.
> >>
> >> > So porting Xen on the RPI 3 is probably going to be quite consequent.
> >> Sounds like Xen & its guests can be made to run on RPi3, assuming
> >> information on BRCM GIC is available on the net.
> >
> > The RPI 3 does not use the GIC so I am not sure what you found online...
> >
> >> If yes,
> >>
> >> * Getting Xen to run on RPi3 would be a very good case study for
> >> understanding its internals, for Xen beginner like me.
> >
> > I don't think trying to port Xen on the RPI 3 is a good way to
> > understand the internals. It would be best to look at platform that are
> > closer to Xen requirements...
> >
> >> * Should anyone manage to get it running on RPi3 ...
> >> o Would Xen be willing to take it upstream ?
> >
> > Not really. See below...
> >
> >> o Or are there reservations about it, since it may NOT be useful
> >> on other hardware and hence not worth maintaining ?
> >
> > AFAIK, the interrupt controller used by the RPI 3 does not support
> > virtualization. So this is quite departure from the minimal requirement
> > used by Xen.
> >
> > There are various way to make it working on Xen:
> > 1) Emulating the interrupt controller
> > 2) Use a PV interface as used by x86
> >
> > In both cases, the work is going to be consequent with very limited
> > benefits to other platforms.
>
> I agree on this, adding support for the RPi3 interrupt controller would
> be quite painful, and also needs to be integrated with the existing
> (V)GIC support, which has the potential to introduce regressions.
> Plus you need to amend the arch timer code, as this is integrated with
> the GIC for virtualisation purposes.
> AFAIK the RPi3 is one of the few (if not the only one) ARM64 platform
> without a GIC, and given that the RPi4 fixes this, I don't see much hope
> for upstreaming this kind of change.
>
> If you are desperate for (hardware) virtualisation, KVM supports guests
> without a (hardware) GIC (via the QEMU emulation), although this is
> slower and less tested. Or you just go for some container solution.
>
> Cheers,
> Andre.
>

I built Xen with your CONFIG_EARLY_PRINTK but couldn't meet with success.

That brings me to your point about decent firmware. I am using u-boot 2019.
Is that good enough?

Thanks,
Rajagopal

>
Re: Running Xen on RaspberryPi-3-B [ In reply to ]
Hello André,

On Tue, Jul 23, 2019 at 1:57 AM Rajagopal Aravindan <
a.rajagopal.81@gmail.com> wrote:

> Hello Andre,
>
> On Sun 21 Jul, 2019, 5:45 AM André Przywara, <andre.przywara@arm.com>
> wrote:
>
>> On 20/07/2019 17:39, Julien Grall wrote:
>>
>> Hi,
>>
>> > On 7/20/19 4:39 AM, Rajagopal Aravindan wrote:
>> >> Hello Julien,
>> >>
>> >> Appreciate your reply, thanks.
>> >>
>> >> > Instead I would recommend to look at the new RPI4.
>> >> Sure, will give it a look-in.
>> >>
>> >> >It contains a GIC and the port should be easier.
>> >> Does that mean Xen won't run straight away and some porting needs to
>> >> be done ?
>>
>> Not much needed on the Xen side, I got away with a small patch for the
>> UART. Using CONFIG_EARLY_PRINTK=8250,0xfe215040,2 should give you some
>> output even without it:
>> https://pastebin.com/uZqQ2XPm
>>
>> You need to have some more decent firmware than what is shipped, though.
>>
>> > I bought one recently but I haven't yet had the chance to see how much
>> > effort is required to getting Xen running on it.
>> >
>> > For some general guidance on porting a new board to Xen, please see [1].
>> >
>> >> Again, any directions would be helpful, on where/what to start with.
>> >>
>> >> > So porting Xen on the RPI 3 is probably going to be quite
>> consequent.
>> >> Sounds like Xen & its guests can be made to run on RPi3, assuming
>> >> information on BRCM GIC is available on the net.
>> >
>> > The RPI 3 does not use the GIC so I am not sure what you found online...
>> >
>> >> If yes,
>> >>
>> >> * Getting Xen to run on RPi3 would be a very good case study for
>> >> understanding its internals, for Xen beginner like me.
>> >
>> > I don't think trying to port Xen on the RPI 3 is a good way to
>> > understand the internals. It would be best to look at platform that are
>> > closer to Xen requirements...
>> >
>> >> * Should anyone manage to get it running on RPi3 ...
>> >> o Would Xen be willing to take it upstream ?
>> >
>> > Not really. See below...
>> >
>> >> o Or are there reservations about it, since it may NOT be useful
>> >> on other hardware and hence not worth maintaining ?
>> >
>> > AFAIK, the interrupt controller used by the RPI 3 does not support
>> > virtualization. So this is quite departure from the minimal requirement
>> > used by Xen.
>> >
>> > There are various way to make it working on Xen:
>> > 1) Emulating the interrupt controller
>> > 2) Use a PV interface as used by x86
>> >
>> > In both cases, the work is going to be consequent with very limited
>> > benefits to other platforms.
>>
>> I agree on this, adding support for the RPi3 interrupt controller would
>> be quite painful, and also needs to be integrated with the existing
>> (V)GIC support, which has the potential to introduce regressions.
>> Plus you need to amend the arch timer code, as this is integrated with
>> the GIC for virtualisation purposes.
>> AFAIK the RPi3 is one of the few (if not the only one) ARM64 platform
>> without a GIC, and given that the RPi4 fixes this, I don't see much hope
>> for upstreaming this kind of change.
>>
>> If you are desperate for (hardware) virtualisation, KVM supports guests
>> without a (hardware) GIC (via the QEMU emulation), although this is
>> slower and less tested. Or you just go for some container solution.
>>
>> Cheers,
>> Andre.
>>
>
> I built Xen with your CONFIG_EARLY_PRINTK but couldn't meet with success.
>
> That brings me to your point about decent firmware. I am using u-boot
> 2019. Is that good enough?
>
> Thanks,
> Rajagopal
>

Could you also please share the fdt set /chosen for xen bootargs, for
RPi-4, if u-boot 2019 is good enough ?

Thanks,
Rajagopal
Re: Running Xen on RaspberryPi-3-B [ In reply to ]
On 23/07/2019 19:09, Rajagopal Aravindan wrote:

Hi

> On Tue, Jul 23, 2019 at 1:57 AM Rajagopal Aravindan
> <a.rajagopal.81@gmail.com <mailto:a.rajagopal.81@gmail.com>> wrote:
>
> Hello Andre,
>
> On Sun 21 Jul, 2019, 5:45 AM André Przywara, <andre.przywara@arm.com
> <mailto:andre.przywara@arm.com>> wrote:
>
> On 20/07/2019 17:39, Julien Grall wrote:
>
> Hi,
>
> > On 7/20/19 4:39 AM, Rajagopal Aravindan wrote:
> >> Hello Julien,
> >>
> >> Appreciate your reply, thanks.
> >>
> >>  > Instead I would recommend to look at the new RPI4.
> >> Sure, will give it a look-in.
> >>
> >>  >It contains a GIC and the port should be easier.
> >> Does that mean Xen won't run straight away and some porting
> needs to
> >> be done ?
>
> Not much needed on the Xen side, I got away with a small patch
> for the
> UART. Using CONFIG_EARLY_PRINTK=8250,0xfe215040,2 should give
> you some
> output even without it:
> https://pastebin.com/uZqQ2XPm
>
> You need to have some more decent firmware than what is shipped,
> though.
>
> > I bought one recently but I haven't yet had the chance to see
> how much
> > effort is required to getting Xen running on it.
> >
> > For some general guidance on porting a new board to Xen,
> please see [1].
> >
> >> Again, any directions would be helpful, on where/what to
> start with.
> >>
> >>  > So porting Xen on the RPI 3 is probably going to be quite
> consequent.
> >> Sounds like Xen & its guests can be made to run on RPi3, assuming
> >> information on BRCM GIC is available on the net.
> >
> > The RPI 3 does not use the GIC so I am not sure what you found
> online...
> >
> >> If yes,
> >>
> >>   * Getting Xen to run on RPi3 would be a very good case
> study for
> >>     understanding its internals, for Xen beginner like me.
> >
> > I don't think trying to port Xen on the RPI 3 is a good way to
> > understand the internals. It would be best to look at platform
> that are
> > closer to Xen requirements...
> >
> >>   * Should anyone manage to get it running on RPi3  ...
> >>       o Would Xen be willing to take it upstream ?
> >
> > Not really. See below...
> >
> >>       o Or are there reservations about it, since it may NOT
> be useful
> >>         on other hardware and hence not worth maintaining ?
> >
> > AFAIK, the interrupt controller used by the RPI 3 does not support
> > virtualization. So this is quite departure from the minimal
> requirement
> > used by Xen.
> >
> > There are various way to make it working on Xen:
> >    1) Emulating the interrupt controller
> >    2) Use a PV interface as used by x86
> >
> > In both cases, the work is going to be consequent with very
> limited
> > benefits to other platforms.
>
> I agree on this, adding support for the RPi3 interrupt
> controller would
> be quite painful, and also needs to be integrated with the existing
> (V)GIC support, which has the potential to introduce regressions.
> Plus you need to amend the arch timer code, as this is
> integrated with
> the GIC for virtualisation purposes.
> AFAIK the RPi3 is one of the few (if not the only one) ARM64
> platform
> without a GIC, and given that the RPi4 fixes this, I don't see
> much hope
> for upstreaming this kind of change.
>
> If you are desperate for (hardware) virtualisation, KVM supports
> guests
> without a (hardware) GIC (via the QEMU emulation), although this is
> slower and less tested. Or you just go for some container solution.
>
> Cheers,
> Andre.
>
>
> I built Xen with your CONFIG_EARLY_PRINTK but couldn't meet with
> success.
>
> That brings me to your point about decent firmware. I am using
> u-boot 2019. Is that good enough?
>
> Thanks,
> Rajagopal
>
>
>  Could you also please share the fdt set /chosen for xen bootargs, for
> RPi-4, if u-boot 2019 is good enough ?

Not sure what you mean with "U-Boot 2019"? Can't find such thing, it's
v2019.07, at best. But AFAIK RPi4 support has been just seen on the
list, I don't see it merged in there.

So for the firmware I am just using some Trusted Firmware port I came up
with:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/1629

You would also need Stefano's series for dealing with reserved-memory,
otherwise Xen will overwrite ATF. Haven't tested this yet, though.

Cheers,
Andre.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-users