Mailing List Archive: evtchn_upcall

evtchn_upcall_mask for PV-on-HVM

Nov 30, 2006, 12:05 AM

Post #1 of 5 (61 views)

We seem to find an interesting bug, but not for sure.

Each time before xen returns to hvm domain, it checks
local_events_need_delivery to see whether any events pending and
if yes, inject event by callback_irq into the virtual interrupt
controller.
However, the interesting point is, we never found any place, either in
xen or in PV drivers, to clear evtchn_upcall_mask at any time. The
initial value of evtchn_upcall_mask is 1 which makes
local_events_need_delivery always return 0.

One possible reason why PV drivers still works in HVM is that
platform pci device happens to share irq line with another pci device
within Qemu. In that case, the irq from other pci device may still
bring event back into PV drivers. But can this condition always
promise?

Anyway, we'd like to know whether this is a real bug. If yes, it may
have some impact on performance, and then people may clear the
evtchn_upcall_mask either at domain creation time or at callback_irq
setting time. In any case, this field does nothing for HVM domain since
the latter already has 'rflags.if' for same purpose.

Thanks,
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: evtchn_upcall_mask for PV-on-HVM [ In reply to ]

keir at xensource

Nov 30, 2006, 2:09 AM

Post #2 of 5 (56 views)

Permalink

On 30/11/06 07:05, "Tian, Kevin" <kevin.tian@intel.com> wrote:

> Anyway, we'd like to know whether this is a real bug. If yes, it may
> have some impact on performance, and then people may clear the
> evtchn_upcall_mask either at domain creation time or at callback_irq
> setting time. In any case, this field does nothing for HVM domain since
> the latter already has 'rflags.if' for same purpose.

Local_events_need_delivery() should check for pending APIC/PIC interrupts on
HVM. That'll fix the block() wakeup-waiting race too.

-- Keir

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: evtchn_upcall_mask for PV-on-HVM [ In reply to ]

kevin.tian at intel

Nov 30, 2006, 6:56 AM

Post #3 of 5 (56 views)

Permalink

>From: Keir Fraser [mailto:keir@xensource.com]
>Sent: 2006Äê11ÔÂ30ÈÕ 17:09
>
>On 30/11/06 07:05, "Tian, Kevin" <kevin.tian@intel.com> wrote:
>
>> Anyway, we'd like to know whether this is a real bug. If yes, it may
>> have some impact on performance, and then people may clear the
>> evtchn_upcall_mask either at domain creation time or at callback_irq
>> setting time. In any case, this field does nothing for HVM domain since
>> the latter already has 'rflags.if' for same purpose.
>
>Local_events_need_delivery() should check for pending APIC/PIC
>interrupts on
>HVM. That'll fix the block() wakeup-waiting race too.
>
> -- Keir

Hi, Keir,
Sorry that I didn't quite catch you. Did you mean check for
pending interrupts for same gsi line (used as callback irq)? Whether
to actually pending an irq instance is decided by APIC/PIC themselves,
right? Vmx_intr_assit just needs to check status of
evtchn_upcall_pending which serves as a virtual wire state. BTW,
what's the block() wakeup-waiting race above? :-)

Thanks,
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: evtchn_upcall_mask for PV-on-HVM [ In reply to ]

keir at xensource

Nov 30, 2006, 7:50 AM

Post #4 of 5 (56 views)

Permalink

On 30/11/06 13:56, "Tian, Kevin" <kevin.tian@intel.com> wrote:

>> Local_events_need_delivery() should check for pending APIC/PIC
>> interrupts on
>> HVM. That'll fix the block() wakeup-waiting race too.
>>
>> -- Keir
>
> Hi, Keir,
> Sorry that I didn't quite catch you. Did you mean check for
> pending interrupts for same gsi line (used as callback irq)? Whether
> to actually pending an irq instance is decided by APIC/PIC themselves,
> right? Vmx_intr_assit just needs to check status of
> evtchn_upcall_pending which serves as a virtual wire state. BTW,
> what's the block() wakeup-waiting race above? :-)

I mean that the evtchn_upcall_pending flag is not the master flag indicating
whether there are pending interrupts for an HVM guest. So it is not the
correct thing to test in local_events_need_delivery(). The correct thing to
test is whether there are any interrupts pending from the PIC/IOAPIC.

The wakeup-waiting race is that a VCPU may block just when, for example, the
qemu-dm rtl8139 device model is asserting an interrupt. It will try to
vcpu_kick() and send an IPI to the CPU that is running the HVM VCPU.
However, before that IPI is received the other CPU may block the HVM VCPU
(after checking for local_events_need_delivery() which will in this case
return 0). So the rtl8139 interrupt does not prevent the HVM VCPU from
blocking, which is a bug.

- Keir

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: evtchn_upcall_mask for PV-on-HVM [ In reply to ]

kevin.tian at intel

Nov 30, 2006, 7:29 PM

Post #5 of 5 (56 views)

Permalink

>From: Keir Fraser [mailto:keir@xensource.com]
>Sent: 2006Äê11ÔÂ30ÈÕ 22:51
>> what's the block() wakeup-waiting race above? :-)
>
>I mean that the evtchn_upcall_pending flag is not the master flag
>indicating
>whether there are pending interrupts for an HVM guest. So it is not the
>correct thing to test in local_events_need_delivery(). The correct thing to
>test is whether there are any interrupts pending from the PIC/IOAPIC.
>
>The wakeup-waiting race is that a VCPU may block just when, for
>example, the
>qemu-dm rtl8139 device model is asserting an interrupt. It will try to
>vcpu_kick() and send an IPI to the CPU that is running the HVM VCPU.
>However, before that IPI is received the other CPU may block the HVM
>VCPU
>(after checking for local_events_need_delivery() which will in this case
>return 0). So the rtl8139 interrupt does not prevent the HVM VCPU from
>blocking, which is a bug.
>
> - Keir

OK, I see. That's a real performance bug..

Thanks for explanation. :-)

Thanks,
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel