Mailing List Archive

credit2 domU freeze at "Writing SMBIOS tables ..."
On 4/27/2011 2:10 AM, George Dunlap wrote:
>> This would seem to confirm that the BUG_ON is being avoided now, but the
>> > frequent SMBIOS issue means that I won't be able to start testing credit2
>> > with a subset of customers yet.
> If you still have time, would you mind:
> * Starting a new e-mail thread (since it's a different bug), and
Done.

To recap here, my Debian Squeeze stubdom-based HVM domU is frequently
not completely starting, freezing after the output "Writing SMBIOS
tables ...".

> * Taking a 30-second trace of the VM booting and hanging with the
> following command:
> # xentrace -D -e all -T 30 [filename]& xl create [configuration]
> * Sending me the resulting trace (it will probably be 100-300MiB in size).

I am emailing you offlist with a link to the trace now. I had to use
"xm" instead of "xl" because "xl" still doesn't seem to work right with
stubdoms (a separate bug); it probably needs to wait a little longer for
the main domU to be created before creating the stubdom.

# xl create testvds4-xl.cfg
Parsing config file testvds4-xl.cfg
xc: info: VIRTUAL MEMORY ARRANGEMENT:
Loader: 0000000000100000->00000000001793f0
TOTAL: 0000000000000000->0000000031800000
ENTRY ADDRESS: 0000000000100000
xc: info: PHYSICAL MEMORY ALLOCATION:
4KB PAGES: 0x0000000000000200
2MB PAGES: 0x000000000000018b
1GB PAGES: 0x0000000000000000
xl: libxl_create.c:292: libxl__domain_make: Assertion
`!libxl_domid_valid_guest(*domid)' failed.
Aborted

-John

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: credit2 domU freeze at "Writing SMBIOS tables ..." [ In reply to ]
Is this just using credit2, or does it do the same thing under credit1?

On my box, stubdoms with a 4-vcpu guest on credit1 fail too, with this
output on the console:

xs_read_watch() -> /local/domain/3/log-throttling /local/domain/3/log-throttling
xs_read(/local/domain/3/log-throttling): ENOENT
(XEN) HVM3: HVM Loader
xs_read_watch() -> /local/domain/0/backend/console/3 be:0x13fb66:3:0x154240
(XEN) HVM3: Detected Xen v4.2-unstable
Page fault at linear address 0x10, eip 0x49d9, regs 0x3cfb3c, sp 0x0,
our_sp 0x3cfb18, code 0
(XEN) HVM3: Xenbus rings @0xfeffc000, event channel 5
Thread: main
(XEN) HVM3: System requested ROMBIOS
EIP: 49d9, EFLAGS 10202.
(XEN) HVM3: CPU speed is 1995 MHz
EBX: 00000000 ECX: 001589b0 EDX: 00000000
ESI: 0000000c EDI: 0000000c EBP: 003cfbb8 EAX: 00000000
DS: e021 ES: e021 orig_eax: ffffffff, eip: 000049d9
CS: e019 EFLAGS: 00010202 esp: 00000000 ss: 3cfba4
base is 0x3cfbb8 caller is 0x24bd3
base is 0x3cfbf8 caller is 0x9814
base is 0x3cfe88 caller is 0xe8e91
base is 0x3cfff0 caller is 0x31ad

3cfba0: 00 00 00 00 28 15 00 00 00 00 00 00 0a 00 00 00
3cfbb0: 67 96 13 00 e8 fb 3c 00 f8 fb 3c 00 d3 4b 02 00
3cfbc0: 00 00 00 00 51 e5 01 00 00 00 00 00 00 00 00 00
3cfbd0: 00 00 00 00 cb 96 13 00 78 94 00 82 03 00 00 00

49c0: c8 c1 e8 05 83 e1 1f 8b 44 85 ec d3 f8 a8 01 74
49d0: 0e 8b 43 14 89 04 24 ff d2 83 7b 10 00 75 23 8b
49e0: 53 0c 85 d2 74 1c 8b 0b 89 c8 c1 e8 05 83 e1 1f
49f0: 8b 44 85 e4 d3 f8 a8 01 74 08 8b 43 14 89 04 24
Pagetable walk from virt 10, base 371000:
L3 = 00000000b074f027 (0x372000) [offset = 0]
L2 = 00000000b074d067 (0x374000) [offset = 0]
L1 = 0000000000000000 [offset = 0]

-George

On Wed, Apr 27, 2011 at 6:24 PM, John Weekes
<lists.xen@nuclearfallout.net> wrote:
> On 4/27/2011 2:10 AM, George Dunlap wrote:
>>>
>>> This would seem to confirm that the BUG_ON is being avoided now, but the
>>> >  frequent SMBIOS issue means that I won't be able to start testing
>>> > credit2
>>> >  with a subset of customers yet.
>>
>> If you still have time, would you mind:
>> * Starting a new e-mail thread (since it's a different bug), and
>
> Done.
>
> To recap here, my Debian Squeeze stubdom-based HVM domU is frequently not
> completely starting, freezing after the output "Writing SMBIOS tables ...".
>
>> * Taking a 30-second trace of the VM booting and hanging with the
>> following command:
>>  # xentrace -D -e all -T 30 [filename]&  xl create [configuration]
>> * Sending me the resulting trace (it will probably be 100-300MiB in size).
>
> I am emailing you offlist with a link to the trace now. I had to use "xm"
> instead of "xl" because "xl" still doesn't seem to work right with stubdoms
> (a separate bug); it probably needs to wait a little longer for the main
> domU to be created before creating the stubdom.
>
> # xl create testvds4-xl.cfg
> Parsing config file testvds4-xl.cfg
> xc: info: VIRTUAL MEMORY ARRANGEMENT:
>  Loader:        0000000000100000->00000000001793f0
>  TOTAL:         0000000000000000->0000000031800000
>  ENTRY ADDRESS: 0000000000100000
> xc: info: PHYSICAL MEMORY ALLOCATION:
>  4KB PAGES: 0x0000000000000200
>  2MB PAGES: 0x000000000000018b
>  1GB PAGES: 0x0000000000000000
> xl: libxl_create.c:292: libxl__domain_make: Assertion
> `!libxl_domid_valid_guest(*domid)' failed.
> Aborted
>
> -John
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: credit2 domU freeze at "Writing SMBIOS tables ..." [ In reply to ]
On 4/28/2011 6:10 AM, George Dunlap wrote:
> Is this just using credit2, or does it do the same thing under credit1?

I just re-tested with credit1, and it seems to do the same thing with
debug=y (on xen-unstable), you're right.

On 4.1 with debug=n and credit1, which is my normal configuration, it
doesn't appear to happen, or at least didn't happen over the 10 attempts
that I just made (it could be a rare problem on that setup). I may have
to do some additional testing with 4.1 credit2, 4.1 debug=y, and
unstable debug=n, but it's good that you're seeing something
reproducible on your end, at least (and with debug=y).

> On my box, stubdoms with a 4-vcpu guest on credit1 fail too, with this
> output on the console:
>
> xs_read_watch() -> /local/domain/3/log-throttling /local/domain/3/log-throttling
> xs_read(/local/domain/3/log-throttling): ENOENT
> (XEN) HVM3: HVM Loader
> xs_read_watch() -> /local/domain/0/backend/console/3 be:0x13fb66:3:0x154240
> (XEN) HVM3: Detected Xen v4.2-unstable
> Page fault at linear address 0x10, eip 0x49d9, regs 0x3cfb3c, sp 0x0,
> our_sp 0x3cfb18, code 0
> (XEN) HVM3: Xenbus rings @0xfeffc000, event channel 5
> Thread: main
> (XEN) HVM3: System requested ROMBIOS
> EIP: 49d9, EFLAGS 10202.
> (XEN) HVM3: CPU speed is 1995 MHz
> EBX: 00000000 ECX: 001589b0 EDX: 00000000
> ESI: 0000000c EDI: 0000000c EBP: 003cfbb8 EAX: 00000000
> DS: e021 ES: e021 orig_eax: ffffffff, eip: 000049d9
> CS: e019 EFLAGS: 00010202 esp: 00000000 ss: 3cfba4
> base is 0x3cfbb8 caller is 0x24bd3
> base is 0x3cfbf8 caller is 0x9814
> base is 0x3cfe88 caller is 0xe8e91
> base is 0x3cfff0 caller is 0x31ad
>
> 3cfba0: 00 00 00 00 28 15 00 00 00 00 00 00 0a 00 00 00
> 3cfbb0: 67 96 13 00 e8 fb 3c 00 f8 fb 3c 00 d3 4b 02 00
> 3cfbc0: 00 00 00 00 51 e5 01 00 00 00 00 00 00 00 00 00
> 3cfbd0: 00 00 00 00 cb 96 13 00 78 94 00 82 03 00 00 00
>
> 49c0: c8 c1 e8 05 83 e1 1f 8b 44 85 ec d3 f8 a8 01 74
> 49d0: 0e 8b 43 14 89 04 24 ff d2 83 7b 10 00 75 23 8b
> 49e0: 53 0c 85 d2 74 1c 8b 0b 89 c8 c1 e8 05 83 e1 1f
> 49f0: 8b 44 85 e4 d3 f8 a8 01 74 08 8b 43 14 89 04 24
> Pagetable walk from virt 10, base 371000:
> L3 = 00000000b074f027 (0x372000) [offset = 0]
> L2 = 00000000b074d067 (0x374000) [offset = 0]
> L1 = 0000000000000000 [offset = 0]

I'm not seeing that one yet.

-John

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: credit2 domU freeze at "Writing SMBIOS tables ..." [ In reply to ]
Do you have "loglvl=all guest_loglvl=all" on your xen boot command-line?

-George

On Thu, 2011-04-28 at 17:25 +0100, John Weekes wrote:
> On 4/28/2011 6:10 AM, George Dunlap wrote:
> > Is this just using credit2, or does it do the same thing under credit1?
>
> I just re-tested with credit1, and it seems to do the same thing with
> debug=y (on xen-unstable), you're right.
>
> On 4.1 with debug=n and credit1, which is my normal configuration, it
> doesn't appear to happen, or at least didn't happen over the 10 attempts
> that I just made (it could be a rare problem on that setup). I may have
> to do some additional testing with 4.1 credit2, 4.1 debug=y, and
> unstable debug=n, but it's good that you're seeing something
> reproducible on your end, at least (and with debug=y).
>
> > On my box, stubdoms with a 4-vcpu guest on credit1 fail too, with this
> > output on the console:
> >
> > xs_read_watch() -> /local/domain/3/log-throttling /local/domain/3/log-throttling
> > xs_read(/local/domain/3/log-throttling): ENOENT
> > (XEN) HVM3: HVM Loader
> > xs_read_watch() -> /local/domain/0/backend/console/3 be:0x13fb66:3:0x154240
> > (XEN) HVM3: Detected Xen v4.2-unstable
> > Page fault at linear address 0x10, eip 0x49d9, regs 0x3cfb3c, sp 0x0,
> > our_sp 0x3cfb18, code 0
> > (XEN) HVM3: Xenbus rings @0xfeffc000, event channel 5
> > Thread: main
> > (XEN) HVM3: System requested ROMBIOS
> > EIP: 49d9, EFLAGS 10202.
> > (XEN) HVM3: CPU speed is 1995 MHz
> > EBX: 00000000 ECX: 001589b0 EDX: 00000000
> > ESI: 0000000c EDI: 0000000c EBP: 003cfbb8 EAX: 00000000
> > DS: e021 ES: e021 orig_eax: ffffffff, eip: 000049d9
> > CS: e019 EFLAGS: 00010202 esp: 00000000 ss: 3cfba4
> > base is 0x3cfbb8 caller is 0x24bd3
> > base is 0x3cfbf8 caller is 0x9814
> > base is 0x3cfe88 caller is 0xe8e91
> > base is 0x3cfff0 caller is 0x31ad
> >
> > 3cfba0: 00 00 00 00 28 15 00 00 00 00 00 00 0a 00 00 00
> > 3cfbb0: 67 96 13 00 e8 fb 3c 00 f8 fb 3c 00 d3 4b 02 00
> > 3cfbc0: 00 00 00 00 51 e5 01 00 00 00 00 00 00 00 00 00
> > 3cfbd0: 00 00 00 00 cb 96 13 00 78 94 00 82 03 00 00 00
> >
> > 49c0: c8 c1 e8 05 83 e1 1f 8b 44 85 ec d3 f8 a8 01 74
> > 49d0: 0e 8b 43 14 89 04 24 ff d2 83 7b 10 00 75 23 8b
> > 49e0: 53 0c 85 d2 74 1c 8b 0b 89 c8 c1 e8 05 83 e1 1f
> > 49f0: 8b 44 85 e4 d3 f8 a8 01 74 08 8b 43 14 89 04 24
> > Pagetable walk from virt 10, base 371000:
> > L3 = 00000000b074f027 (0x372000) [offset = 0]
> > L2 = 00000000b074d067 (0x374000) [offset = 0]
> > L1 = 0000000000000000 [offset = 0]
>
> I'm not seeing that one yet.
>
> -John



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: credit2 domU freeze at "Writing SMBIOS tables ..." [ In reply to ]
On 4/28/2011 9:35 AM, George Dunlap wrote:
> Do you have "loglvl=all guest_loglvl=all" on your xen boot command-line?

Yes. My line is:

kernel /boot/xen-4.2-unstable.gz dom0_mem=1500M dom0_max_vcpus=4
iommu=dom0-passthrough sched=credit loglvl=all guest_loglvl=all
com2=115200,8n1 console=com2

(I've also seen an error very similar to what that you're seeing with
the extra logging turned off, but under different circumstances, so that
one doesn't seem to require it.)

-John

> -George
>
> On Thu, 2011-04-28 at 17:25 +0100, John Weekes wrote:
>> On 4/28/2011 6:10 AM, George Dunlap wrote:
>>> Is this just using credit2, or does it do the same thing under credit1?
>> I just re-tested with credit1, and it seems to do the same thing with
>> debug=y (on xen-unstable), you're right.
>>
>> On 4.1 with debug=n and credit1, which is my normal configuration, it
>> doesn't appear to happen, or at least didn't happen over the 10 attempts
>> that I just made (it could be a rare problem on that setup). I may have
>> to do some additional testing with 4.1 credit2, 4.1 debug=y, and
>> unstable debug=n, but it's good that you're seeing something
>> reproducible on your end, at least (and with debug=y).
>>
>>> On my box, stubdoms with a 4-vcpu guest on credit1 fail too, with this
>>> output on the console:
>>>
>>> xs_read_watch() -> /local/domain/3/log-throttling /local/domain/3/log-throttling
>>> xs_read(/local/domain/3/log-throttling): ENOENT
>>> (XEN) HVM3: HVM Loader
>>> xs_read_watch() -> /local/domain/0/backend/console/3 be:0x13fb66:3:0x154240
>>> (XEN) HVM3: Detected Xen v4.2-unstable
>>> Page fault at linear address 0x10, eip 0x49d9, regs 0x3cfb3c, sp 0x0,
>>> our_sp 0x3cfb18, code 0
>>> (XEN) HVM3: Xenbus rings @0xfeffc000, event channel 5
>>> Thread: main
>>> (XEN) HVM3: System requested ROMBIOS
>>> EIP: 49d9, EFLAGS 10202.
>>> (XEN) HVM3: CPU speed is 1995 MHz
>>> EBX: 00000000 ECX: 001589b0 EDX: 00000000
>>> ESI: 0000000c EDI: 0000000c EBP: 003cfbb8 EAX: 00000000
>>> DS: e021 ES: e021 orig_eax: ffffffff, eip: 000049d9
>>> CS: e019 EFLAGS: 00010202 esp: 00000000 ss: 3cfba4
>>> base is 0x3cfbb8 caller is 0x24bd3
>>> base is 0x3cfbf8 caller is 0x9814
>>> base is 0x3cfe88 caller is 0xe8e91
>>> base is 0x3cfff0 caller is 0x31ad
>>>
>>> 3cfba0: 00 00 00 00 28 15 00 00 00 00 00 00 0a 00 00 00
>>> 3cfbb0: 67 96 13 00 e8 fb 3c 00 f8 fb 3c 00 d3 4b 02 00
>>> 3cfbc0: 00 00 00 00 51 e5 01 00 00 00 00 00 00 00 00 00
>>> 3cfbd0: 00 00 00 00 cb 96 13 00 78 94 00 82 03 00 00 00
>>>
>>> 49c0: c8 c1 e8 05 83 e1 1f 8b 44 85 ec d3 f8 a8 01 74
>>> 49d0: 0e 8b 43 14 89 04 24 ff d2 83 7b 10 00 75 23 8b
>>> 49e0: 53 0c 85 d2 74 1c 8b 0b 89 c8 c1 e8 05 83 e1 1f
>>> 49f0: 8b 44 85 e4 d3 f8 a8 01 74 08 8b 43 14 89 04 24
>>> Pagetable walk from virt 10, base 371000:
>>> L3 = 00000000b074f027 (0x372000) [offset = 0]
>>> L2 = 00000000b074d067 (0x374000) [offset = 0]
>>> L1 = 0000000000000000 [offset = 0]
>> I'm not seeing that one yet.
>>
>> -John
>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel