Mailing List Archive: [xen-unstable test] 10231: regressions

[xen-unstable test] 10231: regressions - FAIL

Dec 1, 2011, 8:23 PM

Post #1 of 3 (153 views)

flight 10231 xen-unstable real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/10231/

Regressions :-(

Tests which did not succeed and are blocking:
test-amd64-amd64-xl-win 8 guest-saverestore fail REGR. vs. 10201
test-amd64-i386-win-vcpus1 8 guest-saverestore fail REGR. vs. 10201
test-amd64-i386-win 8 guest-saverestore fail REGR. vs. 10201
test-i386-i386-win 8 guest-saverestore fail REGR. vs. 10201
test-amd64-i386-xl-win-vcpus1 8 guest-saverestore fail REGR. vs. 10201
test-i386-i386-xl-win 8 guest-saverestore fail REGR. vs. 10201
test-amd64-amd64-win 8 guest-saverestore fail REGR. vs. 10201

Tests which did not succeed, but are not blocking,
including regressions (tests previously passed) regarded as allowable:
test-amd64-i386-xend-winxpsp3 7 windows-install fail never pass
test-amd64-amd64-xl-win7-amd64 7 windows-install fail never pass
test-amd64-amd64-xl-winxpsp3 7 windows-install fail never pass
test-amd64-i386-xl-winxpsp3-vcpus1 7 windows-install fail never pass
test-amd64-i386-xl-win7-amd64 7 windows-install fail never pass
test-i386-i386-xl-winxpsp3 7 windows-install fail never pass
test-amd64-i386-rhel6hvm-intel 9 guest-start.2 fail never pass
test-amd64-i386-rhel6hvm-amd 9 guest-start.2 fail never pass
test-amd64-amd64-xl-pcipt-intel 9 guest-start fail never pass

version targeted for testing:
xen 3f815406feb2
baseline version:
xen 62ff6a318c5d

------------------------------------------------------------
People who touched revisions under test:
Adin Scannell <adin@scannell.ca>
Andres Lagar-Cavilla <andres@lagarcavilla.org>
Anthony PERARD <anthony.perard@citrix.com>
Brendan Cully <brendan@cs.ubc.ca>
Daniel De Graaf <dgdegra@tycho.nsa.gov>
Ian Campbell <ian.campbell@citrix.com>
Ian Jackson <ian.jackson.citrix.com>
Ian Jackson <ian.jackson@eu.citrix.com>
Jan Beulich <jbeulich@suse.com>
Jean Guyader <jean.guyader@eu.citrix.com>
Jonathan Davies <jonathan.davies@citrix.com>
juergen.gross@ts.fujitsu.com
Keir Fraser <keir@xen.org>
Liu, Jinsong <jinsong.liu@intel.com>
Olaf Hering <olaf@aepfle.de>
Philipp Hahn <hahn@univention.de>
Shriram Rajagopalan <rshriram@cs.ubc.ca>
Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tim Deegan <tim@xen.org>
------------------------------------------------------------

jobs:
build-amd64 pass
build-i386 pass
build-amd64-oldkern pass
build-i386-oldkern pass
build-amd64-pvops pass
build-i386-pvops pass
test-amd64-amd64-xl pass
test-amd64-i386-xl pass
test-i386-i386-xl pass
test-amd64-i386-rhel6hvm-amd fail
test-amd64-amd64-xl-win7-amd64 fail
test-amd64-i386-xl-win7-amd64 fail
test-amd64-i386-xl-credit2 pass
test-amd64-amd64-xl-pcipt-intel fail
test-amd64-i386-rhel6hvm-intel fail
test-amd64-i386-xl-multivcpu pass
test-amd64-amd64-pair pass
test-amd64-i386-pair pass
test-i386-i386-pair pass
test-amd64-amd64-pv pass
test-amd64-i386-pv pass
test-i386-i386-pv pass
test-amd64-amd64-xl-sedf pass
test-amd64-i386-win-vcpus1 fail
test-amd64-i386-xl-win-vcpus1 fail
test-amd64-i386-xl-winxpsp3-vcpus1 fail
test-amd64-amd64-win fail
test-amd64-i386-win fail
test-i386-i386-win fail
test-amd64-amd64-xl-win fail
test-i386-i386-xl-win fail
test-amd64-i386-xend-winxpsp3 fail
test-amd64-amd64-xl-winxpsp3 fail
test-i386-i386-xl-winxpsp3 fail

------------------------------------------------------------
sg-report-flight on woking.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images

Logs, config files, etc. are available at
http://www.chiark.greenend.org.uk/~xensrcts/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary

Not pushing.

(No revision log; it would be 1073 lines long.)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [xen-unstable test] 10231: regressions - FAIL [ In reply to ]

Ian.Campbell at citrix

Dec 2, 2011, 2:48 AM

Post #2 of 3 (151 views)

Permalink

On Fri, 2011-12-02 at 04:23 +0000, xen.org wrote:
> flight 10231 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/10231/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking:
> test-amd64-amd64-xl-win 8 guest-saverestore fail REGR. vs. 10201
> test-amd64-i386-win-vcpus1 8 guest-saverestore fail REGR. vs. 10201
> test-amd64-i386-win 8 guest-saverestore fail REGR. vs. 10201
> test-i386-i386-win 8 guest-saverestore fail REGR. vs. 10201
> test-amd64-i386-xl-win-vcpus1 8 guest-saverestore fail REGR. vs. 10201
> test-i386-i386-xl-win 8 guest-saverestore fail REGR. vs. 10201
> test-amd64-amd64-win 8 guest-saverestore fail REGR. vs. 10201

The xl ones are all failing with:
2011-12-02 04:17:52 Z executing ssh ... root@10.80.249.56 xl save win.guest.osstest image
Saving to image new xl format (info 0x0/0x0/625)
libxl: error: libxl_exec.c:257:libxl__wait_for_offspring: Device Model not ready

while the xm ones are failing with:
2011-12-02 03:22:20 Z executing ssh ... root@10.80.250.28 xm save win.guest.osstest image
Usage: xm save [-c] <Domain> <CheckpointFile>

Save a domain state to restore later.
-c, --checkpoint Leave domain running after creating
snapshot

Error: Timed out waiting for device model action

In the xl case the test harness doesn't appear to have noticed the
failure. Might be bad error handling in xl or a bug in the harness. (my
money would be on the former).

I can't see anything in the qemu logs for the xm case but in one xl case
there is:
Error -22 while loading savevm file '/var/lib/xen/qemu-resume.5'

In the xm case xend logs say:
011-12-02 03:22:22 2185] WARNING (image:552) domain migrating-win.guest.osstest: device model failure: pid 4278: malfunctioning (closed sentinel), killed; see /var/log/xen/qemu-dm-win.guest.osstest.log
[2011-12-02 03:22:32 2185] INFO (XendCheckpoint:423) xc: error: Suspend request failed: Internal error
[2011-12-02 03:22:32 2185] INFO (XendCheckpoint:423) xc: error: Domain appears not to have suspended: Internal error

but as I say nothing interesting appears in qemu-dm...log
[...]
> version targeted for testing:
> xen 3f815406feb2
> baseline version:
> xen 62ff6a318c5d

changeset: 24332:6b26fb894ca0
user: Ian Jackson <Ian.Jackson@eu.citrix.com>
date: Thu Dec 01 18:26:45 2011 +0000
summary: Update QEMU_TAG

is in there but appears to represent a bunch of passthrough updates.
There also a bunch of Remus compression stuff which appears to touch
non-Remus suspend/resume code in libxc too, that's a plausible candidate
for the cause, I think

There is also a bunch of paging stuff, which I guess/hope is harmless if
paging is not in use? Tim, Andreas?.

Otherwise it appears to all be docs and emulator changes which all seem
less likely. There's some libxl stuff but you wouldn't expect that to
hurt xend...

I guess the bisector is on the case.

>
> ------------------------------------------------------------
> People who touched revisions under test:
> Adin Scannell <adin@scannell.ca>
> Andres Lagar-Cavilla <andres@lagarcavilla.org>
> Anthony PERARD <anthony.perard@citrix.com>
> Brendan Cully <brendan@cs.ubc.ca>
> Daniel De Graaf <dgdegra@tycho.nsa.gov>
> Ian Campbell <ian.campbell@citrix.com>
> Ian Jackson <ian.jackson.citrix.com>
> Ian Jackson <ian.jackson@eu.citrix.com>
> Jan Beulich <jbeulich@suse.com>
> Jean Guyader <jean.guyader@eu.citrix.com>
> Jonathan Davies <jonathan.davies@citrix.com>
> juergen.gross@ts.fujitsu.com
> Keir Fraser <keir@xen.org>
> Liu, Jinsong <jinsong.liu@intel.com>
> Olaf Hering <olaf@aepfle.de>
> Philipp Hahn <hahn@univention.de>
> Shriram Rajagopalan <rshriram@cs.ubc.ca>
> Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Tim Deegan <tim@xen.org>
> ------------------------------------------------------------
>
> jobs:
> build-amd64 pass
> build-i386 pass
> build-amd64-oldkern pass
> build-i386-oldkern pass
> build-amd64-pvops pass
> build-i386-pvops pass
> test-amd64-amd64-xl pass
> test-amd64-i386-xl pass
> test-i386-i386-xl pass
> test-amd64-i386-rhel6hvm-amd fail
> test-amd64-amd64-xl-win7-amd64 fail
> test-amd64-i386-xl-win7-amd64 fail
> test-amd64-i386-xl-credit2 pass
> test-amd64-amd64-xl-pcipt-intel fail
> test-amd64-i386-rhel6hvm-intel fail
> test-amd64-i386-xl-multivcpu pass
> test-amd64-amd64-pair pass
> test-amd64-i386-pair pass
> test-i386-i386-pair pass
> test-amd64-amd64-pv pass
> test-amd64-i386-pv pass
> test-i386-i386-pv pass
> test-amd64-amd64-xl-sedf pass
> test-amd64-i386-win-vcpus1 fail
> test-amd64-i386-xl-win-vcpus1 fail
> test-amd64-i386-xl-winxpsp3-vcpus1 fail
> test-amd64-amd64-win fail
> test-amd64-i386-win fail
> test-i386-i386-win fail
> test-amd64-amd64-xl-win fail
> test-i386-i386-xl-win fail
> test-amd64-i386-xend-winxpsp3 fail
> test-amd64-amd64-xl-winxpsp3 fail
> test-i386-i386-xl-winxpsp3 fail
>
>
> ------------------------------------------------------------
> sg-report-flight on woking.cam.xci-test.com
> logs: /home/xc_osstest/logs
> images: /home/xc_osstest/images
>
> Logs, config files, etc. are available at
> http://www.chiark.greenend.org.uk/~xensrcts/logs
>
> Test harness code can be found at
> http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
>
>
> Not pushing.
>
> (No revision log; it would be 1073 lines long.)
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: [xen-unstable test] 10231: regressions - FAIL [ In reply to ]

andres at lagarcavilla

Dec 2, 2011, 7:40 AM

Post #3 of 3 (168 views)

Permalink

> On Fri, 2011-12-02 at 04:23 +0000, xen.org wrote:
>> flight 10231 xen-unstable real [real]
>> http://www.chiark.greenend.org.uk/~xensrcts/logs/10231/
>>
>> Regressions :-(
>>
>> Tests which did not succeed and are blocking:
>> test-amd64-amd64-xl-win 8 guest-saverestore fail REGR.
>> vs. 10201
>> test-amd64-i386-win-vcpus1 8 guest-saverestore fail REGR.
>> vs. 10201
>> test-amd64-i386-win 8 guest-saverestore fail REGR.
>> vs. 10201
>> test-i386-i386-win 8 guest-saverestore fail REGR.
>> vs. 10201
>> test-amd64-i386-xl-win-vcpus1 8 guest-saverestore fail REGR.
>> vs. 10201
>> test-i386-i386-xl-win 8 guest-saverestore fail REGR.
>> vs. 10201
>> test-amd64-amd64-win 8 guest-saverestore fail REGR.
>> vs. 10201
>
> The xl ones are all failing with:
> 2011-12-02 04:17:52 Z executing ssh ... root@10.80.249.56 xl save
> win.guest.osstest image
> Saving to image new xl format (info 0x0/0x0/625)
> libxl: error: libxl_exec.c:257:libxl__wait_for_offspring: Device
> Model not ready
>
> while the xm ones are failing with:
> 2011-12-02 03:22:20 Z executing ssh ... root@10.80.250.28 xm save
> win.guest.osstest image
> Usage: xm save [-c] <Domain> <CheckpointFile>
>
> Save a domain state to restore later.
> -c, --checkpoint Leave domain running after
> creating
> snapshot
>
> Error: Timed out waiting for device model action
>
> In the xl case the test harness doesn't appear to have noticed the
> failure. Might be bad error handling in xl or a bug in the harness. (my
> money would be on the former).
>
> I can't see anything in the qemu logs for the xm case but in one xl case
> there is:
> Error -22 while loading savevm file '/var/lib/xen/qemu-resume.5'
>
> In the xm case xend logs say:
> 011-12-02 03:22:22 2185] WARNING (image:552) domain
> migrating-win.guest.osstest: device model failure: pid 4278:
> malfunctioning (closed sentinel), killed; see
> /var/log/xen/qemu-dm-win.guest.osstest.log
> [2011-12-02 03:22:32 2185] INFO (XendCheckpoint:423) xc: error: Suspend
> request failed: Internal error
> [2011-12-02 03:22:32 2185] INFO (XendCheckpoint:423) xc: error: Domain
> appears not to have suspended: Internal error
>
> but as I say nothing interesting appears in qemu-dm...log
> [...]
>> version targeted for testing:
>> xen 3f815406feb2
>> baseline version:
>> xen 62ff6a318c5d
>
> changeset: 24332:6b26fb894ca0
> user: Ian Jackson <Ian.Jackson@eu.citrix.com>
> date: Thu Dec 01 18:26:45 2011 +0000
> summary: Update QEMU_TAG
>
> is in there but appears to represent a bunch of passthrough updates.
> There also a bunch of Remus compression stuff which appears to touch
> non-Remus suspend/resume code in libxc too, that's a plausible candidate
> for the cause, I think
>
> There is also a bunch of paging stuff, which I guess/hope is harmless if
> paging is not in use? Tim, Andreas?.
I'd be surprised. Afaict, the only realistic possibilities from my side are
f71ecec8be2e "x86/mm: Ensure maps used by nested hvm code cannot be paged
out"
and
75f4e4d9f039 "x86/mm: Don't trigger unnecessary shadow scans on p2m entry
update"

Andres

>
> Otherwise it appears to all be docs and emulator changes which all seem
> less likely. There's some libxl stuff but you wouldn't expect that to
> hurt xend...
>
> I guess the bisector is on the case.
>
>>
>> ------------------------------------------------------------
>> People who touched revisions under test:
>> Adin Scannell <adin@scannell.ca>
>> Andres Lagar-Cavilla <andres@lagarcavilla.org>
>> Anthony PERARD <anthony.perard@citrix.com>
>> Brendan Cully <brendan@cs.ubc.ca>
>> Daniel De Graaf <dgdegra@tycho.nsa.gov>
>> Ian Campbell <ian.campbell@citrix.com>
>> Ian Jackson <ian.jackson.citrix.com>
>> Ian Jackson <ian.jackson@eu.citrix.com>
>> Jan Beulich <jbeulich@suse.com>
>> Jean Guyader <jean.guyader@eu.citrix.com>
>> Jonathan Davies <jonathan.davies@citrix.com>
>> juergen.gross@ts.fujitsu.com
>> Keir Fraser <keir@xen.org>
>> Liu, Jinsong <jinsong.liu@intel.com>
>> Olaf Hering <olaf@aepfle.de>
>> Philipp Hahn <hahn@univention.de>
>> Shriram Rajagopalan <rshriram@cs.ubc.ca>
>> Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> Tim Deegan <tim@xen.org>
>> ------------------------------------------------------------
>>
>> jobs:
>> build-amd64 pass
>> build-i386 pass
>> build-amd64-oldkern pass
>> build-i386-oldkern pass
>> build-amd64-pvops pass
>> build-i386-pvops pass
>> test-amd64-amd64-xl pass
>> test-amd64-i386-xl pass
>> test-i386-i386-xl pass
>> test-amd64-i386-rhel6hvm-amd fail
>> test-amd64-amd64-xl-win7-amd64 fail
>> test-amd64-i386-xl-win7-amd64 fail
>> test-amd64-i386-xl-credit2 pass
>> test-amd64-amd64-xl-pcipt-intel fail
>> test-amd64-i386-rhel6hvm-intel fail
>> test-amd64-i386-xl-multivcpu pass
>> test-amd64-amd64-pair pass
>> test-amd64-i386-pair pass
>> test-i386-i386-pair pass
>> test-amd64-amd64-pv pass
>> test-amd64-i386-pv pass
>> test-i386-i386-pv pass
>> test-amd64-amd64-xl-sedf pass
>> test-amd64-i386-win-vcpus1 fail
>> test-amd64-i386-xl-win-vcpus1 fail
>> test-amd64-i386-xl-winxpsp3-vcpus1 fail
>> test-amd64-amd64-win fail
>> test-amd64-i386-win fail
>> test-i386-i386-win fail
>> test-amd64-amd64-xl-win fail
>> test-i386-i386-xl-win fail
>> test-amd64-i386-xend-winxpsp3 fail
>> test-amd64-amd64-xl-winxpsp3 fail
>> test-i386-i386-xl-winxpsp3 fail
>>
>>
>> ------------------------------------------------------------
>> sg-report-flight on woking.cam.xci-test.com
>> logs: /home/xc_osstest/logs
>> images: /home/xc_osstest/images
>>
>> Logs, config files, etc. are available at
>> http://www.chiark.greenend.org.uk/~xensrcts/logs
>>
>> Test harness code can be found at
>> http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary
>>
>>
>> Not pushing.
>>
>> (No revision log; it would be 1073 lines long.)
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel