Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Xen>
  3. Devel
[xen-unstable test] 10135: regressions - FAIL
ian.jackson at eu
Nov 27, 2011, 4:48 PM
Post #1 of 9 (196 views)
Permalink
flight 10135 xen-unstable real [real]
http://www.chiark.greenend.org.uk/~xensrcts/logs/10135/

Regressions :-(

Tests which did not succeed and are blocking:
test-amd64-i386-rhel6hvm-intel 7 redhat-install fail REGR. vs. 9956
test-amd64-i386-rhel6hvm-amd 7 redhat-install fail REGR. vs. 9956
test-amd64-amd64-xl-win 7 windows-install fail REGR. vs. 9956
test-amd64-i386-xl-win-vcpus1 7 windows-install fail REGR. vs. 9956
test-i386-i386-xl-win 7 windows-install fail REGR. vs. 9956

Tests which did not succeed, but are not blocking,
including regressions (tests previously passed) regarded as allowable:
test-amd64-amd64-xl-pcipt-intel 9 guest-start fail never pass
test-amd64-i386-win-vcpus1 16 leak-check/check fail never pass
test-amd64-i386-win 16 leak-check/check fail never pass
test-i386-i386-win 16 leak-check/check fail never pass
test-amd64-amd64-win 16 leak-check/check fail never pass

version targeted for testing:
xen 95d4e2e0bed3
baseline version:
xen 0a0c02a61676

------------------------------------------------------------
People who touched revisions under test:
Adin Scannell <adin@scannell.ca>
Andres Lagar-Cavilla <andres@lagarcavilla.org>
Anil Madhavapeddy <anil@recoil.org>
Daniel De Graaf <dgdegra@tycho.nsa.gov>
George Dunlap <george.dunlap@eu.citrix.com>
Ian Campbell <ian.campbell@citrix.com>
Ian Jackson <ian.jackson.citrix.com>
Ian Jackson <ian.jackson@eu.citrix.com>
Jan Beulich <jbeulich@suse.com>
Jean Guyader <jean.guyader@eu.citrix.com>
Keir Fraser <keir@xen.org>
Olaf Hering <olaf@aepfle.de>
Paul Durrant <paul.durrant@citrix.com>
Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Tim Deegan <tim@xen.org>
Wei Wang <wei.wang2@amd.com>
------------------------------------------------------------

jobs:
build-amd64 pass
build-i386 pass
build-amd64-oldkern pass
build-i386-oldkern pass
build-amd64-pvops pass
build-i386-pvops pass
test-amd64-amd64-xl pass
test-amd64-i386-xl pass
test-i386-i386-xl pass
test-amd64-i386-rhel6hvm-amd fail
test-amd64-i386-xl-credit2 pass
test-amd64-amd64-xl-pcipt-intel fail
test-amd64-i386-rhel6hvm-intel fail
test-amd64-i386-xl-multivcpu pass
test-amd64-amd64-pair pass
test-amd64-i386-pair pass
test-i386-i386-pair pass
test-amd64-amd64-pv pass
test-amd64-i386-pv pass
test-i386-i386-pv pass
test-amd64-amd64-xl-sedf pass
test-amd64-i386-win-vcpus1 fail
test-amd64-i386-xl-win-vcpus1 fail
test-amd64-amd64-win fail
test-amd64-i386-win fail
test-i386-i386-win fail
test-amd64-amd64-xl-win fail
test-i386-i386-xl-win fail


------------------------------------------------------------
sg-report-flight on woking.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images

Logs, config files, etc. are available at
http://www.chiark.greenend.org.uk/~xensrcts/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 906 lines long.)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [xen-unstable test] 10135: regressions - FAIL [ In reply to ]
Ian.Jackson at eu
Nov 28, 2011, 3:58 AM
Post #2 of 9 (192 views)
Permalink
xen.org writes ("[xen-unstable test] 10135: regressions - FAIL"):
> flight 10135 xen-unstable real [real]
> http://www.chiark.greenend.org.uk/~xensrcts/logs/10135/
>
> Regressions :-(
>
> Tests which did not succeed and are blocking:
> test-amd64-i386-rhel6hvm-intel 7 redhat-install fail REGR. vs. 9956

My bisector wasn't able to finger an exact changeset due to a blocking
bug in the vicinity, but:

pass 24183:53bec838bb08 Merge
blocked 24184:4ecd3615e726 tools: use system installed libaio by default.
blocked 24185:f88c745575bb docs: remove some fatally out of date ...
fail 24186:7aa5838499d1 tools: use system libaio for blktap1 as well.

Here the "system installed libaio" is that from Debian squeeze.

So for now I have applied the changeset below in the hope of getting a
pass and push.

Ian.

# HG changeset patch
# User Ian Jackson <Ian.Jackson@eu.citrix.com>
# Date 1322481443 0
# Node ID a9c67c2daf4b0181ef2581471ea920eecb495616
# Parent 95d4e2e0bed374602b5a78ee004b057ad8715d65
tools: Revert to our built-in aio

These two changesets:
24184:4ecd3615e726 tools: use system installed libaio by default.
24186:7aa5838499d1 tools: use system libaio for blktap1 as well.
cause HVM guest installs (both Windows and Redhat) to fail on Debian
squeeze with xl. So change the default for now.

Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>

diff -r 95d4e2e0bed3 -r a9c67c2daf4b Config.mk
--- a/Config.mk Fri Nov 25 20:32:05 2011 +0000
+++ b/Config.mk Mon Nov 28 11:57:23 2011 +0000
@@ -232,7 +232,7 @@ PYTHON_TOOLS ?= y
OCAML_TOOLS ?= y
CONFIG_MINITERM ?= n
CONFIG_LOMOUNT ?= n
-CONFIG_SYSTEM_LIBAIO ?= y
+CONFIG_SYSTEM_LIBAIO ?= n

ifeq ($(OCAML_TOOLS),y)
OCAML_TOOLS := $(shell ocamlopt -v > /dev/null 2>&1 && echo "y" || echo "n")

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [xen-unstable test] 10135: regressions - FAIL [ In reply to ]
Ian.Campbell at citrix
Nov 28, 2011, 5:38 AM
Post #3 of 9 (193 views)
Permalink
On Mon, 2011-11-28 at 11:58 +0000, Ian Jackson wrote:
> xen.org writes ("[xen-unstable test] 10135: regressions - FAIL"):
> > flight 10135 xen-unstable real [real]
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/10135/
> >
> > Regressions :-(
> >
> > Tests which did not succeed and are blocking:
> > test-amd64-i386-rhel6hvm-intel 7 redhat-install fail REGR. vs. 9956
>
> My bisector wasn't able to finger an exact changeset due to a blocking
> bug in the vicinity, but:
>
> pass 24183:53bec838bb08 Merge
> blocked 24184:4ecd3615e726 tools: use system installed libaio by default.
> blocked 24185:f88c745575bb docs: remove some fatally out of date ...
> fail 24186:7aa5838499d1 tools: use system libaio for blktap1 as well.
>
> Here the "system installed libaio" is that from Debian squeeze.

This is really weird since nothing in the failings tests even exercises
this stuff. Only blktap (1 and 2) uses aio and the the tests don't use
it and neither does qemu-dm which is the only other thing I can think
of. I'm setting up a repro to see if I can figure out what is going
wrong.

> So for now I have applied the changeset below in the hope of getting a
> pass and push.

I think that's best for now.

Ian.

>
> Ian.
>
> # HG changeset patch
> # User Ian Jackson <Ian.Jackson@eu.citrix.com>
> # Date 1322481443 0
> # Node ID a9c67c2daf4b0181ef2581471ea920eecb495616
> # Parent 95d4e2e0bed374602b5a78ee004b057ad8715d65
> tools: Revert to our built-in aio
>
> These two changesets:
> 24184:4ecd3615e726 tools: use system installed libaio by default.
> 24186:7aa5838499d1 tools: use system libaio for blktap1 as well.
> cause HVM guest installs (both Windows and Redhat) to fail on Debian
> squeeze with xl. So change the default for now.
>
> Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>
>
> diff -r 95d4e2e0bed3 -r a9c67c2daf4b Config.mk
> --- a/Config.mk Fri Nov 25 20:32:05 2011 +0000
> +++ b/Config.mk Mon Nov 28 11:57:23 2011 +0000
> @@ -232,7 +232,7 @@ PYTHON_TOOLS ?= y
> OCAML_TOOLS ?= y
> CONFIG_MINITERM ?= n
> CONFIG_LOMOUNT ?= n
> -CONFIG_SYSTEM_LIBAIO ?= y
> +CONFIG_SYSTEM_LIBAIO ?= n
>
> ifeq ($(OCAML_TOOLS),y)
> OCAML_TOOLS := $(shell ocamlopt -v > /dev/null 2>&1 && echo "y" || echo "n")
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [xen-unstable test] 10135: regressions - FAIL [ In reply to ]
Ian.Campbell at citrix
Nov 28, 2011, 7:21 AM
Post #4 of 9 (201 views)
Permalink
On Mon, 2011-11-28 at 13:38 +0000, Ian Campbell wrote:
> On Mon, 2011-11-28 at 11:58 +0000, Ian Jackson wrote:
> > xen.org writes ("[xen-unstable test] 10135: regressions - FAIL"):
> > > flight 10135 xen-unstable real [real]
> > > http://www.chiark.greenend.org.uk/~xensrcts/logs/10135/
> > >
> > > Regressions :-(
> > >
> > > Tests which did not succeed and are blocking:
> > > test-amd64-i386-rhel6hvm-intel 7 redhat-install fail REGR. vs. 9956
> >
> > My bisector wasn't able to finger an exact changeset due to a blocking
> > bug in the vicinity, but:
> >
> > pass 24183:53bec838bb08 Merge
> > blocked 24184:4ecd3615e726 tools: use system installed libaio by default.
> > blocked 24185:f88c745575bb docs: remove some fatally out of date ...
> > fail 24186:7aa5838499d1 tools: use system libaio for blktap1 as well.
> >
> > Here the "system installed libaio" is that from Debian squeeze.
>
> This is really weird since nothing in the failings tests even exercises
> this stuff. Only blktap (1 and 2) uses aio and the the tests don't use
> it and neither does qemu-dm which is the only other thing I can think
> of.

Turns out the test system is using the 2.6.32 kernel from xen.git and
hence is using tapdisk2 for the guest cdrom drive.

> I'm setting up a repro to see if I can figure out what is going
> wrong.

I didn't get as far a reproing but I realised that the issue was that
libaio was not installed on the target and so tapdisk was failing to run
but error handling was broken so we didn't notice apart from some
messages deep in the logs. The following fixes the error handling:

8>-------------------------------------------------------------

# HG changeset patch
# User Ian Campbell <ian.campbell@citrix.com>
# Date 1322493402 0
# Node ID 35883c0b285c8dfce341bdd04deda53ab6db03cf
# Parent f688f1c8205f90bbfbd6900ccd410382c676b47f
libxl: propagate error from tap_ctl_spawn.

Failure here means that a disk will not be correctly setup. I briefly
scanned tools/blktap2/control.c for other goto constructs which did not
set their err variable but didn't see any others.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

diff -r f688f1c8205f -r 35883c0b285c tools/blktap2/control/tap-ctl-create.c
--- a/tools/blktap2/control/tap-ctl-create.c Mon Nov 28 12:59:15 2011 +0000
+++ b/tools/blktap2/control/tap-ctl-create.c Mon Nov 28 15:16:42 2011 +0000
@@ -44,8 +44,10 @@ tap_ctl_create(const char *params, char
return err;

id = tap_ctl_spawn();
- if (id < 0)
+ if (id < 0) {
+ err = id;
goto destroy;
+ }

err = tap_ctl_attach(id, minor);
if (err)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [xen-unstable test] 10135: regressions - FAIL [ In reply to ]
Ian.Jackson at eu
Nov 28, 2011, 9:43 AM
Post #5 of 9 (195 views)
Permalink
Ian Campbell writes ("Re: [Xen-devel] [xen-unstable test] 10135: regressions - FAIL"):
> On Mon, 2011-11-28 at 13:38 +0000, Ian Campbell wrote:
> > I'm setting up a repro to see if I can figure out what is going
> > wrong.
>
> I didn't get as far a reproing but I realised that the issue was that
> libaio was not installed on the target and so tapdisk was failing to run
> but error handling was broken so we didn't notice apart from some
> messages deep in the logs. The following fixes the error handling:

Thanks, applied. (I have also fixed the tester.)

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [xen-unstable test] 10135: regressions - FAIL [ In reply to ]
Ian.Campbell at citrix
Nov 30, 2011, 2:26 AM
Post #6 of 9 (189 views)
Permalink
On Mon, 2011-11-28 at 11:58 +0000, Ian Jackson wrote:
> # HG changeset patch
> # User Ian Jackson <Ian.Jackson@eu.citrix.com>
> # Date 1322481443 0
> # Node ID a9c67c2daf4b0181ef2581471ea920eecb495616
> # Parent 95d4e2e0bed374602b5a78ee004b057ad8715d65
> tools: Revert to our built-in aio
>
> These two changesets:
> 24184:4ecd3615e726 tools: use system installed libaio by default.
> 24186:7aa5838499d1 tools: use system libaio for blktap1 as well.
> cause HVM guest installs (both Windows and Redhat) to fail on Debian
> squeeze with xl. So change the default for now.
>
> Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>

AIUI the tester is now installs libaio1 so we can revert this one?

According to
http://www.chiark.greenend.org.uk/~xensrcts/results/bisect.xen-unstable.test-amd64-i386-rhel6hvm-intel.redhat-install.html
changeset 24227:1027e7d13d02 was tested and failed. That would have
included
24206:05dd94652d8d tools/check: Add files missing from 24205:5c88358164cc
24205:5c88358164cc tools: check for libaio unless user has configured CONFIG_SYSTEM_LIBAIO=n

Now I don't think the bisector would ever have given us reason to look
at this commits but if they had gone it at the same time as the new
dependency this could have saved a lot of trouble tracking down the
problem.

In
http://www.chiark.greenend.org.uk/~xensrcts/logs/10032/build-i386/4.ts-xen-build.log (one of the runs finger by the bisector which included the above commits) I can see the CHECK-BUILD runes getting run but not the CHECK-INSTALL ones.

The call to ./chk install in the install target seems to have been
deliberately removed in 16906:f4ee7e5793cf for reasons I don't quite
understand (I suspect because of the install-into-a-staging-dir property
of install). Since you install on a different host to the build host
it's possible that the tester might need to jump through some extra
hoops to cause this stuff to run there anyway. Perhaps the tester could
copy tools/check over and execute "./chk install" or "make
check-install" itself? The top-level install.sh does something like
this, but I'm not sure you are (or should be) using it.

Ian.

>
> diff -r 95d4e2e0bed3 -r a9c67c2daf4b Config.mk
> --- a/Config.mk Fri Nov 25 20:32:05 2011 +0000
> +++ b/Config.mk Mon Nov 28 11:57:23 2011 +0000
> @@ -232,7 +232,7 @@ PYTHON_TOOLS ?= y
> OCAML_TOOLS ?= y
> CONFIG_MINITERM ?= n
> CONFIG_LOMOUNT ?= n
> -CONFIG_SYSTEM_LIBAIO ?= y
> +CONFIG_SYSTEM_LIBAIO ?= n
>
> ifeq ($(OCAML_TOOLS),y)
> OCAML_TOOLS := $(shell ocamlopt -v > /dev/null 2>&1 && echo "y" || echo "n")
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [xen-unstable test] 10135: regressions - FAIL [ In reply to ]
Ian.Jackson at eu
Nov 30, 2011, 6:35 AM
Post #7 of 9 (185 views)
Permalink
~Ian Campbell writes ("Re: [Xen-devel] [xen-unstable test] 10135: regressions - FAIL"):
>> According to
> http://www.chiark.greenend.org.uk/~xensrcts/results/bisect.xen-unstable.test-amd64-i386-rhel6hvm-intel.redhat-install.html
> changeset 24227:1027e7d13d02 was tested and failed. That would have
> included
> 24206:05dd94652d8d tools/check: Add files missing from 24205:5c88358164cc
> 24205:5c88358164cc tools: check for libaio unless user has configured CONFIG_SYSTEM_LIBAIO=n
>
> Now I don't think the bisector would ever have given us reason to look
> at this commits but if they had gone it at the same time as the new
> dependency this could have saved a lot of trouble tracking down the
> problem.

"gone in" I guess. I spent a while staring that trying to make sense
of it assuming it was meant to say "done it".

But, no, because the check would have passed because it's run on the
build host and the build host has the runtime lib installed because
it's a dependency of the dev lib.

> In
> http://www.chiark.greenend.org.uk/~xensrcts/logs/10032/build-i386/4.ts-xen-build.log (one of the runs finger by the bisector which included the above commits) I can see the CHECK-BUILD runes getting run but not the CHECK-INSTALL ones.
>
> The call to ./chk install in the install target seems to have been
> deliberately removed in 16906:f4ee7e5793cf for reasons I don't quite
> understand (I suspect because of the install-into-a-staging-dir property
> of install). Since you install on a different host to the build host
> it's possible that the tester might need to jump through some extra
> hoops to cause this stuff to run there anyway. Perhaps the tester could
> copy tools/check over and execute "./chk install" or "make
> check-install" itself? The top-level install.sh does something like
> this, but I'm not sure you are (or should be) using it.

I could have the tester run ./chk install on the install host and
see. But I don't think we promise that this will work.

Running ./chk install on the build host during "make install" is a
good idea but wouldn't have found this problem more quickly.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [xen-unstable test] 10135: regressions - FAIL [ In reply to ]
Ian.Campbell at citrix
Nov 30, 2011, 7:55 AM
Post #8 of 9 (187 views)
Permalink
On Wed, 2011-11-30 at 14:35 +0000, Ian Jackson wrote:
> ~Ian Campbell writes ("Re: [Xen-devel] [xen-unstable test] 10135: regressions - FAIL"):
> >> According to
> > http://www.chiark.greenend.org.uk/~xensrcts/results/bisect.xen-unstable.test-amd64-i386-rhel6hvm-intel.redhat-install.html
> > changeset 24227:1027e7d13d02 was tested and failed. That would have
> > included
> > 24206:05dd94652d8d tools/check: Add files missing from 24205:5c88358164cc
> > 24205:5c88358164cc tools: check for libaio unless user has configured CONFIG_SYSTEM_LIBAIO=n
> >
> > Now I don't think the bisector would ever have given us reason to look
> > at this commits but if they had gone it at the same time as the new
> > dependency this could have saved a lot of trouble tracking down the
> > problem.
>
> "gone in" I guess. I spent a while staring that trying to make sense
> of it assuming it was meant to say "done it".

Yes, sorry. "... if they had gone in at the ... "

> But, no, because the check would have passed because it's run on the
> build host and the build host has the runtime lib installed because
> it's a dependency of the dev lib.
>
> > In
> > http://www.chiark.greenend.org.uk/~xensrcts/logs/10032/build-i386/4.ts-xen-build.log (one of the runs finger by the bisector which included the above commits) I can see the CHECK-BUILD runes getting run but not the CHECK-INSTALL ones.
> >
> > The call to ./chk install in the install target seems to have been
> > deliberately removed in 16906:f4ee7e5793cf for reasons I don't quite
> > understand (I suspect because of the install-into-a-staging-dir property
> > of install). Since you install on a different host to the build host
> > it's possible that the tester might need to jump through some extra
> > hoops to cause this stuff to run there anyway. Perhaps the tester could
> > copy tools/check over and execute "./chk install" or "make
> > check-install" itself? The top-level install.sh does something like
> > this, but I'm not sure you are (or should be) using it.
>
> I could have the tester run ./chk install on the install host and
> see. But I don't think we promise that this will work.

We could decide to promise it going forward if it is useful.

> Running ./chk install on the build host during "make install" is a
> good idea but wouldn't have found this problem more quickly.

Right, if they are running on the same host I don't think ./chk install
will find much which is not found by ./chk build in practice.

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Re: [xen-unstable test] 10135: regressions - FAIL [ In reply to ]
Ian.Jackson at eu
Dec 1, 2011, 8:20 AM
Post #9 of 9 (182 views)
Permalink
Ian Campbell writes ("Re: [Xen-devel] [xen-unstable test] 10135: regressions - FAIL"):
> AIUI the tester is now installs libaio1 so we can revert this one?

Yes, I've done so.

# HG changeset patch
# User Ian Jackson <Ian.Jackson@eu.citrix.com>
# Date 1322756338 0
# Node ID a07e45d2aa7f792674b6325b11a48d297fd4d075
# Parent 7b180380681db3382f7c2f38e096bc551aa301d2
tools: Switch to system libaio (again)

The test system problems which prompted 24233:a9c67c2daf4b (itself a
tiny partial revert of 24184:4ecd3615e726 / 24186:7aa5838499d1) have
now been resolved, we think. So revert 24233.

Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>

diff -r 7b180380681d -r a07e45d2aa7f Config.mk
--- a/Config.mk Thu Dec 01 16:05:51 2011 +0000
+++ b/Config.mk Thu Dec 01 16:18:58 2011 +0000
@@ -232,7 +232,7 @@ PYTHON_TOOLS ?= y
OCAML_TOOLS ?= y
CONFIG_MINITERM ?= n
CONFIG_LOMOUNT ?= n
-CONFIG_SYSTEM_LIBAIO ?= n
+CONFIG_SYSTEM_LIBAIO ?= y

ifeq ($(OCAML_TOOLS),y)
OCAML_TOOLS := $(shell ocamlopt -v > /dev/null 2>&1 && echo "y" || echo "n")

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal