Mailing List Archive

Compiling Xen from source
Hey,

I am new to both Xen and development in general and am running into some issues when trying to compile and install from source.
I have already been able to get Xen installed and working using apt-get, but since I want to be able to edit the source code, this will not
Be an option for install.

The first thing I noticed that was quite a bit different from installing from the package manager, was the difference in entries shown in grub.
When I installed Xen using apt-get, all I had was one entry for Xen in grub, which I would use this command to have it be default.

"dpkg-divert -divert /etc/grub.d/08_linux_xen -rename /etc/grub.d/20_linux_xen"

Now when installing from source, there are several entries, where only a couple of them will actually work.

After Xen is installed, I typically run the "xl info" command to ensure its running. However, when I try to run it after being compiled from source, I
Get this error:

"libxl: error: libxl.c:87: libxl_ctx_alloc: Is Xenstore daemon running?
Failed to stat /var/run/xenstored.pid
No such file or directory
Cannot init xl content"

I checked and that directory does not exist, but was able to start Xen manually by running xencommons in /etc/init.d

The problem with manually starting it though, is that it causes my terminal to quit echoing my input.

What do I need to do to resolve this error, and get Xen to automatically start?

Thanks for any help!
Josh



________________________________
This message and all attachments are PRIVATE, and contain information that is PROPRIETARY to Intelligent Automation, Inc. You are not authorized to transmit or otherwise disclose this message or any attachments to any third party whatsoever without the express written consent of Intelligent Automation, Inc. If you received this message in error or you are not willing to view this message or any attachments on a confidential basis, please immediately delete this email and any attachments and notify Intelligent Automation, Inc.
Re: Compiling Xen from source [ In reply to ]
On 10/06/13 17:41, Joshua Tuttle wrote:
> Hey,
>
>
>
> I am new to both Xen and development in general and am running into some
> issues when trying to compile and install from source.
>
> I have already been able to get Xen installed and working using apt-get,
> but since I want to be able to edit the source code, this will not
>
> Be an option for install.

I guess you removed the Xen packages that you installed with apt-get
before installing from source, right?

>
>
>
> The first thing I noticed that was quite a bit different from installing
> from the package manager, was the difference in entries shown in grub.
>
> When I installed Xen using apt-get, all I had was one entry for Xen in
> grub, which I would use this command to have it be default.
>
>
>
> “dpkg-divert –divert /etc/grub.d/08_linux_xen –rename
> /etc/grub.d/20_linux_xen”
>
>
>
> Now when installing from source, there are several entries, where only a
> couple of them will actually work.
>
>
>
> After Xen is installed, I typically run the “xl info” command to ensure
> its running. However, when I try to run it after being compiled from
> source, I
>
> Get this error:
>
>
>
> “libxl: error: libxl.c:87: libxl_ctx_alloc: Is Xenstore daemon running?
>
> Failed to stat /var/run/xenstored.pid
>
> No such file or directory
>
> Cannot init xl content”
>
>
>
> I checked and that directory does not exist, but was able to start Xen
> manually by running xencommons in /etc/init.d
>
>
>
> The problem with manually starting it though, is that it causes my
> terminal to quit echoing my input.

This is a Qemu problem that I think was solved in recent versions. "stty
sane" used to resolve this issue.

>
>
> What do I need to do to resolve this error, and get Xen to automatically
> start?

# update-rc.d xencommons defaults

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: Compiling Xen from source [ In reply to ]
Did you look at package source code ? You may need to apply patches related
to your os to source code. Or you could rebuild package with your changes .
On Jun 10, 2013 9:54 AM, "Roger Pau Monné" <roger.pau@citrix.com> wrote:

> On 10/06/13 17:41, Joshua Tuttle wrote:
> > Hey,
> >
> >
> >
> > I am new to both Xen and development in general and am running into some
> > issues when trying to compile and install from source.
> >
> > I have already been able to get Xen installed and working using apt-get,
> > but since I want to be able to edit the source code, this will not
> >
> > Be an option for install.
>
> I guess you removed the Xen packages that you installed with apt-get
> before installing from source, right?
>
> >
> >
> >
> > The first thing I noticed that was quite a bit different from installing
> > from the package manager, was the difference in entries shown in grub.
> >
> > When I installed Xen using apt-get, all I had was one entry for Xen in
> > grub, which I would use this command to have it be default.
> >
> >
> >
> > “dpkg-divert –divert /etc/grub.d/08_linux_xen –rename
> > /etc/grub.d/20_linux_xen”
> >
> >
> >
> > Now when installing from source, there are several entries, where only a
> > couple of them will actually work.
> >
> >
> >
> > After Xen is installed, I typically run the “xl info” command to ensure
> > its running. However, when I try to run it after being compiled from
> > source, I
> >
> > Get this error:
> >
> >
> >
> > “libxl: error: libxl.c:87: libxl_ctx_alloc: Is Xenstore daemon running?
> >
> > Failed to stat /var/run/xenstored.pid
> >
> > No such file or directory
> >
> > Cannot init xl content”
> >
> >
> >
> > I checked and that directory does not exist, but was able to start Xen
> > manually by running xencommons in /etc/init.d
> >
> >
> >
> > The problem with manually starting it though, is that it causes my
> > terminal to quit echoing my input.
>
> This is a Qemu problem that I think was solved in recent versions. "stty
> sane" used to resolve this issue.
>
> >
> >
> > What do I need to do to resolve this error, and get Xen to automatically
> > start?
>
> # update-rc.d xencommons defaults
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>
Re: Compiling Xen from source [ In reply to ]
On 15/09/2020 21:16, David I wrote:
> Hello,
>
> I am unable to mcompile xen-tools from source.
>
> here is the "stacktrace":
>
> ln -f xenstore xenstore-watch
> gcc xenstore_control.o    
>  -Wl,-rpath-link=/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore
> /home/david/xen/xen/tools/xenstore/../../tools/xenstore/libxenstore.so
> -ldl
>  /home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/libxentoolcore.so
>  -o xenstore-control
> gcc  -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
> -Wstrict-prototypes -Wdeclaration-after-statement
> -Wno-unused-but-set-variable -Wno-unused-local-typedefs   -O2
> -fomit-frame-pointer
> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF
> .xs_tdb_dump.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE   -Werror
> -I. -include /home/david/xen/xen/tools/xenstore/../../tools/config.h
> -I./include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/evtchn/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libxc/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toollog/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/foreignmemory/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/devicemodel/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -D__XEN_TOOLS__
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -DXEN_LIB_STORED="\"/var/lib/xenstored\""
> -DXEN_RUN_STORED="\"/var/run/xenstored\""   -c -o xs_tdb_dump.o
> xs_tdb_dump.c
> gcc  -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
> -Wstrict-prototypes -Wdeclaration-after-statement
> -Wno-unused-but-set-variable -Wno-unused-local-typedefs   -O2
> -fomit-frame-pointer
> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF
> .utils.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE   -Werror -I.
> -include /home/david/xen/xen/tools/xenstore/../../tools/config.h
> -I./include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/evtchn/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libxc/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toollog/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/foreignmemory/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/devicemodel/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -D__XEN_TOOLS__
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> -DXEN_LIB_STORED="\"/var/lib/xenstored\""
> -DXEN_RUN_STORED="\"/var/run/xenstored\""  
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/gnttab/include
> -I/home/david/xen/xen/tools/xenstore/../../tools/include  -c -o
> utils.o utils.c
> gcc xs_tdb_dump.o utils.o tdb.o talloc.o     -o xs_tdb_dump
> /usr/bin/ld : utils.o:(.data.rel.local+0x0) : définitions multiples de
> « xprintf »; xs_tdb_dump.o:(.bss+0x0) : défini pour la première fois ici
> collect2: error: ld returned 1 exit status
> make[4]: *** [Makefile:97 : xs_tdb_dump] Erreur 1
> make[4] : on quitte le répertoire « /home/david/xen/xen/tools/xenstore »
> make[3]: *** [/home/david/xen/xen/tools/../tools/Rules.mk:249 :
> subdir-install-xenstore] Erreur 2
> make[3] : on quitte le répertoire « /home/david/xen/xen/tools »
> make[2]: *** [/home/david/xen/xen/tools/../tools/Rules.mk:244 :
> subdirs-install] Erreur 2
> make[2] : on quitte le répertoire « /home/david/xen/xen/tools »
> make[1]: *** [Makefile:74 : install] Erreur 2
> make[1] : on quitte le répertoire « /home/david/xen/xen/tools »
> make: *** [Makefile:127 : install-tools] Erreur 2
> david@debian:~/xen/xen$ uname -a
> Linux debian 5.7.0-2-amd64 #1 SMP Debian 5.7.10-1 (2020-07-26) x86_64
> GNU/Linux
> david@debian:~/xen/xen$ gcc --version
> gcc (Debian 10.2.0-5) 10.2.0
> Copyright (C) 2020 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions.  There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> PURPOSE.
>
> david@debian:~/xen/xen$ /usr/bin/ld --version
> GNU ld (GNU Binutils for Debian) 2.35
> Copyright (C) 2020 Free Software Foundation, Inc.
> Ce logiciel est libre; si vous le redistribuez, vous devez le faire
> selon les termes
> de la licence GNU General Public License version 3 ou postérieure
> selon votre besoin.
> Ce logiciel n'est couvert par aucune GARANTIE.
> david@debian:~/xen/xen$ git branch
>   master
> * stable-4.11
>   staging
> david@debian:~/xen/xen$
>
> I was able to compile dist-xen correctly though. But dist-tools fails
> with this error, I have seen a similar request here:
> https://stackoverflow.com/questions/63525163/usr-bin-ld-utils-o-data-rel-local0x0-multiple-definition-of-xprintf-xs
>
> I have tried to compile xen from source also on this same configuration,
> in stable-4.12, stable-4.13, stable-4.14, each time I was able to 
> compile xen but failed to compile xen-tools.
> the debian package works fine and is in version 4.11.
>
> So my question would be, is there a patch for this issue ? and how the
> debian team could compile originally the debian source package ? I
> suppose I am not the first user to encounter this.
>
> Thank you for your answers ;-)

Its a change in the toolchain from -fcommon to -fno-common by default.

Already fixed upstream by: 
https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=dacdbf7088d6a3705a9831e73991c2b14c519a65

Looks like that change wants backporting.

~Andrew
Re: Compiling Xen from source [ In reply to ]
Hi,

On 9/16/20 12:00 AM, Andrew Cooper wrote:
> On 15/09/2020 21:16, David I wrote:
>> Hello,
>>
>> I am unable to mcompile xen-tools from source.
>>
>> here is the "stacktrace":
>>
>> ln -f xenstore xenstore-watch
>> gcc xenstore_control.o    
>>  -Wl,-rpath-link=/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore
>> /home/david/xen/xen/tools/xenstore/../../tools/xenstore/libxenstore.so
>> -ldl
>>  /home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/libxentoolcore.so
>>  -o xenstore-control
>> gcc  -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
>> -Wstrict-prototypes -Wdeclaration-after-statement
>> -Wno-unused-but-set-variable -Wno-unused-local-typedefs   -O2
>> -fomit-frame-pointer
>> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF
>> .xs_tdb_dump.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE   -Werror
>> -I. -include /home/david/xen/xen/tools/xenstore/../../tools/config.h
>> -I./include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/evtchn/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libxc/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toollog/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/foreignmemory/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/devicemodel/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -D__XEN_TOOLS__
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -DXEN_LIB_STORED="\"/var/lib/xenstored\""
>> -DXEN_RUN_STORED="\"/var/run/xenstored\""   -c -o xs_tdb_dump.o
>> xs_tdb_dump.c
>> gcc  -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
>> -Wstrict-prototypes -Wdeclaration-after-statement
>> -Wno-unused-but-set-variable -Wno-unused-local-typedefs   -O2
>> -fomit-frame-pointer
>> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF
>> .utils.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE   -Werror -I.
>> -include /home/david/xen/xen/tools/xenstore/../../tools/config.h
>> -I./include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/evtchn/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libxc/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toollog/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/foreignmemory/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/devicemodel/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -D__XEN_TOOLS__
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> -DXEN_LIB_STORED="\"/var/lib/xenstored\""
>> -DXEN_RUN_STORED="\"/var/run/xenstored\""  
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/gnttab/include
>> -I/home/david/xen/xen/tools/xenstore/../../tools/include  -c -o
>> utils.o utils.c
>> gcc xs_tdb_dump.o utils.o tdb.o talloc.o     -o xs_tdb_dump
>> /usr/bin/ld : utils.o:(.data.rel.local+0x0) : définitions multiples de
>> « xprintf »; xs_tdb_dump.o:(.bss+0x0) : défini pour la première fois ici
>> collect2: error: ld returned 1 exit status
>> make[4]: *** [Makefile:97 : xs_tdb_dump] Erreur 1
>> make[4] : on quitte le répertoire « /home/david/xen/xen/tools/xenstore »
>> make[3]: *** [/home/david/xen/xen/tools/../tools/Rules.mk:249 :
>> subdir-install-xenstore] Erreur 2
>> make[3] : on quitte le répertoire « /home/david/xen/xen/tools »
>> make[2]: *** [/home/david/xen/xen/tools/../tools/Rules.mk:244 :
>> subdirs-install] Erreur 2
>> make[2] : on quitte le répertoire « /home/david/xen/xen/tools »
>> make[1]: *** [Makefile:74 : install] Erreur 2
>> make[1] : on quitte le répertoire « /home/david/xen/xen/tools »
>> make: *** [Makefile:127 : install-tools] Erreur 2
>> david@debian:~/xen/xen$ uname -a
>> Linux debian 5.7.0-2-amd64 #1 SMP Debian 5.7.10-1 (2020-07-26) x86_64
>> GNU/Linux
>> david@debian:~/xen/xen$ gcc --version
>> gcc (Debian 10.2.0-5) 10.2.0
>> Copyright (C) 2020 Free Software Foundation, Inc.
>> This is free software; see the source for copying conditions.  There is NO
>> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
>> PURPOSE.
>>
>> david@debian:~/xen/xen$ /usr/bin/ld --version
>> GNU ld (GNU Binutils for Debian) 2.35
>> Copyright (C) 2020 Free Software Foundation, Inc.
>> Ce logiciel est libre; si vous le redistribuez, vous devez le faire
>> selon les termes
>> de la licence GNU General Public License version 3 ou postérieure
>> selon votre besoin.
>> Ce logiciel n'est couvert par aucune GARANTIE.
>> david@debian:~/xen/xen$ git branch
>>   master
>> * stable-4.11
>>   staging
>> david@debian:~/xen/xen$
>>
>> I was able to compile dist-xen correctly though. But dist-tools fails
>> with this error, I have seen a similar request here:
>> https://stackoverflow.com/questions/63525163/usr-bin-ld-utils-o-data-rel-local0x0-multiple-definition-of-xprintf-xs
>>
>> I have tried to compile xen from source also on this same configuration,
>> in stable-4.12, stable-4.13, stable-4.14, each time I was able to 
>> compile xen but failed to compile xen-tools.
>> the debian package works fine and is in version 4.11.
>>
>> So my question would be, is there a patch for this issue ? and how the
>> debian team could compile originally the debian source package ? I
>> suppose I am not the first user to encounter this.
>>
>> Thank you for your answers ;-)
>
> Its a change in the toolchain from -fcommon to -fno-common by default.
>
> Already fixed upstream by: 
> https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=dacdbf7088d6a3705a9831e73991c2b14c519a65
>
> Looks like that change wants backporting.

Currently, I'm working together with Ian to get Xen 4.14 into Debian
unstable. dacdbf708 is in 4.14 already, but we also had to pick

d25cc3ec93eb ("libxl: workaround gcc 10.2 maybe-uninitialized warning")

and

fff1b7f50e75 ("libxl: fix -Werror=stringop-truncation in
libxl__prepare_sockaddr_un")

into 4.14 to fix further gcc 10 FTBFS (yay, backport candidates). Maybe
this info helps a bit if you also run into these issues after fixing the
first one.

Hans
Re: Compiling Xen from source [ In reply to ]
Hello,

thanks for your infos. it appears the compilation process is not really
straightforward.
I had to patch some header files. then some of the firmwares were bugged.
So I disabled some various checks in the firmwares Makefiles.
I ended up with a:
"no target for make all":
configure passed all tests with werror disabled, but failed some tests with
werror enabled.

finally "make dist-xen" always succeeds", but "make dist-tools" seems not
bug-free.

So if you have additional advice I will try further. Also if you need
volunteers to test extensively maybe i can help.

David




Le mer. 16 sept. 2020 à 00:34, Hans van Kranenburg <hans@knorrie.org> a
écrit :

> Hi,
>
> On 9/16/20 12:00 AM, Andrew Cooper wrote:
> > On 15/09/2020 21:16, David I wrote:
> >> Hello,
> >>
> >> I am unable to mcompile xen-tools from source.
> >>
> >> here is the "stacktrace":
> >>
> >> ln -f xenstore xenstore-watch
> >> gcc xenstore_control.o
> >>
> -Wl,-rpath-link=/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore
> >> /home/david/xen/xen/tools/xenstore/../../tools/xenstore/libxenstore.so
> >> -ldl
> >>
> /home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/libxentoolcore.so
> >> -o xenstore-control
> >> gcc -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
> >> -Wstrict-prototypes -Wdeclaration-after-statement
> >> -Wno-unused-but-set-variable -Wno-unused-local-typedefs -O2
> >> -fomit-frame-pointer
> >> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF
> >> .xs_tdb_dump.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Werror
> >> -I. -include /home/david/xen/xen/tools/xenstore/../../tools/config.h
> >> -I./include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/evtchn/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libxc/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toollog/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >>
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/foreignmemory/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >>
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/devicemodel/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >> -D__XEN_TOOLS__
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >> -DXEN_LIB_STORED="\"/var/lib/xenstored\""
> >> -DXEN_RUN_STORED="\"/var/run/xenstored\"" -c -o xs_tdb_dump.o
> >> xs_tdb_dump.c
> >> gcc -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
> >> -Wstrict-prototypes -Wdeclaration-after-statement
> >> -Wno-unused-but-set-variable -Wno-unused-local-typedefs -O2
> >> -fomit-frame-pointer
> >> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF
> >> .utils.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Werror -I.
> >> -include /home/david/xen/xen/tools/xenstore/../../tools/config.h
> >> -I./include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/evtchn/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libxc/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toollog/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >>
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/foreignmemory/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >>
> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/devicemodel/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >> -D__XEN_TOOLS__
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
> >> -DXEN_LIB_STORED="\"/var/lib/xenstored\""
> >> -DXEN_RUN_STORED="\"/var/run/xenstored\""
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/gnttab/include
> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include -c -o
> >> utils.o utils.c
> >> gcc xs_tdb_dump.o utils.o tdb.o talloc.o -o xs_tdb_dump
> >> /usr/bin/ld : utils.o:(.data.rel.local+0x0) : définitions multiples de
> >> « xprintf »; xs_tdb_dump.o:(.bss+0x0) : défini pour la première fois ici
> >> collect2: error: ld returned 1 exit status
> >> make[4]: *** [Makefile:97 : xs_tdb_dump] Erreur 1
> >> make[4] : on quitte le répertoire « /home/david/xen/xen/tools/xenstore »
> >> make[3]: *** [/home/david/xen/xen/tools/../tools/Rules.mk:249 :
> >> subdir-install-xenstore] Erreur 2
> >> make[3] : on quitte le répertoire « /home/david/xen/xen/tools »
> >> make[2]: *** [/home/david/xen/xen/tools/../tools/Rules.mk:244 :
> >> subdirs-install] Erreur 2
> >> make[2] : on quitte le répertoire « /home/david/xen/xen/tools »
> >> make[1]: *** [Makefile:74 : install] Erreur 2
> >> make[1] : on quitte le répertoire « /home/david/xen/xen/tools »
> >> make: *** [Makefile:127 : install-tools] Erreur 2
> >> david@debian:~/xen/xen$ uname -a
> >> Linux debian 5.7.0-2-amd64 #1 SMP Debian 5.7.10-1 (2020-07-26) x86_64
> >> GNU/Linux
> >> david@debian:~/xen/xen$ gcc --version
> >> gcc (Debian 10.2.0-5) 10.2.0
> >> Copyright (C) 2020 Free Software Foundation, Inc.
> >> This is free software; see the source for copying conditions. There is
> NO
> >> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
> >> PURPOSE.
> >>
> >> david@debian:~/xen/xen$ /usr/bin/ld --version
> >> GNU ld (GNU Binutils for Debian) 2.35
> >> Copyright (C) 2020 Free Software Foundation, Inc.
> >> Ce logiciel est libre; si vous le redistribuez, vous devez le faire
> >> selon les termes
> >> de la licence GNU General Public License version 3 ou postérieure
> >> selon votre besoin.
> >> Ce logiciel n'est couvert par aucune GARANTIE.
> >> david@debian:~/xen/xen$ git branch
> >> master
> >> * stable-4.11
> >> staging
> >> david@debian:~/xen/xen$
> >>
> >> I was able to compile dist-xen correctly though. But dist-tools fails
> >> with this error, I have seen a similar request here:
> >>
> https://stackoverflow.com/questions/63525163/usr-bin-ld-utils-o-data-rel-local0x0-multiple-definition-of-xprintf-xs
> >>
> >> I have tried to compile xen from source also on this same configuration,
> >> in stable-4.12, stable-4.13, stable-4.14, each time I was able to
> >> compile xen but failed to compile xen-tools.
> >> the debian package works fine and is in version 4.11.
> >>
> >> So my question would be, is there a patch for this issue ? and how the
> >> debian team could compile originally the debian source package ? I
> >> suppose I am not the first user to encounter this.
> >>
> >> Thank you for your answers ;-)
> >
> > Its a change in the toolchain from -fcommon to -fno-common by default.
> >
> > Already fixed upstream by:
> >
> https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=dacdbf7088d6a3705a9831e73991c2b14c519a65
> >
> > Looks like that change wants backporting.
>
> Currently, I'm working together with Ian to get Xen 4.14 into Debian
> unstable. dacdbf708 is in 4.14 already, but we also had to pick
>
> d25cc3ec93eb ("libxl: workaround gcc 10.2 maybe-uninitialized warning")
>
> and
>
> fff1b7f50e75 ("libxl: fix -Werror=stringop-truncation in
> libxl__prepare_sockaddr_un")
>
> into 4.14 to fix further gcc 10 FTBFS (yay, backport candidates). Maybe
> this info helps a bit if you also run into these issues after fixing the
> first one.
>
> Hans
>
Re: Compiling Xen from source [ In reply to ]
Examples of compile bugs for xen-stable-4.11
for make dist-tools:

"include/ipxe/uri.h:178:12: error: taking address of packed member of
‘struct uri’ may result in an unaligned pointer value
[-Werror=address-of-packed-member]
178 | ref_get ( &uri->refcnt );"
-> fixed by adding the Cflag "-Wno-address-of-packed-member" in the
Makefile (not sure this is the right solution !).

next firmware error:
"drivers/infiniband/flexboot_nodnic.c: In function
‘flexboot_nodnic_create_qp’:
drivers/infiniband/flexboot_nodnic.c:368:53: error: implicit conversion
from ‘enum ib_queue_pair_type’ to ‘nodnic_queue_pair_type’
[-Werror=enum-conversion]"
-> fixed by adding the flag -Wno-enum-conversion in the same
tools/firmware/etherboot/ipxe/src/Makefile

(some other compile errors and dirty tricks not included)

So, how did the debian package was compiled ? is this the same source code
with different options ?

David








Le mer. 16 sept. 2020 à 15:22, David I <dav.sec.lists@gmail.com> a écrit :

> Hello,
>
> thanks for your infos. it appears the compilation process is not really
> straightforward.
> I had to patch some header files. then some of the firmwares were bugged.
> So I disabled some various checks in the firmwares Makefiles.
> I ended up with a:
> "no target for make all":
> configure passed all tests with werror disabled, but failed some tests
> with werror enabled.
>
> finally "make dist-xen" always succeeds", but "make dist-tools" seems not
> bug-free.
>
> So if you have additional advice I will try further. Also if you need
> volunteers to test extensively maybe i can help.
>
> David
>
>
>
>
> Le mer. 16 sept. 2020 à 00:34, Hans van Kranenburg <hans@knorrie.org> a
> écrit :
>
>> Hi,
>>
>> On 9/16/20 12:00 AM, Andrew Cooper wrote:
>> > On 15/09/2020 21:16, David I wrote:
>> >> Hello,
>> >>
>> >> I am unable to mcompile xen-tools from source.
>> >>
>> >> here is the "stacktrace":
>> >>
>> >> ln -f xenstore xenstore-watch
>> >> gcc xenstore_control.o
>> >>
>> -Wl,-rpath-link=/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore
>> >> /home/david/xen/xen/tools/xenstore/../../tools/xenstore/libxenstore.so
>> >> -ldl
>> >>
>> /home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/libxentoolcore.so
>> >> -o xenstore-control
>> >> gcc -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
>> >> -Wstrict-prototypes -Wdeclaration-after-statement
>> >> -Wno-unused-but-set-variable -Wno-unused-local-typedefs -O2
>> >> -fomit-frame-pointer
>> >> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF
>> >> .xs_tdb_dump.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Werror
>> >> -I. -include /home/david/xen/xen/tools/xenstore/../../tools/config.h
>> >> -I./include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/evtchn/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libxc/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toollog/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >>
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/foreignmemory/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >>
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/devicemodel/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >> -D__XEN_TOOLS__
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >> -DXEN_LIB_STORED="\"/var/lib/xenstored\""
>> >> -DXEN_RUN_STORED="\"/var/run/xenstored\"" -c -o xs_tdb_dump.o
>> >> xs_tdb_dump.c
>> >> gcc -m64 -DBUILD_ID -fno-strict-aliasing -std=gnu99 -Wall
>> >> -Wstrict-prototypes -Wdeclaration-after-statement
>> >> -Wno-unused-but-set-variable -Wno-unused-local-typedefs -O2
>> >> -fomit-frame-pointer
>> >> -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__ -MMD -MF
>> >> .utils.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Werror -I.
>> >> -include /home/david/xen/xen/tools/xenstore/../../tools/config.h
>> >> -I./include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/evtchn/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libxc/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toollog/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >>
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/foreignmemory/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >>
>> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/devicemodel/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >> -D__XEN_TOOLS__
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/toolcore/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include
>> >> -DXEN_LIB_STORED="\"/var/lib/xenstored\""
>> >> -DXEN_RUN_STORED="\"/var/run/xenstored\""
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/libs/gnttab/include
>> >> -I/home/david/xen/xen/tools/xenstore/../../tools/include -c -o
>> >> utils.o utils.c
>> >> gcc xs_tdb_dump.o utils.o tdb.o talloc.o -o xs_tdb_dump
>> >> /usr/bin/ld : utils.o:(.data.rel.local+0x0) : définitions multiples de
>> >> « xprintf »; xs_tdb_dump.o:(.bss+0x0) : défini pour la première fois
>> ici
>> >> collect2: error: ld returned 1 exit status
>> >> make[4]: *** [Makefile:97 : xs_tdb_dump] Erreur 1
>> >> make[4] : on quitte le répertoire
>> « /home/david/xen/xen/tools/xenstore »
>> >> make[3]: *** [/home/david/xen/xen/tools/../tools/Rules.mk:249 :
>> >> subdir-install-xenstore] Erreur 2
>> >> make[3] : on quitte le répertoire « /home/david/xen/xen/tools »
>> >> make[2]: *** [/home/david/xen/xen/tools/../tools/Rules.mk:244 :
>> >> subdirs-install] Erreur 2
>> >> make[2] : on quitte le répertoire « /home/david/xen/xen/tools »
>> >> make[1]: *** [Makefile:74 : install] Erreur 2
>> >> make[1] : on quitte le répertoire « /home/david/xen/xen/tools »
>> >> make: *** [Makefile:127 : install-tools] Erreur 2
>> >> david@debian:~/xen/xen$ uname -a
>> >> Linux debian 5.7.0-2-amd64 #1 SMP Debian 5.7.10-1 (2020-07-26) x86_64
>> >> GNU/Linux
>> >> david@debian:~/xen/xen$ gcc --version
>> >> gcc (Debian 10.2.0-5) 10.2.0
>> >> Copyright (C) 2020 Free Software Foundation, Inc.
>> >> This is free software; see the source for copying conditions. There
>> is NO
>> >> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR
>> >> PURPOSE.
>> >>
>> >> david@debian:~/xen/xen$ /usr/bin/ld --version
>> >> GNU ld (GNU Binutils for Debian) 2.35
>> >> Copyright (C) 2020 Free Software Foundation, Inc.
>> >> Ce logiciel est libre; si vous le redistribuez, vous devez le faire
>> >> selon les termes
>> >> de la licence GNU General Public License version 3 ou postérieure
>> >> selon votre besoin.
>> >> Ce logiciel n'est couvert par aucune GARANTIE.
>> >> david@debian:~/xen/xen$ git branch
>> >> master
>> >> * stable-4.11
>> >> staging
>> >> david@debian:~/xen/xen$
>> >>
>> >> I was able to compile dist-xen correctly though. But dist-tools fails
>> >> with this error, I have seen a similar request here:
>> >>
>> https://stackoverflow.com/questions/63525163/usr-bin-ld-utils-o-data-rel-local0x0-multiple-definition-of-xprintf-xs
>> >>
>> >> I have tried to compile xen from source also on this same
>> configuration,
>> >> in stable-4.12, stable-4.13, stable-4.14, each time I was able to
>> >> compile xen but failed to compile xen-tools.
>> >> the debian package works fine and is in version 4.11.
>> >>
>> >> So my question would be, is there a patch for this issue ? and how the
>> >> debian team could compile originally the debian source package ? I
>> >> suppose I am not the first user to encounter this.
>> >>
>> >> Thank you for your answers ;-)
>> >
>> > Its a change in the toolchain from -fcommon to -fno-common by default.
>> >
>> > Already fixed upstream by:
>> >
>> https://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=dacdbf7088d6a3705a9831e73991c2b14c519a65
>> >
>> > Looks like that change wants backporting.
>>
>> Currently, I'm working together with Ian to get Xen 4.14 into Debian
>> unstable. dacdbf708 is in 4.14 already, but we also had to pick
>>
>> d25cc3ec93eb ("libxl: workaround gcc 10.2 maybe-uninitialized warning")
>>
>> and
>>
>> fff1b7f50e75 ("libxl: fix -Werror=stringop-truncation in
>> libxl__prepare_sockaddr_un")
>>
>> into 4.14 to fix further gcc 10 FTBFS (yay, backport candidates). Maybe
>> this info helps a bit if you also run into these issues after fixing the
>> first one.
>>
>> Hans
>>
>
Re: Compiling Xen from source [ In reply to ]
On Wed, Sep 16, David I wrote:

> So, how did the debian package was compiled ? is this the same source code with
> different options ?

Xen 4.11 is from 2018. Use also a compiler from that year.
Using this years compiler will lead to errors...


Olaf
Re: Compiling Xen from source [ In reply to ]
On 9/16/20 6:12 PM, Olaf Hering wrote:
> On Wed, Sep 16, David I wrote:
>
>> So, how did the debian package was compiled ? is this the same source code with
>> different options ?
>
> Xen 4.11 is from 2018. Use also a compiler from that year.
> Using this years compiler will lead to errors...

In Debian, Buster with Xen 4.11 uses gcc 8.

The Xen 4.11 that is in Debian unstable now does not compile any more
with gcc 10. That's why we really need to get Xen 4.14 in there now to
un-stuck that (even with additional picked patches again already).

Hans
Re: Compiling Xen from source [ In reply to ]
Ok, thanks.
So I have done a fresh checkout of the master branch, but unfortunately,
there's a bug in infiniband drivers.
the error is : "drivers/infiniband/flexboot_nodnic.c:368:53: error:
implicit conversion from 'enum ib_queue_pair_type' to
'nodnic_queue_pair_type' [-Werror=enum-conversion]".
this occurs with gcc 10 and developers seem to be aware of this:

https://lists.ipxe.org/pipermail/ipxe-devel/2020-May/007049.html

but it seems the master branch has not been patched yet.

Anyway now I will concentrate on 4.14 if I need to compile from source :-)

David


Le mer. 16 sept. 2020 à 22:59, Hans van Kranenburg <hans@knorrie.org> a
écrit :

> On 9/16/20 6:12 PM, Olaf Hering wrote:
> > On Wed, Sep 16, David I wrote:
> >
> >> So, how did the debian package was compiled ? is this the same source
> code with
> >> different options ?
> >
> > Xen 4.11 is from 2018. Use also a compiler from that year.
> > Using this years compiler will lead to errors...
>
> In Debian, Buster with Xen 4.11 uses gcc 8.
>
> The Xen 4.11 that is in Debian unstable now does not compile any more
> with gcc 10. That's why we really need to get Xen 4.14 in there now to
> un-stuck that (even with additional picked patches again already).
>
> Hans
>
>
Re: Compiling Xen from source [ In reply to ]
update (for Hans and Ian):

I encounter the same issue when compiling the 4.14 branch:


git checkout -b stable-4.14 origin/stable-4.14
./configure && make

make[8]: Entering directory '/root/xen/tools/firmware/etherboot/ipxe/src'
[BUILD] bin/flexboot_nodnic.o
drivers/infiniband/flexboot_nodnic.c: In function
'flexboot_nodnic_create_qp':
drivers/infiniband/flexboot_nodnic.c:368:53: error: implicit conversion
from 'enum ib_queue_pair_type' to 'nodnic_queue_pair_type'
[-Werror=enum-conversion]

this is with gcc 10 indeed.
Anyway I'm trying to compile under xen4.11 (binary debian package) that is
working perfectly.
But the compiling process of xen tools is bugged because of the drivers.
for example i have tried to comment the enum-conversion warning in the
Makefile, but i encounter other bugs with other drivers.

David

Le jeu. 17 sept. 2020 à 20:05, David I <dav.sec.lists@gmail.com> a écrit :

> Ok, thanks.
> So I have done a fresh checkout of the master branch, but unfortunately,
> there's a bug in infiniband drivers.
> the error is : "drivers/infiniband/flexboot_nodnic.c:368:53: error:
> implicit conversion from 'enum ib_queue_pair_type' to
> 'nodnic_queue_pair_type' [-Werror=enum-conversion]".
> this occurs with gcc 10 and developers seem to be aware of this:
>
> https://lists.ipxe.org/pipermail/ipxe-devel/2020-May/007049.html
>
> but it seems the master branch has not been patched yet.
>
> Anyway now I will concentrate on 4.14 if I need to compile from source :-)
>
> David
>
>
> Le mer. 16 sept. 2020 à 22:59, Hans van Kranenburg <hans@knorrie.org> a
> écrit :
>
>> On 9/16/20 6:12 PM, Olaf Hering wrote:
>> > On Wed, Sep 16, David I wrote:
>> >
>> >> So, how did the debian package was compiled ? is this the same source
>> code with
>> >> different options ?
>> >
>> > Xen 4.11 is from 2018. Use also a compiler from that year.
>> > Using this years compiler will lead to errors...
>>
>> In Debian, Buster with Xen 4.11 uses gcc 8.
>>
>> The Xen 4.11 that is in Debian unstable now does not compile any more
>> with gcc 10. That's why we really need to get Xen 4.14 in there now to
>> un-stuck that (even with additional picked patches again already).
>>
>> Hans
>>
>>
Re: Compiling Xen from source [ In reply to ]
Hello,

for information, I was able to compile xen from source on Debian 10, with
the following configuration:


david@debian:~/xen/xen$ uname -a
Linux debian 5.7.0-2-amd64 #1 SMP Debian 5.7.10-1 (2020-07-26) x86_64
GNU/Linux
david@debian:~/xen/xen$ gcc --version
gcc (Debian 8.4.0-4) 8.4.0
Copyright © 2018 Free Software Foundation, Inc.
Ce logiciel est un logiciel libre; voir les sources pour les conditions de
copie. Il n'y a
AUCUNE GARANTIE, pas même pour la COMMERCIALISATION ni L'ADÉQUATION À UNE
TÂCHE PARTICULIÈRE.

david@debian:~/xen/xen$ git branch
master
stable-4.11
* stable-4.13
stable-4.14
staging
david@debian:~/xen/xen$

with a simple: "./configure && make dist".
But I had to modify some config files following the thread here:

http://buildroot-busybox.2317881.n4.nabble.com/PATCH-1-2-xen-Disable-Werror-when-building-td145149.html

the first build failed with a message: "configure test passed without
-werror but failed with -error".
I had to resolve this to edit these files:

xen/Config.mk:

+-HOSTCFLAGS = -Wall -Werror -Wstrict-prototypes -O2 -fomit-frame-pointer
++HOSTCFLAGS = -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer

xen/xen/arch/x86/Rules.mk:

+-CFLAGS += '-D__OBJECT_LABEL__=$(subst /,$$,$(subst -,_,$(subst
$(BASEDIR)/,,$(CURDIR))/$@))'
++CFLAGS += -U__OBJECT_LABEL__ '-D__OBJECT_LABEL__=$(subst /,$$,$(subst
-,_,$(subst $(BASEDIR)/,,$(CURDIR))/$@))'

xen/Rules.mk:
+-CFLAGS += '-D__OBJECT_FILE__="$@"'
++CFLAGS += -U__OBJECT_FILE__ '-D__OBJECT_FILE__="$@"'

I haven't tested the binary yet.

Kind regards,

David

Le mer. 16 sept. 2020 à 22:59, Hans van Kranenburg <hans@knorrie.org> a
écrit :

> On 9/16/20 6:12 PM, Olaf Hering wrote:
> > On Wed, Sep 16, David I wrote:
> >
> >> So, how did the debian package was compiled ? is this the same source
> code with
> >> different options ?
> >
> > Xen 4.11 is from 2018. Use also a compiler from that year.
> > Using this years compiler will lead to errors...
>
> In Debian, Buster with Xen 4.11 uses gcc 8.
>
> The Xen 4.11 that is in Debian unstable now does not compile any more
> with gcc 10. That's why we really need to get Xen 4.14 in there now to
> un-stuck that (even with additional picked patches again already).
>
> Hans
>
>