Mailing List Archive

[PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time
It is sometimes hard to discover all the optional tools that should be
on a system to build all available Xen documentation. By checking for
documentation generation tools at ./configure time and displaying a
warning, Xen packagers will more easily learn about new optional build
dependencies, like markdown, when they are introduced.

Signed-off-by: Matt Wilson <msw@amazon.com>

---
Changes since v1:
* require that ./configure be run before building docs
* remove Docs.mk and make Tools.mk the canonical locaiton where
docs tools are defined (via ./configure)
* fold in checking for markdown_py

Changes since v2:
* drop the AX_DOCS_TOOL_PROG variant, just use AX_DOCS_TOOL_PROGS
* use the lowercase version of the autoconf variable name for the
help text

diff -r d7e4efa17fb0 -r 3b3582f4dc42 README
--- a/README Tue Aug 28 15:35:08 2012 -0700
+++ b/README Thu Sep 06 21:31:51 2012 -0700
@@ -28,8 +28,9 @@
your system. For full documentation, see the Xen User Manual. If this
is a pre-built release then you can find the manual at:
dist/install/usr/share/doc/xen/pdf/user.pdf
-If you have a source release, then 'make -C docs' will build the
-manual at docs/pdf/user.pdf.
+If you have a source release and the required documentation generation
+tools, then './configure; make -C docs' will build the manual at
+docs/pdf/user.pdf.

Quick-Start Guide
=================
@@ -59,7 +60,6 @@
* GNU gettext
* 16-bit x86 assembler, loader and compiler (dev86 rpm or bin86 & bcc debs)
* ACPI ASL compiler (iasl)
- * markdown

In addition to the above there are a number of optional build
prerequisites. Omitting these will cause the related features to be
@@ -67,6 +67,7 @@
* Development install of Ocaml (e.g. ocaml-nox and
ocaml-findlib). Required to build ocaml components which
includes the alternative ocaml xenstored.
+ * markdown

Second, you need to acquire a suitable kernel for use in domain 0. If
possible you should use a kernel provided by your OS distributor. If
diff -r d7e4efa17fb0 -r 3b3582f4dc42 config/Tools.mk.in
--- a/config/Tools.mk.in Tue Aug 28 15:35:08 2012 -0700
+++ b/config/Tools.mk.in Thu Sep 06 21:31:51 2012 -0700
@@ -22,6 +22,18 @@
LD86 := @LD86@
BCC := @BCC@
IASL := @IASL@
+PS2PDF := @PS2PDF@
+DVIPS := @DVIPS@
+LATEX := @LATEX@
+FIG2DEV := @FIG2DEV@
+LATEX2HTML := @LATEX2HTML@
+DOXYGEN := @DOXYGEN@
+POD2MAN := @POD2MAN@
+POD2HTML := @POD2HTML@
+POD2TEXT := @POD2TEXT@
+DOT := @DOT@
+NEATO := @NEATO@
+MARKDOWN := @MARKDOWN@

# Extra folder for libs/includes
PREPEND_INCLUDES := @PREPEND_INCLUDES@
diff -r d7e4efa17fb0 -r 3b3582f4dc42 docs/Makefile
--- a/docs/Makefile Tue Aug 28 15:35:08 2012 -0700
+++ b/docs/Makefile Thu Sep 06 21:31:51 2012 -0700
@@ -2,7 +2,7 @@

XEN_ROOT=$(CURDIR)/..
include $(XEN_ROOT)/Config.mk
-include $(XEN_ROOT)/docs/Docs.mk
+-include $(XEN_ROOT)/config/Tools.mk

VERSION = xen-unstable

@@ -26,10 +26,12 @@

.PHONY: build
build: html txt man-pages
- @if which $(DOT) 1>/dev/null 2>/dev/null ; then \
- $(MAKE) -C xen-api build ; else \
- echo "Graphviz (dot) not installed; skipping xen-api." ; fi
+ifdef DOT
+ $(MAKE) -C xen-api build
rm -f *.aux *.dvi *.bbl *.blg *.glo *.idx *.ilg *.log *.ind *.toc
+else
+ @echo "Graphviz (dot) not installed; skipping xen-api."
+endif

.PHONY: dev-docs
dev-docs: python-dev-docs
@@ -42,18 +44,21 @@

.PHONY: python-dev-docs
python-dev-docs:
- @mkdir -v -p api/tools/python
- @set -e ; if which $(DOXYGEN) 1>/dev/null 2>/dev/null; then \
- echo "Running doxygen to generate Python tools APIs ... "; \
- $(DOXYGEN) Doxyfile; \
- $(MAKE) -C api/tools/python/latex ; else \
- echo "Doxygen not installed; skipping python-dev-docs."; fi
+ifdef DOXYGEN
+ @echo "Running doxygen to generate Python tools APIs ... "
+ mkdir -v -p api/tools/python
+ $(DOXYGEN) Doxyfile && $(MAKE) -C api/tools/python/latex
+else
+ @echo "Doxygen not installed; skipping python-dev-docs."
+endif

.PHONY: man-pages
man-pages:
- @if which $(POD2MAN) 1>/dev/null 2>/dev/null; then \
- $(MAKE) $(DOC_MAN1) $(DOC_MAN5); else \
- echo "pod2man not installed; skipping man-pages."; fi
+ifdef POD2MAN
+ $(MAKE) $(DOC_MAN1) $(DOC_MAN5)
+else
+ @echo "pod2man not installed; skipping man-pages."
+endif

man1/%.1: man/%.pod.1 Makefile
$(INSTALL_DIR) $(@D)
@@ -94,30 +99,40 @@
perl -w -- ./gen-html-index -i INDEX html $(DOC_HTML)

html/%.html: %.markdown
- @$(INSTALL_DIR) $(@D)
- @set -e ; if which $(MARKDOWN) 1>/dev/null 2>/dev/null; then \
- echo "Running markdown to generate $*.html ... "; \
+ $(INSTALL_DIR) $(@D)
+ifdef MARKDOWN
+ @echo "Running markdown to generate $*.html ... "
$(MARKDOWN) $< > $@.tmp ; \
- $(call move-if-changed,$@.tmp,$@) ; else \
- echo "markdown not installed; skipping $*.html."; fi
+ $(call move-if-changed,$@.tmp,$@)
+else
+ @echo "markdown not installed; skipping $*.html."
+endif

html/%.txt: %.txt
- @$(INSTALL_DIR) $(@D)
+ $(INSTALL_DIR) $(@D)
cp $< $@

html/man/%.1.html: man/%.pod.1 Makefile
$(INSTALL_DIR) $(@D)
+ifdef POD2HTML
$(POD2HTML) --infile=$< --outfile=$@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "pod2html not installed; skipping $<."
+endif

html/man/%.5.html: man/%.pod.5 Makefile
$(INSTALL_DIR) $(@D)
+ifdef POD2HTML
$(POD2HTML) --infile=$< --outfile=$@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "pod2html not installed; skipping $<."
+endif

html/hypercall/index.html: ./xen-headers
rm -rf $(@D)
- @$(INSTALL_DIR) $(@D)
+ $(INSTALL_DIR) $(@D)
./xen-headers -O $(@D) \
-T 'arch-x86_64 - Xen public headers' \
-X arch-ia64 -X arch-x86_32 -X xen-x86_32 -X arch-arm \
@@ -137,11 +152,24 @@

txt/man/%.1.txt: man/%.pod.1 Makefile
$(INSTALL_DIR) $(@D)
+ifdef POD2TEXT
$(POD2TEXT) $< $@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "pod2text not installed; skipping $<."
+endif

txt/man/%.5.txt: man/%.pod.5 Makefile
$(INSTALL_DIR) $(@D)
+ifdef POD2TEXT
$(POD2TEXT) $< $@.tmp
$(call move-if-changed,$@.tmp,$@)
+else
+ @echo "pod2text not installed; skipping $<."
+endif

+ifeq (,$(findstring clean,$(MAKECMDGOALS)))
+$(XEN_ROOT)/config/Tools.mk:
+ $(error You have to run ./configure before building docs)
+endif
+
diff -r d7e4efa17fb0 -r 3b3582f4dc42 docs/xen-api/Makefile
--- a/docs/xen-api/Makefile Tue Aug 28 15:35:08 2012 -0700
+++ b/docs/xen-api/Makefile Thu Sep 06 21:31:51 2012 -0700
@@ -2,7 +2,7 @@

XEN_ROOT=$(CURDIR)/../..
include $(XEN_ROOT)/Config.mk
-include $(XEN_ROOT)/docs/Docs.mk
+-include $(XEN_ROOT)/config/Tools.mk


TEX := $(wildcard *.tex)
@@ -42,3 +42,8 @@
.PHONY: clean
clean:
rm -f *.pdf *.ps *.dvi *.aux *.log *.out $(EPSDOT)
+
+ifeq (,$(findstring clean,$(MAKECMDGOALS)))
+$(XEN_ROOT)/config/Tools.mk:
+ $(error You have to run ./configure before building docs)
+endif
diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/configure.ac
--- a/tools/configure.ac Tue Aug 28 15:35:08 2012 -0700
+++ b/tools/configure.ac Thu Sep 06 21:31:51 2012 -0700
@@ -34,6 +34,7 @@
m4_include([m4/curses.m4])
m4_include([m4/pthread.m4])
m4_include([m4/ptyfuncs.m4])
+m4_include([m4/docs_tool.m4])

# Enable/disable options
AX_ARG_DEFAULT_DISABLE([githttp], [Download GIT repositories via HTTP])
@@ -80,6 +81,18 @@
AC_PATH_PROG([BISON], [bison])
AC_PATH_PROG([FLEX], [flex])
AX_PATH_PROG_OR_FAIL([PERL], [perl])
+AX_DOCS_TOOL_PROGS([PS2PDF], [ps2pdf])
+AX_DOCS_TOOL_PROGS([DVIPS], [dvips])
+AX_DOCS_TOOL_PROGS([LATEX], [latex])
+AX_DOCS_TOOL_PROGS([FIG2DEV], [fig2dev])
+AX_DOCS_TOOL_PROGS([LATEX2HTML], [latex2html])
+AX_DOCS_TOOL_PROGS([DOXYGEN], [doxygen])
+AX_DOCS_TOOL_PROGS([POD2MAN], [pod2man])
+AX_DOCS_TOOL_PROGS([POD2HTML], [pod2html])
+AX_DOCS_TOOL_PROGS([POD2TEXT], [pod2text])
+AX_DOCS_TOOL_PROGS([DOT], [dot])
+AX_DOCS_TOOL_PROGS([NEATO], [neato])
+AX_DOCS_TOOL_PROGS([MARKDOWN], [markdown markdown_py])
AS_IF([test "x$xapi" = "xy"], [
AX_PATH_PROG_OR_FAIL([CURL], [curl-config])
AX_PATH_PROG_OR_FAIL([XML], [xml2-config])
diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/m4/docs_tool.m4
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/tools/m4/docs_tool.m4 Thu Sep 06 21:31:51 2012 -0700
@@ -0,0 +1,10 @@
+# $1: autoconf variable name
+# $2: list of programs to check for
+AC_DEFUN([AX_DOCS_TOOL_PROGS], [
+dnl
+ AC_ARG_VAR([$1], [Path to ] m4_tolower($1) [ tool])
+ AC_PATH_PROGS([$1], [$2])
+ AS_IF([! test -x "$ac_cv_path_$1"], [.
+ AC_MSG_WARN([$2 is not available so some documentation won't be built])
+ ])
+])

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
> diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/m4/docs_tool.m4
> --- /dev/null Thu Jan 01 00:00:00 1970 +0000
> +++ b/tools/m4/docs_tool.m4 Thu Sep 06 21:31:51 2012 -0700
> @@ -0,0 +1,10 @@
> +# $1: autoconf variable name
> +# $2: list of programs to check for
> +AC_DEFUN([AX_DOCS_TOOL_PROGS], [
> +dnl
> + AC_ARG_VAR([$1], [Path to ] m4_tolower($1) [ tool])
> + AC_PATH_PROGS([$1], [$2])
> + AS_IF([! test -x "$ac_cv_path_$1"], [.
> + AC_MSG_WARN([$2 is not available so some documentation won't be built])

Did you mean m4_tolower($1) for this last $2 as well? Since $2 is the
list of potential command names this will look a bit odd I think?



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
On Mon, Sep 10, 2012 at 05:09:10PM +0100, Ian Campbell wrote:
>
> > diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/m4/docs_tool.m4
> > --- /dev/null Thu Jan 01 00:00:00 1970 +0000
> > +++ b/tools/m4/docs_tool.m4 Thu Sep 06 21:31:51 2012 -0700
> > @@ -0,0 +1,10 @@
> > +# $1: autoconf variable name
> > +# $2: list of programs to check for
> > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [
> > +dnl
> > + AC_ARG_VAR([$1], [Path to ] m4_tolower($1) [ tool])
> > + AC_PATH_PROGS([$1], [$2])
> > + AS_IF([! test -x "$ac_cv_path_$1"], [.
> > + AC_MSG_WARN([$2 is not available so some documentation won't be built])
>
> Did you mean m4_tolower($1) for this last $2 as well? Since $2 is the
> list of potential command names this will look a bit odd I think?

Yea, that could be better. Though I kind of like that it shows the
programs that were checked. It'd look like:

markdown markdown_py is not available so some documentation won't be built

Maybe just drop 'is' to avoid the verb mismatch.

Matt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
On Mon, 2012-09-10 at 21:46 +0100, Matt Wilson wrote:
> On Mon, Sep 10, 2012 at 05:09:10PM +0100, Ian Campbell wrote:
> >
> > > diff -r d7e4efa17fb0 -r 3b3582f4dc42 tools/m4/docs_tool.m4
> > > --- /dev/null Thu Jan 01 00:00:00 1970 +0000
> > > +++ b/tools/m4/docs_tool.m4 Thu Sep 06 21:31:51 2012 -0700
> > > @@ -0,0 +1,10 @@
> > > +# $1: autoconf variable name
> > > +# $2: list of programs to check for
> > > +AC_DEFUN([AX_DOCS_TOOL_PROGS], [
> > > +dnl
> > > + AC_ARG_VAR([$1], [Path to ] m4_tolower($1) [ tool])
> > > + AC_PATH_PROGS([$1], [$2])
> > > + AS_IF([! test -x "$ac_cv_path_$1"], [.
> > > + AC_MSG_WARN([$2 is not available so some documentation won't be built])
> >
> > Did you mean m4_tolower($1) for this last $2 as well? Since $2 is the
> > list of potential command names this will look a bit odd I think?
>
> Yea, that could be better. Though I kind of like that it shows the
> programs that were checked. It'd look like:
>
> markdown markdown_py is not available so some documentation won't be built
>
> Maybe just drop 'is' to avoid the verb mismatch.

Or "m4_tolower($1) ($2) is not available ..."? (properly m4 quoted by
someone who knows what they are doing, of course ;-))

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
On Tue, Sep 11, 2012 at 08:54:10AM +0100, Ian Campbell wrote:
> >
> > Yea, that could be better. Though I kind of like that it shows the
> > programs that were checked. It'd look like:
> >
> > markdown markdown_py is not available so some documentation won't be built
> >
> > Maybe just drop 'is' to avoid the verb mismatch.
>
> Or "m4_tolower($1) ($2) is not available ..."? (properly m4 quoted by
> someone who knows what they are doing, of course ;-))

Right, so the desired output would be:
markdown (markdown markdown_py) is not available so some documentation won't be built

I can work on that, but it'll probably be a few days before I can get
back to it.

Matt


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
On Fri, 2012-09-07 at 05:35 +0100, Matt Wilson wrote:
> It is sometimes hard to discover all the optional tools that should be
> on a system to build all available Xen documentation. By checking for
> documentation generation tools at ./configure time and displaying a
> warning, Xen packagers will more easily learn about new optional build
> dependencies, like markdown, when they are introduced.
>
> Signed-off-by: Matt Wilson <msw@amazon.com>

While investigating an (unrelated) issue with the cronjob that generates
http://xenbits.xen.org/docs/unstable/ I had a sudden though about a
downside of this approach.

Trying to run ./configure on xenbits results in:
$ ./configure
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking for gcc... no
checking for cc... no
checking for cl.exe... no
configure: error: in `/home/xendocs/cronjobs/working/docs/unstable/tree/tools':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details

Which is a bit annoying if all you wanted was to build docs. I'm not
sure I really want to advocate putting gcc on xenbits. I suppose I could
put a dummy shell script in $PATH.

AIUI autoconf allows you to define that a configure script calls a
sub-configure script. Could we use this at the toplevel to call both
tools/configure and a new docs/configure? Then my docs scripts could
just call docs/configure?

FWIW the unrelated issue was:
$ make -C docs/ clean
/bin/sh: gcc: not found
/bin/sh: arithmetic expression: expecting primary: ""
which I think is down to the gcc version check in Config.mk.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
On Mon, Sep 17, 2012 at 10:42:44AM +0100, Ian Campbell wrote:
> configure: error: in `/home/xendocs/cronjobs/working/docs/unstable/tree/tools':
> configure: error: no acceptable C compiler found in $PATH
> See `config.log' for more details
>
> Which is a bit annoying if all you wanted was to build docs. I'm not
> sure I really want to advocate putting gcc on xenbits. I suppose I could
> put a dummy shell script in $PATH.

./configure is probably going to need a compiler that produces
executables.

> AIUI autoconf allows you to define that a configure script calls a
> sub-configure script. Could we use this at the toplevel to call both
> tools/configure and a new docs/configure? Then my docs scripts could
> just call docs/configure?

This sounds like a better idea.

> FWIW the unrelated issue was:
> $ make -C docs/ clean
> /bin/sh: gcc: not found
> /bin/sh: arithmetic expression: expecting primary: ""
> which I think is down to the gcc version check in Config.mk.

While we're discussing expanding the coverage of ./configure, I
wonder: what (if any) is the resistance to requiring ./configure to
build the xen/ tree?

Matt

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
On Mon, 2012-09-17 at 18:08 +0100, Matt Wilson wrote:
> On Mon, Sep 17, 2012 at 10:42:44AM +0100, Ian Campbell wrote:
> > configure: error: in `/home/xendocs/cronjobs/working/docs/unstable/tree/tools':
> > configure: error: no acceptable C compiler found in $PATH
> > See `config.log' for more details
> >
> > Which is a bit annoying if all you wanted was to build docs. I'm not
> > sure I really want to advocate putting gcc on xenbits. I suppose I could
> > put a dummy shell script in $PATH.
>
> ./configure is probably going to need a compiler that produces
> executables.

True.

>
> > AIUI autoconf allows you to define that a configure script calls a
> > sub-configure script. Could we use this at the toplevel to call both
> > tools/configure and a new docs/configure? Then my docs scripts could
> > just call docs/configure?
>
> This sounds like a better idea.
>
> > FWIW the unrelated issue was:
> > $ make -C docs/ clean
> > /bin/sh: gcc: not found
> > /bin/sh: arithmetic expression: expecting primary: ""
> > which I think is down to the gcc version check in Config.mk.
>
> While we're discussing expanding the coverage of ./configure, I
> wonder: what (if any) is the resistance to requiring ./configure to
> build the xen/ tree?

The hypervisor maintainers were pretty much unanimously against it.

There's some discussion in the archives around the time autoconf was
added and even a patch (applied and now reverted IIRC) to do as you
suggest.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
Matt Wilson writes ("Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time"):
> On Mon, Sep 17, 2012 at 10:42:44AM +0100, Ian Campbell wrote:
> > configure: error: in `/home/xendocs/cronjobs/working/docs/unstable/tree/tools':
> > configure: error: no acceptable C compiler found in $PATH
> > See `config.log' for more details
> >
> > Which is a bit annoying if all you wanted was to build docs. I'm not
> > sure I really want to advocate putting gcc on xenbits. I suppose I could
> > put a dummy shell script in $PATH.
>
> ./configure is probably going to need a compiler that produces
> executables.

Not necessarily. If we don't ask for AC_CC or whatever it is, it
shouldn't.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [PATCH 1 of 2 v3] tools: check for documentation generation tools at configure time [ In reply to ]
At 10:08 -0700 on 17 Sep (1347876501), Matt Wilson wrote:
> While we're discussing expanding the coverage of ./configure, I
> wonder: what (if any) is the resistance to requiring ./configure to
> build the xen/ tree?

There's nothing in the hypervisor build that needs to be adjusted to
match the run-time userspace. So if you're a hypervisor maintainer it
just introduces all the charming idiosyncracies of autotools without
providing any benefit. E.g., this:

checking for yajl_alloc in -lyajl... no
configure: error: Could not find yajl

should not stop me building Xen. Also, as I said before, autoconf is
the thin end of a slippery wedge: the answer to any autotools problem is
always more autotools, and I choose to make my stand here rather than
have the argument later about why we shouldn't be using libtool. :)

Tim.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel