Mailing List Archive

reST + Sphinx (was: [VOTE] Update our documentation building tool chain)
On 2/29/20 11:19 PM, Christophe JAILLET wrote:
> I also had some looks at RST, but IMHO some functionalities available
> with our current framework are not possible with RST.

I took a look at Sphinx this week. Attached are two tarballs, one with
the source and one with a sample of the generated HTML output. (Take a
look at html/mod/core.html.* for the actual content.)

My goal was to try to cover a broad range of things we'd want to do, not
to get it pretty. And I wanted to get it done in a limited amount of
time. So here's the list of things I did:

- I added a Sphinx extension that recognizes the :httpd:directive: reST
directive, which encapsulates some pieces of our existing
<directivesynopsis>.
- I ported the AcceptFilter and Protocol directives from the Core module
documentation.
- I pulled the German and Japanese translations for those pages --
German because I can read it well enough to match up the translations,
and Japanese because I thought it would be a good stress test for the
overall approach.
- I added a simple language switcher to the sidebar.

Known issues:

- Links to directives don't work yet because I haven't taught myself how
to create a sphinx "Domain". This is why you see links called e.g.
:httpd:directive:`AcceptFilter` and so on.
- It's the default theme. Just high-contrast dark grey on white.
- Untranslated paragraphs in German and Japanese aren't called out or
marked in any way; you just suddenly switch to English while you're reading.
- The search index is based on the Japanese docs whether or not you're
currently viewing the Japanese text, oops.

I like the readability of the reST sources compared to our XML solution.
It's easy to write. Vim does the syntax highlighting in a decent way.
Using Sphinx's `samp` domain as a default means that you get
Markdown-style code highlights using inline `backticks`, which is
basically my favorite feature of Markdown anyway. And writing our own
:httpd: directive domain gives us the power to do pretty much whatever
we want as long as we're compatible with the docutils object model.

I'm not as impressed with the documentation for the Sphinx internals or
for docutils, ironically. It was difficult for me to find the
information I needed to build the extension module, with a lot of trial
and error and more than a few completely indecipherable stack traces.
(Sphinx doesn't tell you nicely when you accidentally misuse its APIs,
it just crashes.) I also don't know how people will react to the sudden
inlining of untranslated English for out-of-date pages, but we already
have the .po thread for that discussion.

I added two helper recipes to the standard Sphinx Makefile. They rely on
sphinx-build and sphinx-intl. If you'd like to generate the full output
yourself, run `make html-all` from the source root. And if you want to
make changes to see what the translation workflow is like, then after
you've made updates to the reST source, run `make update-po` to update
the .po snippets in the locales/ folder, perform the translations, and
then `make html-all` again.

I can also take a shot at answering some of your original questions:

> does RST or RST->HTML generator have the following functionalities: -
> sorting: our directives are sorted in the generated files

I don't think there's any builtin way to, for example, reorder sections
automatically. But is there any problem with doing that manually, in
practice? The directive names don't get translated; can't we just keep
ASCII sort order?

> - "theming": I like the way we display links to modules and
> directives with different CSS. Can the same be achieved with RST?
> (not that important anyway, just a mater of taste)

Sphinx allows you to choose a theme, which determines how the page
itself is laid out, and then those themes offer their own options for
CSS customization. If you really don't like what's available you can
always write your own.

> - "syntax checking": XML and underlying DTD have the big advantage to
> check that the input file is correct (i.e. all fields associated to
> the description of a directive are there, and in the correct order
> for example). Can the same "checks" can be achieved one way or
> another with RST?

When writing new Sphinx directives, you get the full generic power of
Python; I think it's likely we could do whatever we wanted. Sphinx uses
a global hook system for extensibility.

--Jacob