Mailing List Archive

Wiki for official docs only
Hi all,

I know an official Gentoo Wiki has been discussed fairly recently, but
I'd like to throw a different spin out: A wiki for official
documentation (handbooks, other guides) only.

The wiki would only be editable by Gentoo Devs. For authentication, I'd
be surprised if there's not an extension that can hook into the existing
LDAP or whatever databases.

The basic idea is to replace the current documentation with something
that's much easier to edit.

I recently looked at editing the handbooks to provide patches to update
them for autobuilds, death of GRP, using "eselect profile" for profile
management, etc. My problem is I have limited time, and while I have
tackled GuideXML in the past, it's not something I enjoy doing at all -
I have to lookup the syntax every time and double check everything. On
top of that the Handbooks in particular are (at a glance) a maze of
multiple files pieced together into the finished product (I haven't got
round to checking whether this maze is mapped out anywhere yet - I had
multiple things on my todo list at the time and decided to just move on
to the next item).

On the other hand, I (and I suspect a large number of other people in
general) use wiki's quite a lot, am familiar with the syntax and find
them a breeze to edit. In my opinion, it is likely the the official
documentation would receive more, faster contributions if they were on a
wiki instead of built using GuideXML.

Providing static copies shouldn't be an issue. There are various
extensions to provide wiki documents in various formats and I believe
that, if a system doesn't already exist, one could be built relatively
easily, to allow automatic generation of static versions.

I think that even if some documentation ends up having to be duplicated
(eg. between the x86 and amd64 handbooks), the losses through such
duplication would be outweighed by improved contributions.

I haven't mentioned an engine at all, but I'd suggest mediawiki because
it's popular, has a lot of extensions already available and the syntax
is familiar to many.

Thoughts? Concerns?

AllenJB
Re: Wiki for official docs only [ In reply to ]
AllenJB wrote:

> The basic idea is to replace the current documentation with something
> that's much easier to edit.
>

> On top of that the Handbooks in particular are (at a glance) a maze of
> multiple files pieced together into the finished product (I haven't got
> round to checking whether this maze is mapped out anywhere yet - I had
> multiple things on my todo list at the time and decided to just move on
> to the next item).

Easy, refresh your anoncvs, create a debug.xml file in your document root that
contains
<debug on="1"/>

and voilà, you see the file names.

> On the other hand, I (and I suspect a large number of other people in
> general) use wiki's quite a lot, am familiar with the syntax and find
> them a breeze to edit. In my opinion, it is likely the the official
> documentation would receive more, faster contributions if they were on a
> wiki instead of built using GuideXML.

In other words, you want the current docs team to learn a new syntax and a new
tool, and infra to install a new system because you can't learn a dozen simple
tags, you know like <p> for a paragraph...


Wkr,
--
/ Xavier Neys
\_ Gentoo Documentation Project
/
/\ http://www.gentoo.org/doc/en/
Re: Wiki for official docs only [ In reply to ]
Xavier Neys wrote:
> AllenJB wrote:
>
>> The basic idea is to replace the current documentation with something
>> that's much easier to edit.
>>
>
>> On top of that the Handbooks in particular are (at a glance) a maze of
>> multiple files pieced together into the finished product (I haven't got
>> round to checking whether this maze is mapped out anywhere yet - I had
>> multiple things on my todo list at the time and decided to just move on
>> to the next item).
>
> Easy, refresh your anoncvs, create a debug.xml file in your document
> root that contains
> <debug on="1"/>
>
> and voilà, you see the file names.

Thanks for the tip.

>
>> On the other hand, I (and I suspect a large number of other people in
>> general) use wiki's quite a lot, am familiar with the syntax and find
>> them a breeze to edit. In my opinion, it is likely the the official
>> documentation would receive more, faster contributions if they were on a
>> wiki instead of built using GuideXML.
>
> In other words, you want the current docs team to learn a new syntax and
> a new tool, and infra to install a new system because you can't learn a
> dozen simple tags, you know like <p> for a paragraph...
>
>
> Wkr,

The existing docs team may know GuideXML well, but it seems to me that
the existing docs team isn't sufficient and needs some new blood that it
currently isn't getting. 2008.0 has been deprecated for months,
autobuilds have been out for months, eselect profile is now available
for profile selection and yet the handbooks haven't been updated at all
(they even still mention the GRP, which iirc was never released for 2008.0!)

How many of the existing docs team have never worked with mediawiki (or
another wiki - I'm using mediawiki as an example because I suspect it's
the most likely choice)? It might be an idea to do a quick survey before
claiming the entire team would have to learn a new syntax - you might
find that everyone on the docs team is already familiar with mediawiki.
Even if they're not, mediawiki syntax is extremely well documented and
easy to learn, in my experience.

At the end up the day, this is a suggestion. I've made the suggestion
because I think that the way Gentoo documentation currently works (ie.
because it's written in GuideXML) is cumbersome and makes it difficult
to easily pick up. I believe this has led to a lack of volunteers with
the available time and inclination to learn GuideXML just for Gentoo
documentation work, which has led to the current situation with sorely
needed handbook updates not getting done in a timely manner.

Following the documented instructions, you have to check out a large cvs
tree, which takes some time and disk space, set up gorg (which in my
opinion isn't very well documented) and possibly a web server (I think
gorg may be able to run standalone, but I don't know how well it works
and I could be wrong). Then you have to learn GuideXML. Then you have to
learn how the handbook files are structured. Only then might a newbie to
Gentoo documentation editing be able to actually start editing some docs.

Having a mediawiki based setup would, in my opinion, allow newbies to
get involved much more quickly. A local install of mediawiki and apache
takes literally minutes. A "fast" install would be something like:
1) emerge apache mediawiki;
2) Create database (& db user) (I think instructions for this are in the
post-install for mediawiki);
3) Configure mediawiki using the included wizard). You can easily export
content from one wiki to import to another.

As stated before, I believe many are already familiar with mediawiki
syntax due to its already extensive use in open source documentation and
it's extremely well documented and easy to learn, with lots of support
already available.

I realize that there's work for infra involved in installing and
maintaining a wiki. I realized that there's possibly work for some of
the docs team in learning a new system and in initially porting
documentation over (but some of this porting work may be offset by
people like me who already know mediawiki well who would be willing to
lend a hand in this task). However, I believe that in the long term, the
benefits will easily outweigh these issues.

(Sorry if that's a bit long-winded. It started as a 2 paragraph reply
and kept growing =o )

AllenJB
Re: Wiki for official docs only [ In reply to ]
AllenJB wrote:
> The existing docs team may know GuideXML well, but it seems to me that
> the existing docs team isn't sufficient and needs some new blood that it
> currently isn't getting. 2008.0 has been deprecated for months,
> autobuilds have been out for months, eselect profile is now available
> for profile selection and yet the handbooks haven't been updated at all
> (they even still mention the GRP, which iirc was never released for 2008.0!)

I dunno if new blood is needed . . . we have perfectly capable folks in
the docs team. They've been around for a few (or more) years, and
they've invested the time and trouble to write perfect GuideXML. The
problem is getting them up *off* their asses to do the work.

> How many of the existing docs team have never worked with mediawiki (or
> another wiki - I'm using mediawiki as an example because I suspect it's
> the most likely choice)?

I sure don't know how the hell it works. I've half-heartedly poked at
Wikipedia and the old gentoo-wiki when trying to fix really egregious
errors, but it's still nigh-uncomprehensible.

> Following the documented instructions, you have to check out a large cvs
> tree, which takes some time and disk space, set up gorg (which in my
> opinion isn't very well documented) and possibly a web server (I think
> gorg may be able to run standalone, but I don't know how well it works
> and I could be wrong). Then you have to learn GuideXML. Then you have to
> learn how the handbook files are structured. Only then might a newbie to
> Gentoo documentation editing be able to actually start editing some docs.

On an unoptimized ReiserFS v3 layout, ~/cvs/gentoo/xml takes up 281MB.
More than 82MB of that is images.

I've never needed to compile and install gorg. In fact, I've refused to,
since I don't want to install a webserver just to work with docs, and I
don't want the bloat of Ruby for the same reason. I just use gvim and
nano. To each their own.

Also, we *do* have good documents for writing GuideXML, and we even tell
users how to look at the code for our existing docs to get a feel for
them. Heck, just the other day, someone I've never run into before
turned in an almost flawless new document to bugzilla.
http://bugs.gentoo.org/show_bug.cgi?id=275816

None of us helped him with his GuideXML as far as I know. This one came
out of the blue -- and it's a pretty damned fantastic doc. Users *are*
more capable than either of us typically give them credit. :)

> As stated before, I believe many are already familiar with mediawiki
> syntax due to its already extensive use in open source documentation and
> it's extremely well documented and easy to learn, with lots of support
> already available.

I don't know who these many others are, but I know the docs team doesn't
know the syntax. I, myself, find any and all wiki syntax completely
illegible and very difficult to parse.

As the most (only?) active member of the GDP, I can tell you that I'd
probably quit if we switched to a wiki right now.

It's too much upheaval to try to switch everything over to a wiki. We'd
need all hands on deck for several months just to get our existing
content base over, and that doesn't take into account the continual
influx of new bugs, updates, and whatnot that would roll in during that
time. Trying to do all that with just one person . . . not gonna happen.

> I realize that there's work for infra involved in installing and
> maintaining a wiki. I realized that there's possibly work for some of
> the docs team in learning a new system and in initially porting
> documentation over (but some of this porting work may be offset by
> people like me who already know mediawiki well who would be willing to
> lend a hand in this task). However, I believe that in the long term, the
> benefits will easily outweigh these issues.

Wikis are huge security risks, which is why historically infra has never
wanted to run a public wiki. In the previous (recent) three discussions
of the idea of an "official" sanctioned wiki, infra has told the GDP on
the MLs and on IRC that they are reluctant to run a public wiki because
of the security risks.
Re: Wiki for official docs only [ In reply to ]
Keep up the good work team, I prefer Gentoo official docs over
mediawiki since it's easier to maintain on the back-end.

Perhaps as a happy medium someone could demonstrate how a mediawiki
interface could be used for writing documents which can be converted
into the GuideXML system.

James Robertson (Gentoo enthusiast),
Edinburgh, UK
Re: Wiki for official docs only [ In reply to ]
Hi Josh,

Josh Saddler <nightmorph <at> gentoo.org> writes:
> > How many of the existing docs team have never worked with mediawiki (or
> > another wiki - I'm using mediawiki as an example because I suspect it's
> > the most likely choice)?
>
> I sure don't know how the hell it works. I've half-heartedly poked at
> Wikipedia and the old gentoo-wiki when trying to fix really egregious
> errors, but it's still nigh-uncomprehensible.

I for one find Wiki Syntax a lot easier to read since the text-only (source)
version looks not much different from the rendered version. There are two main
advantages to the existing docs:

1) The preview feature is built-in

I have written and edited some GuideXML documents. I have never done so without
making a mistake. Getting the document rendered is a huge hassle (for me). I
need to install a web server and some additions (which I need root for usually).
Or I need to ssh to dev and see the doc there. Images and links will break.

2) The change submission system is built in

This is two sided: To commit something I do not need CVS, saving is built in. I
do not need to prepare unreadable* patches. I do not need Bugzilla to make a
change, and merging changes is a one-click operation for the wiki editors.
MediaWiki has all these functions built in. Users can sign up, make a change
from anywhere. And the docs team can use the "reviewed revision" feature to hide
changes from others until they are reviewed.

* because of realignment of word wraps, for instance


> I don't know who these many others are, but I know the docs team doesn't
> know the syntax. I, myself, find any and all wiki syntax completely
> illegible and very difficult to parse.
>
> As the most (only?) active member of the GDP, I can tell you that I'd
> probably quit if we switched to a wiki right now.

Now I'm not telling you to change to a Wiki. I agree the docs team is doing a
great job in keeping their pages up to date. However, I see advantages with
fixing errors in old docs if people can just do updates themselves.

There are also other benefits in having a wiki: Fedora is preparing their
newsletter wiki-style and they just send it when it's done. I does not need the
huge step of integrating articles sent in via email.

You have to acknowledge the fact that there is a market for wikis in general,
and for Gentoo in particular as well (see gentoo-wiki.com). I think it is
something we could use to get fresh blood, and a higher flow into the docs.
And that said, I completely acknowledge the fact that you run the team and it is
ultimately your choice based on taste, experience and different consideration on
argument. I just try to give my view here.


> It's too much upheaval to try to switch everything over to a wiki. We'd
> need all hands on deck for several months just to get our existing
> content base over, and that doesn't take into account the continual
> influx of new bugs, updates, and whatnot that would roll in during that
> time. Trying to do all that with just one person . . . not gonna happen.

Moving the docs is indeed not an easy task. I think the step of converting the
existing docs is feasible to automate. What's not so easy is how we
integrate/replace the existing page structure. Will XML project pages die? What
about all the specific extensions (like auto-generated roll pages, glsa index,
insertion of dev names, etc.).


> Wikis are huge security risks, which is why historically infra has never
> wanted to run a public wiki. In the previous (recent) three discussions
> of the idea of an "official" sanctioned wiki, infra has told the GDP on
> the MLs and on IRC that they are reluctant to run a public wiki because
> of the security risks.

Infra is doing a tremendous amount of work keeping our machines running. But
eventually, their task is to enable Gentoo devs to do their work. If the
community agrees they need a wiki, I am convinced they can make it happen.
We are running a phpBB and Bugzilla instance as well. And MediaWiki has a very
good track record of (1) no high-impact security issues and (2) fixing them fast
and documenting them properly.



Robert