Mailing List Archive

On Tue, 16 Nov 2021 at 09:42, Tomasz Konojacki <me@xenu.pl> wrote:

> Tables are one of the most frequently requested features for Pod.
>

+1 on having tables in POD in general.

The following is an observation, rather than a negative vote:

This proposal is inconsistent with the paragraph nature of POD - we end up
with "rectangular POD", where paragraph formatting directives are embedded
within visual sections of the markup rather than starting at the beginning
of each line:

| B<Header 1> | B<Header 2>
|
|-------------------------------|-------------------------------------------------|
|=over 4 |
|
| |
|
|=item * C<$x> - is a variable |=for html <img src="
https://perl.org/logo.bmp"> |
| |
|
|=back |
|

As such, I think it'll be a mess - for authors and toolchain
implementations alike - and we'll end up wishing we'd stuck with markdown.

A paragraph-based table definition would be tedious to write and hard to
visualise, but would be more in the spirit of the existing POD definition.

On Mon, Nov 15, 2021 at 9:52 PM Tom Molesworth via perl5-porters <
perl5-porters@perl.org> wrote:

> On Tue, 16 Nov 2021 at 09:42, Tomasz Konojacki <me@xenu.pl> wrote:
>
>> Tables are one of the most frequently requested features for Pod.
>>
>
> +1 on having tables in POD in general.
>
> The following is an observation, rather than a negative vote:
>
> This proposal is inconsistent with the paragraph nature of POD - we end up
> with "rectangular POD", where paragraph formatting directives are embedded
> within visual sections of the markup rather than starting at the beginning
> of each line:
>
> | B<Header 1> | B<Header 2>
> |
>
> |-------------------------------|-------------------------------------------------|
> |=over 4 |
> |
> | |
> |
> |=item * C<$x> - is a variable |=for html <img src="
> https://perl.org/logo.bmp"> |
> | |
> |
> |=back |
> |
>
> As such, I think it'll be a mess - for authors and toolchain
> implementations alike - and we'll end up wishing we'd stuck with markdown.
>
> A paragraph-based table definition would be tedious to write and hard to
> visualise, but would be more in the spirit of the existing POD definition.
>

I don't understand what this example is meant to portray. Formatting codes
would be allowed in tables, but directives would and should not be -
directives must be at the start of a line and isn't logically coherent as
to what it is meant to render, within an HTML or text-art table.

The contents of a table as proposed would be a specific subformat, like a
verbatim block. It need not be subject to paragraph-based directives
itself, and as you note, that would be excessively verbose. I am +1 to the
general shape proposed.

-Dan

On Tue, 16 Nov 2021 10:51:53 +0800
Tom Molesworth via perl5-porters <perl5-porters@perl.org> wrote:

> The following is an observation, rather than a negative vote:
>
> This proposal is inconsistent with the paragraph nature of POD - we end up
> with "rectangular POD", where paragraph formatting directives are embedded
> within visual sections of the markup rather than starting at the beginning
> of each line:
>
> | B<Header 1> | B<Header 2>
> |
> |-------------------------------|-------------------------------------------------|
> |=over 4 |
> |
> | |
> |
> |=item * C<$x> - is a variable |=for html <img src="
> https://perl.org/logo.bmp"> |
> | |
> |
> |=back |
> |
>
> As such, I think it'll be a mess - for authors and toolchain
> implementations alike - and we'll end up wishing we'd stuck with markdown.
>
> A paragraph-based table definition would be tedious to write and hard to
> visualise, but would be more in the spirit of the existing POD definition.

I imagine using command paragraphs inside tables will be forbidden, only
formatting codes will be allowed. I think it's the sanest way out.

On Tue, 16 Nov 2021 at 10:58, Tomasz Konojacki <me@xenu.pl> wrote:

> On Tue, 16 Nov 2021 10:51:53 +0800
> Tom Molesworth via perl5-porters <perl5-porters@perl.org> wrote:
> > A paragraph-based table definition would be tedious to write and hard to
> > visualise, but would be more in the spirit of the existing POD
> definition.
>
> I imagine using command paragraphs inside tables will be forbidden, only
> formatting codes will be allowed. I think it's the sanest way out.
>

Sure - that makes the job slightly harder for the toolchain: instead of a
generic POD parser for each rectangular block, you need a subset. Losing
=item support is a minor annoyance, not having access to =for/=begin is
limiting but liveable, nested tables have their place but "one table" is
better than "no tables at all".

Again, not a negative vote - just an expectation of challenges and
impedance mismatch if we try to embed a visual table layout into a world of
70-character lines, paragraph formatting, and verbose syntax.

> Tables are one of the most frequently requested features for Pod

I’m curious: requested where? I couldn’t find much discussion / requests online for tables in pod. Doesn’t mean it hasn’t happened. I just don’t remember it being asked for.

Would cell contents be auto wrapped, or would every cell be treated as nowrap? If so, how would wide tables be handled when looking a pod in a tty with perldoc? Would they just be rendered wide, and thus the individual lines would wrap?

If I wanted to include a table in pod, I’d probably format an ascii table either by hand, or using one of the modules on CPAN, and put it in a verbatim paragraph. With the format proposed, a lot of the time you’re almost
formatting the table anyway, so what’s the delta benefit of supporting a table capability over that?

Raku’s pod supports tables – how much is this capability used? I tried a quick search, but couldn’t find any documentation using tables (but I’m not familiar with the Raku ecosystem, so I could easily have missed a bunch).


Neil

On Thu, 16 Jun 2022 23:02:39 +0100
Neil Bowers <neilb@neilb.org> wrote:

> I?m curious: requested where? I couldn?t find much discussion / requests online for tables in pod. Doesn?t mean it hasn?t happened. I just don?t remember it being asked for.

I guess indeed there probably wasn't anyone formally asking for it, but:

1. There are at least two Pod extensions that add tables (PseudoPod,
Pod6).

2. Despite being poorly documented and not working in non-HTML renderers,
plenty of distributions use the "=begin HTML" hack for tables (and
images).

3. Competing markup formats often include tables. Vanilla Markdown
doesn't have tables, but there are countless extensions that add them.
That indicates it's a pretty popular feature.

> Would cell contents be auto wrapped, or would every cell be treated as nowrap? If so, how would wide tables be handled when looking a pod in a tty with perldoc? Would they just be rendered wide, and thus the individual lines would wrap?

I think that should be left to the renderer. As I said in the images
thread, each of them has completely different constraints.

> If I wanted to include a table in pod, I?d probably format an ascii table either by hand, or using one of the modules on CPAN, and put it in a verbatim paragraph. With the format proposed, a lot of the time you?re almost
> formatting the table anyway, so what?s the delta benefit of supporting a table capability over that?

The main benefits are:

1. Not looking completely terrible and out of place in non-plain-text
renderers (metacpan, github).

2. The ability to use formatting codes inside tables, including the
proposed image tag.

3. "Real" tables won't have fixed width.

> Raku?s pod supports tables ??how much is this capability used? I tried a quick search, but couldn?t find any documentation using tables (but I?m not familiar with the Raku ecosystem, so I could easily have missed a bunch).

I don't know about Raku, but you can find plenty of tables made with
"=begin HTML" on CPAN. For example:

https://metacpan.org/release/MATY/PMLTQ-Commands-2.0.3/view/lib/PMLTQ/Commands.pm#Commands-and-options

https://metacpan.org/release/KIMOTO/SPVM-0.9608/view/lib/SPVM/Document/Tutorial.pm#Numeric-Types

https://metacpan.org/release/BKB/HTML-Make-Calendar-0.01/view/lib/HTML/Make/Calendar.pm#SYNOPSIS

https://metacpan.org/release/CHAMAS/Apache-ASP-2.63/view/ASP.pm#CacheDB

Not to mention countless ASCII-art tables.

> Despite being poorly documented and not working in non-HTML renderers,
> plenty of distributions use the "=begin HTML" hack for tables

I found 42 distributions, which represents 0.09% of CPAN distributions.
Some of them aren’t using tables to present tabular data (for example Image::CairoSVG uses it to format captions below images: https://metacpan.org/pod/Image::CairoSVG#BUGS)

Roughly a quarter of those I classified as abuses of tables. For example https://metacpan.org/pod/Devel::PDB

But at least half of them are things best presented as tables.

If table formatting were provided in pod, then we’d want a decent percentage of people to change over to using pod tables. If they look at pod tables and decide to stick with HTML tables, then what’s the point?


> Competing markup formats often include tables. Vanilla Markdown
> doesn't have tables, but there are countless extensions that add them.
> That indicates it's a pretty popular feature.

For general document formatting systems, tables are a necessity. I’m a big fan of tables – I maintain one of the CPAN modules for rendering text tables.

I’m concerned that getting tables "right" across all the ways and places pod is rendered is non trivial, and that it won’t be widely used, and will also be abused.


I asked:
>> Would cell contents be auto wrapped, or would every cell be treated as nowrap?
You replied:
> I think that should be left to the renderer.

I disagree. This would leave people unsure as to what they can put in tables, and whether they can expect them to come out right.

If this progresses further, I think we should enumerate the different ways/places that people look at pod, and how tables are expected to work/render in each of those places. E.g. perldoc, metacpan, perldoc.org, man, and ... ?


Neil

On Mon, 20 Jun 2022 19:21:38 +0100
Neil Bowers <neilb@neilb.org> wrote:

> If this progresses further, I think we should enumerate the different
> ways/places that people look at pod, and how tables are expected to
> work/render in each of those places. E.g. perldoc, metacpan,
> perldoc.org, man, and ... ?

Hoboy, manpages are fun.

Most POD features are fairly simple copies of things in nroff (the
backing format behind man). It may not therefore be a surprise to
discover that nroff doesn't natively handle tables, any more than POD
does.

There is however a common preprocessor that takes an extension format
and renders it down to simpler positional instructions used to actually
draw the table; it is called `tbl`:

https://linux.die.net/man/1/tbl

Overall, this verymuch is not a markup style I suggest we consider for
a possible POD idea. But I'll describe it here, to give a brief insight
into the kind of abilities it has.


In summary: it's a straaaange format. You write the table contents
twice - first in a grid of letters that describe how the cells should
be arranged, and then a grid of the actual content, separated by some
character (by default a tab, but you can use some printing one instead).

For example; here's 3 rows of 3 cells, separated by a pipe symbol

.TS
tab(|);
c c c
l l r .
Headings|Go|Here
then|the|contents
of|the|cells
.TE

Clever use of the cell layout letters allows things like vertical or
horizontal spans that cover multiple cells. There's lots of flexibility
here, but the overall markup is verymuch not very readable without
knowing what's going on.

Plus, since the content of each table is just more nroff content string
you can still apply all the expected text-level formatting controls and
whatnot inside there. Infact about the only thing you can't do is nest
a table inside another table.

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

2021-11-16 10:42 Tomasz Konojacki <me@xenu.pl> wrote:

> Tables are one of the most frequently requested features for Pod.
>
> There's some prior art in PseudoPod[1] and Pod6[2], but I dislike them
> both. PseudoPod's table syntax is way too verbose, while Pod6 is too
> complicated and whitespace sensitive.
>
> I came up the following syntax:
>
> =begin table
>
> table specification
>
> =end table
>
> The table specification syntax is based on GitHub Markdown with some
> extensions (most notably rowspan and colspan). Please read its
> documentation first:
>
>
> https://docs.github.com/en/github/writing-on-github/working-with-advanced-formatting/organizing-information-with-tables
>
> Using Pod formatting codes (e.g. B<foo>) is allowed inside tables.
>
> Since this is just an elevator pitch, instead of writing a full
> specification, I will just show a few examples:
>
> ------------------------------------------------------------------------
>
> Basic table. Leading and terminating pipes are optional.
>
> Input:
>
> | foo | bar | baz |
> | oof | rab | zab |
>
> Output:
>
> +-----------------+
> | foo | bar | baz |
> +-----------------+
> | oof | rab | zab |
> +------------------
>
> ------------------------------------------------------------------------
>
> Table with headers. The "header2" column is right-aligned. In GitHub
> headers are required, in this specification they will be optional (as
> you've seen in the previous example).
>
> Input:
>
> | header1 | header2 | header3 |
> | --- | ---: | --- |
> | abc | def | ghi |
>
> Output (it's hard to show in plain text, but the first row will be
> rendered as <thead> in HTML):
>
> +-----------------------------+
> | header1 | header2 | header3 |
> +-----------------------------+
> | abc | def | ghi |
> +-----------------------------+
>
> ------------------------------------------------------------------------
>
> "-" merges with the previous cell horizontally (cellspan); "^",
> vertically (rowspan). They can be escaped with E<> or a backslash.
> GitHub Markdown doesn't have this feature.
>
> Input:
>
> | foo | - | baz |
> | ^ | - | zab |
>
> Output:
>
> +-----------------+
> | foo | baz |
> + +-----+
> | | zab |
> +------------------
>
> ------------------------------------------------------------------------
>
> [1] -
> https://metacpan.org/dist/Pod-PseudoPod/view/lib/Pod/PseudoPod/Tutorial.pod#Tables
> [2] - https://docs.raku.org/language/tables


I'm interested in tables in POD because I use tables in POD in SPVM
documentation.

What I only need is the expression of <th> tag in any position.

=begin table

|H<head>|H<head>|
|item|item|
|item|item|

=end table

=begin table

|H<head>|item|
|H<head>|item|
|H<head>|item|

=end table

I also hope the feature of tables in POD is implemented.

** TL;DR: All: Please test the attached nroff file to see if it
** correctly renders tables on your obscure systems.


On Tue, 16 Nov 2021 02:42:02 +0100
Tomasz Konojacki <me@xenu.pl> wrote:

> Tables are one of the most frequently requested features for Pod.

This thread has stalled a bit, but we keep mentioning it at the PSC
meetings so now's a good opportunity to resurrect it a bit.

The overall feeling is that while we're not *opposed* to the idea of
table support in Pod, it seems there isn't a huge amount of forward
momentum so far. Beyond defining the formatting spec of what it should
look like in the Pod itself, there needs to be support by the actual
processors to output it.

And here's where it may become a problem. Of the various core-shipped
or easily-available processors, there are converters to

HTML
Markdown
Manpages (i.e. nroff)
PDF
Plain text

and probably several others. If we're going to declare tables-in-POD as
a supported feature, it must perform reasonably well in all of these
outputs at least.

Of these, only HTML natively has its own concept of "tables" that can
be used. PDF rendering can presumably put text in a tabular layout and
draw some gridlines. Plaintext can just do its own thing (with Unicode
linedrawing if necessary).

The two problematic ones are Markdown and nroff. Neither of these
formats *technically* support tables, though both have fairly common
extensions to basically add the idea.

Perhaps other people can comment about Markdown, but here I'd like to
draw attention to nroff. Since many people's main interaction with Pod
embedded in Perl modules is via the `man` command, if nroff (i.e. the
rendering engine behind `man`) can't be relied on to cope with tables,
then realistically there isn't much point in us supporting them in Pod.

I don't know how prevalent actual support for tbl macros is across
nroff systems - for this I need everyone else's help.


The attached file contains tbl-extended nroff markup, which hopefully
should render in some text-like form. Please test if it does so; if
successful it should look something like the following:

$ man ./test-tables.1 | cat
TEST TABLES(1ex) TEST TABLES(1ex)

SYNOPSIS
A test of tables in manpages

TABLES
Basic
Basic use of the .TS macro showing column alignment

Left Centre Right
This column This column This column
should be should be should be
left-aligned centred right-aligned

Boxed
This table should have a box around the outside and also
between all the cells

????????
?a ? b ?
????????
?c ? d ?
????????
?e ? f ?
????????
Formatted
Cell contents can be formatted; either by default applied to the
whole cell, or by using the usual font selection escapes inside
cells.

Headings Here
^-- enboldened headings --^
Cells can also contain formatted text

AUTHOR
Paul "LeoNerd" Evans


Please everyone, go test that on a variety of obscure systems and report
back what does/doesn't work. If this all works fine on a variety of
different systems then maybe we have a chance at moving this forward.
If most nroff systems (as used by `man`) don't handle tables, then
realistically I don't see a point in us adopting it.

Thanks,

--
Paul "LeoNerd" Evans

leonerd@leonerd.org.uk | https://metacpan.org/author/PEVANS
http://www.leonerd.org.uk/ | https://www.tindie.com/stores/leonerd/

Hi there,

On Fri, 15 Jul 2022, Paul "LeoNerd" Evans wrote:

> ** TL;DR: All: Please test the attached nroff file to see if it
> ** correctly renders tables on your obscure systems.

We don't have any obscure systems here. It looks pretty much OK.

The only comment I'd make is that the boxes aren't all the same size,
those on the left are significantly narrower than those on the right.
It's ugly but I guess it isn't important.

If I insert a space before 'a', 'c' and 'e' in the source, the boxes
are then all rendered the same width which is much more pleasing.

If I add a third column, the boxes in the third column are rendered
the same width as those in the second (with or without a space added
before 'a', 'c' and 'e').

$ nroff -v
GNU nroff (groff) version 1.22.4

--

73,
Ged.

On 7/15/22 10:13, Paul "LeoNerd" Evans wrote:
> I don't know how prevalent actual support for tbl macros is across
> nroff systems - for this I need everyone else's help.

The nroff manual dates to 1976; tbl 11 months later. These both came
along when ed was the editing tool on Unix. And personally I was using
both soon after. It was and still is a safe assumption that all nroff
systems always support tbl. Things like grap came along later, but tbl
(and eqn) were available near the beginning

On Fri, 15 Jul 2022 17:13:25 +0100
"Paul \"LeoNerd\" Evans" <leonerd@leonerd.org.uk> wrote:

> This thread has stalled a bit, but we keep mentioning it at the PSC
> meetings so now's a good opportunity to resurrect it a bit.

To be honest, I got burned out on this because my feeling was that PSC
isn't interested in this feature, as evidenced by bizarre arguments
against it such as "only 0.09% CPAN distributions are using a feature
that isn't officially supported, doesn't fully work and isn't even
really documented".

> The overall feeling is that while we're not *opposed* to the idea of
> table support in Pod, it seems there isn't a huge amount of forward
> momentum so far. Beyond defining the formatting spec of what it should
> look like in the Pod itself, there needs to be support by the actual
> processors to output it.
>
> And here's where it may become a problem. Of the various core-shipped
> or easily-available processors, there are converters to
>
> HTML
> Markdown
> Manpages (i.e. nroff)
> PDF
> Plain text
>
> and probably several others. If we're going to declare tables-in-POD as
> a supported feature, it must perform reasonably well in all of these
> outputs at least.
>
> Of these, only HTML natively has its own concept of "tables" that can
> be used. PDF rendering can presumably put text in a tabular layout and
> draw some gridlines. Plaintext can just do its own thing (with Unicode
> linedrawing if necessary).

The only Pod to PDF processor I'm aware of is Pod::Pdf, which had the
last release in 2000 and I doubt it's being widely used.

> The two problematic ones are Markdown and nroff. Neither of these
> formats *technically* support tables, though both have fairly common
> extensions to basically add the idea.
>
> Perhaps other people can comment about Markdown,

There isn't one Markdown, so Pod to Markdown translator would need an
option to specify the desired flavour. I image most people are
interested in GitHub Markdown. Also, falling back to plain-text tables
is always an option.

> but here I'd like to
> draw attention to nroff. Since many people's main interaction with Pod
> embedded in Perl modules is via the `man` command, if nroff (i.e. the
> rendering engine behind `man`) can't be relied on to cope with tables,
> then realistically there isn't much point in us supporting them in Pod.
>
> I don't know how prevalent actual support for tbl macros is across
> nroff systems - for this I need everyone else's help.

The two most popular manpage renderers are groff (usually on Linux) and
mandoc (BSDs). groff always comes with the tbl preprocessor and
automatically invokes it when needed, whereas in mandoc support for tbl
directives is built-in. That pretty much covers all the modern
unix-likes.

On most legacy unixes it will be probably necessary to manually
preprocess the output with tbl. For example, on an old version of
Solaris 10:

$ tbl man1/a.1 > man1/b.1
$ man -M . b
Reformatting page. Please Wait... done

Misc. Reference Manual Pages TEST TABLES(1ex)

SYNOPSIS
A test of tables in manpages

TABLES
Basic
Basic use of the .TS macro showing column alignment
Left Centre Right
This column This column This column
should be should be should be
left-aligned centred right-aligned

Boxed
This table should have a box around the outside and also
between all the cells
_______
| a| b|
|__|___|
| c| d|
|__|___|
| e| f|
|__|___|

Formatted
Cell contents can be formatted; either by default applied to
the whole cell, or by using the usual font selection escapes
inside cells.
Headings Here
^-- enboldened headings --^
Cells can also contain formatted text

AUTHOR
Paul "LeoNerd" Evans

SunOS 5.10 Last change: 1

Support for the feature.  If it were available I would take advantage of
it, especially when describing multiple simple options or arguments to a
function or method.

Also:

On 7/15/22 15:47, Tomasz Konojacki wrote:
> The only Pod to PDF processor I'm aware of is Pod::Pdf, which had the
> last release in 2000 and I doubt it's being widely used.

There's also

https://metacpan.org/dist/pod2pdf

from 2007-10.  I use that occasionally; paper is still easier to read
than a screen sometimes.

— \\/