Mailing List Archive

Console scrollback is back again!
Hello, Gentoo.

Yes, console soft scrolling is back! That essential feature that was
stripped out of the kernel at around 5.4.x has returned!

Only this time, it's even better! Instead of one scrollback buffer
shared between all tty's, there's now a buffer for each tty.

How to get it working:
(i) Extract the enclosed patch file to your /usr/src/linux-5.10.27-gentoo
directory (or probably any 5.10 kernel's).
(ii) From that directory run $ patch -p0 < diff.20210405.diff.
(iii)(a) If you've already got a working 5.10.27, do a $ make oldconfig.
That should bring up the new configuration items, for which you can
accept the defaults. These two items are a flag to enable the
scrollback, and a buffer size defaulting to 128kB.
(iii)(b) If you haven't yet got a 5.10.27, just configure your kernel in
the usual way. The two new items are under Device Drivers/Graphics
support/Console display driver support.
(iv) Build the kernel.
(v) Put the new kernel into your usual boot manager.
(vi) Reboot and enjoy!

Admittedly, the exercise isn't quite finished - the patched source files
still have my personal change markings in them, to make debugging easier.
But the problems I reported here a few days ago are now solved. One or
two features haven't (yet) been implemented - having a single scroll
buffer shared amongst all tty's isn't there, and there're no kernel
command line parameters to control the feature.

Also, you may wonder about how safe the patch is. All I can say is that
there is nothing malicious in it, and I am known on this list, but of
course if it breaks I won't be to blame.

Bug reports and other comments are welcome, of course.

If anybody would like the corresponding patch which works on 5.4.n, for n
>= 80, that is available, too.

--
Alan Mackenzie (Nuremberg, Germany).
Re: Console scrollback is back again! [ In reply to ]
On 05/04/2021 18:12, Alan Mackenzie wrote:
> Bug reports and other comments are welcome, of course.
>
> If anybody would like the corresponding patch which works on 5.4.n, for n
>> = 80, that is available, too.

Why did it get removed from the kernel?

If you want to get it back in, couldn't you ping Greg KH and get some
help - or even just advice - getting it accepted?

Cheers,
Wol
Re: Console scrollback is back again! [ In reply to ]
Hello, Wol.

On Mon, Apr 05, 2021 at 18:44:15 +0100, antlists wrote:
> On 05/04/2021 18:12, Alan Mackenzie wrote:
> > Bug reports and other comments are welcome, of course.
> >
> > If anybody would like the corresponding patch which works on 5.4.n, for n
> >> = 80, that is available, too.

> Why did it get removed from the kernel?

Allegedly, there were security problems with it. I think the kernel
people had a program which fired random inputs at lots of components,
and noted when things went wrong; and things went wrong in the scroll
back component. And they couldn't find anybody to look into these
problems.

That's the official position. I'm a bit more cynical about it.

> If you want to get it back in, couldn't you ping Greg KH and get some
> help - or even just advice - getting it accepted?

I have no reason to believe all the "security problems" will have been
resolved by my hacking. I took the algorithmic bits from 4.19.97
basically unchanged.

Maybe there's not much enthusiasm for this feature, in which case I will
keep it working for myself. Linux is basically unusable to me without
it. There are probably quite a few patches to restore the scrollback
floating around the web by now: the changes were not particularly
difficult to anybody who knows the code, but getting to know the code
was unusually difficult.

We'll see how people react to it here, first.

> Cheers,
> Wol

--
Alan Mackenzie (Nuremberg, Germany).
Re: Console scrollback is back again! [ In reply to ]
On Mon, Apr 5, 2021 at 6:12 PM Alan Mackenzie <acm@muc.de> wrote:
>
> Hello, Gentoo.
>
> Yes, console soft scrolling is back! That essential feature that was
> stripped out of the kernel at around 5.4.x has returned!
>
> Only this time, it's even better! Instead of one scrollback buffer
> shared between all tty's, there's now a buffer for each tty.
>

I just tested it with kernel 5.11.11. Seems fine.

Jorge Almeida
Re: Console scrollback is back again! [ In reply to ]
Alan Mackenzie:
> Yes, console soft scrolling is back! That essential feature that was
> stripped out of the kernel at around 5.4.x has returned!

It is commit 087b6cb17df5834d395ab72da3f937380470ba15,
between v5.4.65 and 66.
Perhaps also 0d123a8c64fde433a, cf5a7ded53652c3d63d72, and possible
other stuff.

> Only this time, it's even better! Instead of one scrollback buffer
> shared between all tty's, there's now a buffer for each tty.

Great, no more missing data when switching vts.

Btw, wasn't that added in commit aabd31c421ddc730edf6d89c4ed3885e4fca5e30
but turned off by default.

...
> One ord two features haven't (yet) been implemented - having a
> single scroll buffer shared amongst all tty's isn't there,
...

Is there any reason to share the buffer ?

> Bug reports and other comments are welcome, of course.

git log drivers/video/console/vgacon.c etc. can give some ideas
what one is up against.

Regards,
/Karl Hammar
Re: Console scrollback is back again! [ In reply to ]
On Mon, Apr 5, 2021, 13:13 Alan Mackenzie <acm@muc.de> wrote:

> Hello, Wol.
>
> On Mon, Apr 05, 2021 at 18:44:15 +0100, antlists wrote:
> > On 05/04/2021 18:12, Alan Mackenzie wrote:
> > > Bug reports and other comments are welcome, of course.
> > >
> > > If anybody would like the corresponding patch which works on 5.4.n,
> for n
> > >> = 80, that is available, too.
>
> > Why did it get removed from the kernel?
>
> Allegedly, there were security problems with it. I think the kernel
> people had a program which fired random inputs at lots of components,
> and noted when things went wrong; and things went wrong in the scroll
> back component. And they couldn't find anybody to look into these
> problems.
>
> That's the official position. I'm a bit more cynical about it.
>
> > If you want to get it back in, couldn't you ping Greg KH and get some
> > help - or even just advice - getting it accepted?
>
> I have no reason to believe all the "security problems" will have been
> resolved by my hacking. I took the algorithmic bits from 4.19.97
> basically unchanged.
>
> Maybe there's not much enthusiasm for this feature, in which case I will
> keep it working for myself. Linux is basically unusable to me without
> it. There are probably quite a few patches to restore the scrollback
> floating around the web by now: the changes were not particularly
> difficult to anybody who knows the code, but getting to know the code
> was unusually difficult.
>
> We'll see how people react to it here, first.
>
> > Cheers,
> > Wol
>
> --
> Alan Mackenzie (Nuremberg, Germany).


Without looking at the patch itself:

Have you considered something like kmscon as a userland alternative?

I installed it on my laptop, and aside from not working with gpmd or
consolation for mouse support, it's a perfectly functional local tty.
Re: Console scrollback is back again! [ In reply to ]
On Tue, Apr 6, 2021 at 1:12 AM Alan Mackenzie <acm@muc.de> wrote:
>
> Hello, Gentoo.
>
> Yes, console soft scrolling is back! That essential feature that was
> stripped out of the kernel at around 5.4.x has returned!

Thank you. I'll try to make this work in 5.10.28 once it's released.

--
konsolebox
Re: Console scrollback is back again! [ In reply to ]
Hello, Michael.

On Mon, Apr 05, 2021 at 18:08:58 -0500, Michael Jones wrote:

[ .... ]

> Without looking at the patch itself:

> Have you considered something like kmscon as a userland alternative?

To be honest, no. I wasn't really aware of it when I starting working on
the Linux console.

According to Wikipedia, the last release was in 2013, and development
stopped altogether in 2015. The Gentoo package for it is masked.

> I installed it on my laptop, and aside from not working with gpmd or
> consolation for mouse support, it's a perfectly functional local tty.

I suppose it's one of these programs that once working, pretty much
carries on working for ever.

--
Alan Mackenzie (Nuremberg, Germany).
Re: Console scrollback is back again! [ In reply to ]
Hello, Jorge.

On Mon, Apr 05, 2021 at 19:38:57 +0100, Jorge Almeida wrote:
> On Mon, Apr 5, 2021 at 6:12 PM Alan Mackenzie <acm@muc.de> wrote:

> > Hello, Gentoo.

> > Yes, console soft scrolling is back! That essential feature that was
> > stripped out of the kernel at around 5.4.x has returned!

> > Only this time, it's even better! Instead of one scrollback buffer
> > shared between all tty's, there's now a buffer for each tty.


> I just tested it with kernel 5.11.11. Seems fine.

Thanks!

> Jorge Almeida

--
Alan Mackenzie (Nuremberg, Germany).
Re: Console scrollback is back again! [ In reply to ]
On Tue, Apr 6, 2021 at 10:56 AM Alan Mackenzie <acm@muc.de> wrote:
>

>
>
> > I just tested it with kernel 5.11.11. Seems fine.
>
> Thanks!
>
Thank _you_!

It is regrettable that console scrolling is out of the kernel. Even if
one doesn't use the console that much, it is used once in a while, and
those occasions tend to be precisely the ones back scrolling will be
most missed.

Jorge Almeida
Re: Console scrollback is back again! [ In reply to ]
Hello, Karl.

On Mon, Apr 05, 2021 at 21:58:37 +0200, karl@aspodata.se wrote:
> Alan Mackenzie:
> > Yes, console soft scrolling is back! That essential feature that was
> > stripped out of the kernel at around 5.4.x has returned!

> It is commit 087b6cb17df5834d395ab72da3f937380470ba15,
> between v5.4.65 and 66.
> Perhaps also 0d123a8c64fde433a, cf5a7ded53652c3d63d72, and possible
> other stuff.

I will confess that I don't actually have a git repository of Linux on
my machine. I'd have to back it up, somehow. It would be a fair amount
of work to write an efficient back-up procedure - the one I have at the
moment for git repos backs up the entire repository, which would
probably be excessive for Linux.

Currently, I've changed a mere five files, for which I'm getting by with
symbolic links in the kernel tree pointing into my home directory.

> > Only this time, it's even better! Instead of one scrollback buffer
> > shared between all tty's, there's now a buffer for each tty.

> Great, no more missing data when switching vts.

> Btw, wasn't that added in commit aabd31c421ddc730edf6d89c4ed3885e4fca5e30
> but turned off by default.

There was no capability at all for one buffer per tty in framebuffer
consoles. It was there for VGA consoles, and, I think, one or two
others. The configuration page for it was confusing - the option for
"persistent scrollback" was purely for VGAcon, though that wasn't
obvious. I spent time in the past trying to get multiple buffers
working, and was even considering reporting a bug.

> ...
> > One or two features haven't (yet) been implemented - having a
> > single scroll buffer shared amongst all tty's isn't there,
> ...

> Is there any reason to share the buffer ?

Not that I can think of. The only reason might be if somebody wanted a
very large scrollback buffer, but didn't have a lot of RAM. But even
so, the current default takes a total of less than 1 MB of RAM with the
standard 7 tty's. Even increasing the buffer size to 10 MB, that would
only come to 70 MB. Such a size of buffer would really need more
sophisticated handling, with search facilities, some marking of
positions, and so on. That would be a large project.

> > Bug reports and other comments are welcome, of course.

> git log drivers/video/console/vgacon.c etc. can give some ideas
> what one is up against.

Once I find some way to read it. ;-(

Anyhow, the patch I have is working software, and is not going away any
time soon. It means I can carry on using Linux.

Just in case you're interested, the problem I had with no scrolling on
tty1 was due to the way the console was initialised early on in the boot
process. Its structures were initialised in a separate place from the
normal init function, thus bypassing the scrollbuffer's init. When it
came to re-initialising it for scrollbuffer, this partial initialisation
acted as a flag not to carry on with the init. At least it caused me to
learn about printk. ;-)

> Regards,
> /Karl Hammar

--
Alan Mackenzie (Nuremberg, Germany).
Re: Console scrollback is back again! [ In reply to ]
On Monday, April 5, 2021 7:12:07 PM CEST Alan Mackenzie wrote:
> Hello, Gentoo.
>
> Yes, console soft scrolling is back! That essential feature that was
> stripped out of the kernel at around 5.4.x has returned!
>
> Only this time, it's even better! Instead of one scrollback buffer
> shared between all tty's, there's now a buffer for each tty.
>
> How to get it working:
> (i) Extract the enclosed patch file to your /usr/src/linux-5.10.27-gentoo
> directory (or probably any 5.10 kernel's).
> (ii) From that directory run $ patch -p0 < diff.20210405.diff.
> (iii)(a) If you've already got a working 5.10.27, do a $ make oldconfig.
> That should bring up the new configuration items, for which you can
> accept the defaults. These two items are a flag to enable the
> scrollback, and a buffer size defaulting to 128kB.
> (iii)(b) If you haven't yet got a 5.10.27, just configure your kernel in
> the usual way. The two new items are under Device Drivers/Graphics
> support/Console display driver support.
> (iv) Build the kernel.
> (v) Put the new kernel into your usual boot manager.
> (vi) Reboot and enjoy!
>

Thank you!

I will add this to my "/etc/portage/patches/sys-kernel/gentoo-sources"
directory. Right next to the gpl-defang patch.
Re: Console scrollback is back again! [ In reply to ]
On Monday, 5 April 2021 19:13:18 BST Alan Mackenzie wrote:

> We'll see how people react to it here, first.

You're my hero!

--
Regards,
Peter.
Re: Console scrollback is back again! [ In reply to ]
On Tue, 06 Apr 2021 09:14:23 -0400,
Peter Humphrey wrote:
>
> On Monday, 5 April 2021 19:13:18 BST Alan Mackenzie wrote:
>
> > We'll see how people react to it here, first.
>
> You're my hero!

Would this patch work on 5.4.96 and following?

--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?

John Covici wb2una
covici@ccs.covici.com
Re: Console scrollback is back again! [ In reply to ]
Hello, John

On Tue, Apr 06, 2021 at 10:13:39 -0400, John Covici wrote:

> On Tue, 06 Apr 2021 09:14:23 -0400,
> Peter Humphrey wrote:

> > On Monday, 5 April 2021 19:13:18 BST Alan Mackenzie wrote:

> > > We'll see how people react to it here, first.

> > You're my hero!

> Would this patch work on 5.4.96 and following?

No, there is a slight difference (two struct fields moved and renamed)
between 5.4 and 5.10.

For 5.4, please use the attached patch instead. It has been tested on
5.4.80-r1 and 5.4.97.

> --
> Your life is like a penny. You're going to lose it. The question is:
> How do
> you spend it?

> John Covici wb2una
> covici@ccs.covici.com

--
Alan Mackenzie (Nuremberg, Germany).
Re: Console scrollback is back again! [ In reply to ]
On Tue, Apr 6, 2021 at 4:54 AM Alan Mackenzie <acm@muc.de> wrote:

> Hello, Michael.
>
> On Mon, Apr 05, 2021 at 18:08:58 -0500, Michael Jones wrote:
>
> [ .... ]
>
> > Without looking at the patch itself:
>
> > Have you considered something like kmscon as a userland alternative?
>
> To be honest, no. I wasn't really aware of it when I starting working on
> the Linux console.
>
> According to Wikipedia, the last release was in 2013, and development
> stopped altogether in 2015. The Gentoo package for it is masked.
>

Good points, and I can't speak to the code cleanliness of kmscon, or
potential security problems.

That being said, even though I was annoyed when the scrollback was removed
from the kernel, I do understand the reason for it being removed. It does
seem like the kind of functionality that would be better provided by
userspace.


> > I installed it on my laptop, and aside from not working with gpmd or
> > consolation for mouse support, it's a perfectly functional local tty.
>
> I suppose it's one of these programs that once working, pretty much
> carries on working for ever.
>

That's entirely possible :-)
Maybe someone (not necessarily you) will find the removal of the scrollback
buffer to be enough motivation to bring kmscon back to life as a project.
Re: Console scrollback is back again! [ In reply to ]
On Tue, Apr 6, 2021 at 1:12 AM Alan Mackenzie <acm@muc.de> wrote:
> How to get it working:
> (i) Extract the enclosed patch file to your /usr/src/linux-5.10.27-gentoo
> directory (or probably any 5.10 kernel's).

Works with 5.10.28-gentoo. Thanks again.

I finally can uninstall my 5.8.6 which still has the old scrollback.

--
konsolebox
Re: Console scrollback is back again! [ In reply to ]
Hello, Gentoo.

A new kernel, gentoo-sources-5.10.49-r1 has become current.

For those who wish to have console soft scrolling, the patch which worked
with 5.10.27 actually still works in 5.10.49-r1. But I've tidied that
patch up somewhat for 5.10.49-r1 and have attached the patch file to this
post.

The instructions for using it are basically unchanged, apart from file
names, etc. Updating those instructions gives us:
(i) Extract the attached patch file to your
/usr/src/linux-5.10.49-gentoo-r1 directory (or probably any 5.10
kernel's).
(ii) From that directory run $ patch -p0 < 5.10.49-scroll.20210715.diff.
(iii)(a) If you've already got a working 5.10.49-r1, do a $ make oldconfig.
That should bring up the new configuration items, for which you can
accept the defaults. These two items are a flag to enable the
scrollback, and a buffer size defaulting to 128kB.
(iii)(b) If you haven't yet got a 5.10.49-r1, just configure your kernel
in the usual way. The two new items are under Device Drivers/Graphics
support/Console display driver support.
(iv) Build the kernel.
(v) Put the new kernel into your usual boot manager.
(vi) Reboot and enjoy!

As before, it works for me, and there's nothing malicious in it, but if
it breaks for you, I'm sorry but I won't be responsible.

--
Alan Mackenzie (Nuremberg, Germany).



On Mon, Apr 05, 2021 at 17:12:07 +0000, Alan Mackenzie wrote:
> Hello, Gentoo.

> Yes, console soft scrolling is back! That essential feature that was
> stripped out of the kernel at around 5.4.x has returned!

> Only this time, it's even better! Instead of one scrollback buffer
> shared between all tty's, there's now a buffer for each tty.

> How to get it working:
> (i) Extract the enclosed patch file to your /usr/src/linux-5.10.27-gentoo
> directory (or probably any 5.10 kernel's).
> (ii) From that directory run $ patch -p0 < diff.20210405.diff.
> (iii)(a) If you've already got a working 5.10.27, do a $ make oldconfig.
> That should bring up the new configuration items, for which you can
> accept the defaults. These two items are a flag to enable the
> scrollback, and a buffer size defaulting to 128kB.
> (iii)(b) If you haven't yet got a 5.10.27, just configure your kernel in
> the usual way. The two new items are under Device Drivers/Graphics
> support/Console display driver support.
> (iv) Build the kernel.
> (v) Put the new kernel into your usual boot manager.
> (vi) Reboot and enjoy!

> Admittedly, the exercise isn't quite finished - the patched source files
> still have my personal change markings in them, to make debugging easier.
> But the problems I reported here a few days ago are now solved. One or
> two features haven't (yet) been implemented - having a single scroll
> buffer shared amongst all tty's isn't there, and there're no kernel
> command line parameters to control the feature.

> Also, you may wonder about how safe the patch is. All I can say is that
> there is nothing malicious in it, and I am known on this list, but of
> course if it breaks I won't be to blame.

> Bug reports and other comments are welcome, of course.

> If anybody would like the corresponding patch which works on 5.4.n, for n
> >= 80, that is available, too.

> --
> Alan Mackenzie (Nuremberg, Germany).