Mailing List Archive

Re: INSERT INTO recordedmarkup fails w/duplicate; recording status changes [Workaround]
All of the weird behavior I've been seeing seems to have gone away
when I downgraded all mariadb packages from the current stable version
in Debian, 1:10.5.19-0+deb11u1, to the previous stable version,
1:10.5.18-0+deb11u1.* I believe the key difference is in the version
information provided in
https://sources.debian.org/src/mariadb-10.5/1%3A10.5.19-0%2Bdeb11u1/libmariadb/libmariadb/mariadb_lib.c/,
which follows the new scheme used in Maria 10.6, vs
https://sources.debian.org/src/mariadb-10.5/1%3A10.5.18-0%2Bdeb11u1/libmariadb/libmariadb/mariadb_lib.c/,
which follows the old scheme.

See this proposed patch (against 10.3, however) to undo the transition
to the new scheme, and discussion in
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1031863.
Apparently the maria 10.6 scheme reports a major version of 3, while
the old one reports a major version of 10, and that is throwing other
things off. In particular, Qt's SQL library (libqt5sql5 on Debian)
gets thrown by this. There are various proposals to patch Qt floating
around.

This change in versioning seems likely to screw up the conditional SQL
code we discussed a little earlier as well--in fact it makes me wonder
which parts of it got run.

Note that this maria versioning explanation is not necessarily in
conflict with those centering about the handling of a trailing Z in
the datetimestamp; the version problems may be causing the
datetimestamp problems.

This is a hack because downgrading is a hack. Even if it works my
package tools want to upgrade immediately; presumably I could pin them
to that version. And the newer release fixes some bugs and security
holes, as later releases will too.

More fundamentally, MariaDB is moving to the new versioning scheme,
and so other software using it will need to adapt. Note there are
likely 3 different places that adjustments could be made to work
around this problem:
1. myth (though it's not clear to me it can work around problems in Qt).
2. Qt's SQL library.
3. Maria.

Apart from the version info maria reports, its handling of trailing
Z's in datetimes could be more forgiving, particularly in keys. Some
reports suggest later versions of maria can handle times with Z OK,
and possibly earlier versions did as well.

My experience seems to imply that the upgrade to maria 10.5.19, which
happened on 3/11 according to my logs, did not take full effect until
the system restarted on 4/4. Which seems surprising, particularly
since my recent downgrade became effective immediately. For the
recent downgrade I stopped, myth, the SQL sessions I had open, and
maria before downgrading. Maybe I missed some of those steps earlier
(though I thought the packaging system itself stops maria before
changing it). Perhaps myth checks maria version info when it starts,
and I didn't restart myth?

Ross

*I downgraded them all together out of caution and deference to prior
admonitions in another thread to do them all together. It might be
that downgrading libmariadb3 alone, which I believe is where the
version reporting code lives, would suffice. But that's riskier, and
the packaging system might not even support it. BTW, "stable" is an
attribute of the entire distribution, not a particular package.
"stable" is the mainline version of Debian = Debian 11 = bullseye.
Packages in it are supposed to keep a consistent API even if upgraded
for critical fixes. The maria packages obviously have not managed
that.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: INSERT INTO recordedmarkup fails w/duplicate; recording status changes [Workaround] [ In reply to ]
On Saturday 15 April 2023 05:29:11 PM (-05:00), Ross Boylan wrote:

> All of the weird behavior I've been seeing seems to have gone away
> when I downgraded all mariadb packages from the current stable version
> in Debian, 1:10.5.19-0+deb11u1, to the previous stable version,
> 1:10.5.18-0+deb11u1.* I believe the key difference is in the version
> information provided in
>
https://sources.debian.org/src/mariadb-10.5/1%3A10.5.19-0%2Bdeb11u1/libmariadb/libmariadb/mariadb_lib.c/,

> which follows the new scheme used in Maria 10.6, vs
>
https://sources.debian.org/src/mariadb-10.5/1%3A10.5.18-0%2Bdeb11u1/libmariadb/libmariadb/mariadb_lib.c/,

> which follows the old scheme.
>
> See this proposed patch (against 10.3, however) to undo the transition
> to the new scheme, and discussion in
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1031863.
> Apparently the maria 10.6 scheme reports a major version of 3, while
> the old one reports a major version of 10, and that is throwing other
> things off. In particular, Qt's SQL library (libqt5sql5 on Debian)
> gets thrown by this. There are various proposals to patch Qt floating
> around.
>
> This change in versioning seems likely to screw up the conditional SQL
> code we discussed a little earlier as well--in fact it makes me wonder
> which parts of it got run.
>
> Note that this maria versioning explanation is not necessarily in
> conflict with those centering about the handling of a trailing Z in
> the datetimestamp; the version problems may be causing the
> datetimestamp problems.
>
> This is a hack because downgrading is a hack. Even if it works my
> package tools want to upgrade immediately; presumably I could pin them
> to that version. And the newer release fixes some bugs and security
> holes, as later releases will too.
>
> More fundamentally, MariaDB is moving to the new versioning scheme,
> and so other software using it will need to adapt. Note there are
> likely 3 different places that adjustments could be made to work
> around this problem:
> 1. myth (though it's not clear to me it can work around problems in Qt).
> 2. Qt's SQL library.
> 3. Maria.
>
> Apart from the version info maria reports, its handling of trailing
> Z's in datetimes could be more forgiving, particularly in keys. Some
> reports suggest later versions of maria can handle times with Z OK,
> and possibly earlier versions did as well.
>
> My experience seems to imply that the upgrade to maria 10.5.19, which
> happened on 3/11 according to my logs, did not take full effect until
> the system restarted on 4/4. Which seems surprising, particularly
> since my recent downgrade became effective immediately. For the
> recent downgrade I stopped, myth, the SQL sessions I had open, and
> maria before downgrading. Maybe I missed some of those steps earlier
> (though I thought the packaging system itself stops maria before
> changing it). Perhaps myth checks maria version info when it starts,
> and I didn't restart myth?
>
> Ross
>
> *I downgraded them all together out of caution and deference to prior
> admonitions in another thread to do them all together. It might be
> that downgrading libmariadb3 alone, which I believe is where the
> version reporting code lives, would suffice. But that's riskier, and
> the packaging system might not even support it. BTW, "stable" is an
> attribute of the entire distribution, not a particular package.
> "stable" is the mainline version of Debian = Debian 11 = bullseye.
> Packages in it are supposed to keep a consistent API even if upgraded
> for critical fixes. The maria packages obviously have not managed
> that.

Monday a fix will be pushed tomaster that guarantees that the timezone (Z)
is removed from DATETIME values (like starttime). It will soak for a week
and
if all is well will go in v33 and v32. It's been soaking in 2 systems with
no ill
affects. But the code is called in 2600+ places, so better to be safe.
--
Bill
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: INSERT INTO recordedmarkup fails w/duplicate; recording status changes [Workaround] [ In reply to ]
On Sat, 15 Apr 2023 15:29:11 -0700, you wrote:

>My experience seems to imply that the upgrade to maria 10.5.19, which
>happened on 3/11 according to my logs, did not take full effect until
>the system restarted on 4/4. Which seems surprising, particularly
>since my recent downgrade became effective immediately. For the
>recent downgrade I stopped, myth, the SQL sessions I had open, and
>maria before downgrading. Maybe I missed some of those steps earlier
>(though I thought the packaging system itself stops maria before
>changing it). Perhaps myth checks maria version info when it starts,
>and I didn't restart myth?

Linux allows you to update a running file. I think it does that by
pushing all the code in the running file into RAM and then (if
necessary) out into swap space before it overwrites the old file. So
as long as the program using that file keeps running, it keeps using
the older version. So if the MariaDB upgrade happened while MythTV
was still running, MythTV would have kept on using the .so library
files from the older version of MariaDB. This behaviour is very, very
useful, but does lead to a bit of confusion at times. When major
underlying things (such as the video drivers) get updated, the new
ones do not get used until after a reboot. For MariaDB and MythTV,
the systemd setup does not cause MythTV to be shut down before a
MariaDB update can happen. And MythTV is quite resilient - it will
handle the database going away and coming back again without raising
any alarms. If it happens when there are no writes to the database,
there is no damage done. But if it did want to write to the database,
this behaviour is problematic as all the database transactions that
happen when the database was down are lost, potentially leaving the
database logically corrupt, apart from not having those transactions
done.

It is possible to fix the systemd setup so that mythbackend will be
shut down before MariaDB can be shut down. I think this is done by
putting this in the mythbackend .service file or an override file:

[Unit]
PartOf=mysql.service
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org
Re: INSERT INTO recordedmarkup fails w/duplicate; recording status changes [Workaround] [ In reply to ]
On 16/04/2023 04:47, Stephen Worthington wrote:
> Linux allows you to update a running file. I think it does that by
> pushing all the code in the running file into RAM and then (if
> necessary) out into swap space before it overwrites the old file.

No, this is done by making use of Posix-standardised filesystem features
in the mv and rm commands (and syscalls):

First the package manager writes the new file, giving it a temporary
filename. It has to take care to create the new file within the same
filesystem as the one hosting the file that is about to be replaced.

Then the old file is renamed (by the package manager) using the mv
command. When the destination of the mv command is situated on the same
filesystem the command only modifies the directory entry; the file data
itself remains untouched. The package manager then uses the mv command
to rename the new file from its initial temporary filename to the target
filename. Again, this only involves updating the directory entry.

Finally the package manager deletes the old file using rm. But the rm
command merely unlinks the file's data from the directory entry:
existing users of that data continue having access to it because they
don't need the directory entry to continue using the file. For example,
if the file in question is an executable file then the kernel will have
memory-mapped the file (using the mmap system call), as a side-effect of
which the filesystem knows not yet to recycle the disk space the file
occupies.

> So
> as long as the program using that file keeps running, it keeps using
> the older version.

Correct.

HTH, Jan

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org