Mailing List Archive

On Thu, Dec 24, 2020 at 04:30:47PM +1300, Stephen Worthington wrote:
> # MariaDB seems to have a systemd related bug and tells systemd that
> it is
> # ready to accept connections before it really is. If you have this
> problem
> # try uncommenting the following lines:
> # Restart=on-failure
> # StartLimitBurst=5
> # StartLimitInterval=1

Yeah, that's what Geoff had pointed out..

> So the above settings are not a good workaround at all. It would be
> better to use something like this:
>
> Restart=on-failure
> RestartSec=5
> StartLimitBurst=5
> StartLimitInterval=30

I had at the time installed an override file with Restart=always and
RestartSec=3 in it. As you pointed out, that might flood things if
mythbackend started failing rapidly, but it seemed like a "good enough"
fix. But, systemd fires up mythbackend once and then leaves it in a
failed state.

If mythbackend dies for some other reason, I would really want it to be
restarted. Like the old-school inittab. And systemd's configuration
clearly implies that it is intended to do this, but I was unable to find
any clue from systemd as to why it was ignoring the Restart setting.
If it was something that could be broken apart into pieces, I could
easily instrument pieces, or run strace on it, or something to get an
idea of why different paths are taken, but systemd is too monolithic
and impenetrable to yeild to such techniques. So I gave up.

marcus hall
marcsu@tuells.org
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 24/12/2020 04:06, marcus hall wrote:
>
> I had at the time installed an override file with Restart=always and
> RestartSec=3 in it. As you pointed out, that might flood things if
> mythbackend started failing rapidly, but it seemed like a "good enough"
> fix. But, systemd fires up mythbackend once and then leaves it in a
> failed state.
>
That may appear to be the case but not always. Sometimes it shows you the "failed state" even though
a subsequent attempt has actually got $PROGRAM running properly. Sometimes the journal has rolled
over between attempts and what you are seeing is the last entry of the previous journal.

Ask me how I know...

I always do a "ps -ef" to see what is /actually/ running in these circumstances. systemd fights
tooth and nail to avoid showing you any meaningful information.

--

Mike Perkins

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Sat, 19 Dec 2020 at 21:34, marcus hall <marcus@equinox.tuells.org> wrote:

>
>
> Case in point is my systemd unit to start mythbackend. It appparently
> fails.
> That's about all I have been able to get out of it. But after the system
> boots, systemctl restart mythbackend gets it started just fine.
>
>
>
Me and at least one other person have run into an issue recently where the
backend starts but comes to a halt after a few seconds.

The backend is apparently still running (systemctrl reports it as active)
but there's nothing being written to the logfile (even at debug level) and
it doesn't accept connections.

The halt coincides with the backend's first connection to the capture cards
and my uneducated guess it that the backend is waiting forever for some
response from one of my cards.

I've worked around the issue by changing some of the card parameters on the
Capture Card Setup page and all appears OK now.

I don't know what your issue is but the symptoms fit the problem I was
having, which were not systemd related because as far it was concerned
mythbackend had started and was running.

It's always worth looking at the backend log.

On 2020-12-23 10:30 p.m., Stephen Worthington wrote:
> On Wed, 23 Dec 2020 17:10:43 -0700, you wrote:
>
>> On Sun, Dec 20, 2020 at 02:39:17PM +1300, Stephen Worthington wrote:
>>> How about giving us a bit more detail about the problem so we can
>>> help? The output into mythbackend.log and the output of "journalctl
>>> -eu mythbackend" would be a start. And "systemctl cat mythbackend",
>>> so we can see exactly what the unit files are.
>>
>> I've pretty much given up trying to get systemd to work. This was a year or
>> two ago and my recollection matches what Geoff had posted that mythbackend
>> wasn't starting because mariadb wasn't really ready. That's your main
>> guess as well. I'm not particularly concerned about that, but rather
>> why systemd doesn't try to run mythbackend again.

Because it was not told to try again (Restart=Always) or it tried to,
the number of times it was told to try (all in a very short time,
usually), and those attempts also failed.

<snip>
> So the above settings are not a good workaround at all. It would be
> better to use something like this:
>
> Restart=on-failure
> RestartSec=5
> StartLimitBurst=5
> StartLimitInterval=30
>
> What that does is to change the RestartSec from its default of 100 ms,
> so that restarts are done only very 5 seconds. Then it increases the
> StartLimitInterval to account for that change which means that 5
> restarts done at 5 s intervals takes 25 seconds, then mythbackend
> needs time to start.
>
> However, as with all race conditions, it is never a good idea to use
> delays to fix them. The proper fix would be to get
> mythbackend.service to actually wait for networking to be up before it
> allows mythbackend to be started. One way to do that is to use my fix
> that works on Ubuntu for this when people are using network tuners or
> external frontends. If you are not set up to use network tuners or
> external frontends, you would just set the ping target to be localhost
> (127.0.0.1). I am going to try this out as soon as I can work out how
> to get the mythtv packages to install correctly.


The [Unit] section of my mythbackend.service file has this:
[Unit]
Description=MythTV backend service
After=network-online.target mariadb.service mysqld.service time-sync.target
# Uncomment the following line if you will be using the mythweb plugin
on the
# same system as mythbackend.
Wants=httpd.service

That set of 'After' requirements will exactly fix the startup problem
which it appears you are experiencing. Stephen is correct about Restart,
but that does not deal with the real problem, which is that your system
is attempting to start mythbackend when the dependencies are not yet
ready. The Restart settings are a parallel workaround, but not
guaranteed to work in all cases (such as an external mariadb box, which
is slow to respond, causing the mysql client on the mythbox to take a
long time... or even a login error between the mysql client box and the
mysql server...)
Geoff





_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Thu, 24 Dec 2020 15:58:17 -0500, you wrote:

>On 2020-12-23 10:30 p.m., Stephen Worthington wrote:
>> On Wed, 23 Dec 2020 17:10:43 -0700, you wrote:
>>
>>> On Sun, Dec 20, 2020 at 02:39:17PM +1300, Stephen Worthington wrote:
>>>> How about giving us a bit more detail about the problem so we can
>>>> help? The output into mythbackend.log and the output of "journalctl
>>>> -eu mythbackend" would be a start. And "systemctl cat mythbackend",
>>>> so we can see exactly what the unit files are.
>>>
>>> I've pretty much given up trying to get systemd to work. This was a year or
>>> two ago and my recollection matches what Geoff had posted that mythbackend
>>> wasn't starting because mariadb wasn't really ready. That's your main
>>> guess as well. I'm not particularly concerned about that, but rather
>>> why systemd doesn't try to run mythbackend again.
>
>Because it was not told to try again (Restart=Always) or it tried to,
>the number of times it was told to try (all in a very short time,
>usually), and those attempts also failed.
>
><snip>
>> So the above settings are not a good workaround at all. It would be
>> better to use something like this:
>>
>> Restart=on-failure
>> RestartSec=5
>> StartLimitBurst=5
>> StartLimitInterval=30
>>
>> What that does is to change the RestartSec from its default of 100 ms,
>> so that restarts are done only very 5 seconds. Then it increases the
>> StartLimitInterval to account for that change which means that 5
>> restarts done at 5 s intervals takes 25 seconds, then mythbackend
>> needs time to start.
>>
>> However, as with all race conditions, it is never a good idea to use
>> delays to fix them. The proper fix would be to get
>> mythbackend.service to actually wait for networking to be up before it
>> allows mythbackend to be started. One way to do that is to use my fix
>> that works on Ubuntu for this when people are using network tuners or
>> external frontends. If you are not set up to use network tuners or
>> external frontends, you would just set the ping target to be localhost
>> (127.0.0.1). I am going to try this out as soon as I can work out how
>> to get the mythtv packages to install correctly.
>
>
>The [Unit] section of my mythbackend.service file has this:
>[Unit]
>Description=MythTV backend service
>After=network-online.target mariadb.service mysqld.service time-sync.target
># Uncomment the following line if you will be using the mythweb plugin
>on the
># same system as mythbackend.
>Wants=httpd.service
>
>That set of 'After' requirements will exactly fix the startup problem
>which it appears you are experiencing. Stephen is correct about Restart,
>but that does not deal with the real problem, which is that your system
>is attempting to start mythbackend when the dependencies are not yet
>ready. The Restart settings are a parallel workaround, but not
>guaranteed to work in all cases (such as an external mariadb box, which
>is slow to respond, causing the mysql client on the mythbox to take a
>long time... or even a login error between the mysql client box and the
>mysql server...)
>Geoff

My guess would be that time-sync.target is what is fixing things. That
means that networking has been up long enough for the ntp daemon to
exchange packets with at least one other time service. Which means
that external networking is up and actually working. The external
time service could be another box on the local network, but unless you
have set the ntp config for that, it will likely be Internet time
services.

In my Fedora 33 VM, I have mythbackend starting properly using an IPTV
tuner. Note that mythbackend does not actually test IPTV tuners
during startup, so mythbackend probably does not need networking to be
fully up (external IP addresses available) in order to start with this
config. Mythbackend does not fail and restart at all, and I have not
changed its systemd config. The /etc/mythtv/config.xml file is
currently:

<Configuration>
<UPnP>
<UDN>
<MediaRenderer></MediaRenderer>
</UDN>
</UPnP>
<Database>
<UserName>mythtv</UserName>
<PingHost>0</PingHost>
<Host>localhost</Host>
<DatabaseName>mythconverg</DatabaseName>
<Password>********</Password>
<Port>3306</Port>
</Database>
<WakeOnLan>
<Enabled>0</Enabled>
<SQLReconnectWaitTime>0</SQLReconnectWaitTime>
<SQLConnectRetry>5</SQLConnectRetry>
<Command>echo 'WOLsqlServerCommand not set'</Command>
</WakeOnLan>
</Configuration>

I had to create that manually by copying and exiting the config.xml
from my main Ubuntu MythTV box as the MythTV packages and mythtv-setup
failed to create it. Looking at the config.xml options, setting the
PingHost option might also be a good idea as it means that mythbackend
will try to ping the Host address before going any further trying to
connect to the database.

I still have to try with using the external IP address of the VM as
the database connection - that is where I think I will get problems.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 2020-12-24 10:45 p.m., Stephen Worthington wrote:
> On Thu, 24 Dec 2020 15:58:17 -0500, you wrote:
>
>> On 2020-12-23 10:30 p.m., Stephen Worthington wrote:
>>> On Wed, 23 Dec 2020 17:10:43 -0700, you wrote:
>>>
>>>> On Sun, Dec 20, 2020 at 02:39:17PM +1300, Stephen Worthington wrote:
>>>>> How about giving us a bit more detail about the problem so we can
>>>>> help? The output into mythbackend.log and the output of "journalctl
>>>>> -eu mythbackend" would be a start. And "systemctl cat mythbackend",
>>>>> so we can see exactly what the unit files are.
>>>>
>>>> I've pretty much given up trying to get systemd to work. This was a year or
>>>> two ago and my recollection matches what Geoff had posted that mythbackend
>>>> wasn't starting because mariadb wasn't really ready. That's your main
>>>> guess as well. I'm not particularly concerned about that, but rather
>>>> why systemd doesn't try to run mythbackend again.
>>
>> Because it was not told to try again (Restart=Always) or it tried to,
>> the number of times it was told to try (all in a very short time,
>> usually), and those attempts also failed.
>>
>> <snip>
>>> So the above settings are not a good workaround at all. It would be
>>> better to use something like this:
>>>
>>> Restart=on-failure
>>> RestartSec=5
>>> StartLimitBurst=5
>>> StartLimitInterval=30
>>>
>>> What that does is to change the RestartSec from its default of 100 ms,
>>> so that restarts are done only very 5 seconds. Then it increases the
>>> StartLimitInterval to account for that change which means that 5
>>> restarts done at 5 s intervals takes 25 seconds, then mythbackend
>>> needs time to start.
>>>
>>> However, as with all race conditions, it is never a good idea to use
>>> delays to fix them. The proper fix would be to get
>>> mythbackend.service to actually wait for networking to be up before it
>>> allows mythbackend to be started. One way to do that is to use my fix
>>> that works on Ubuntu for this when people are using network tuners or
>>> external frontends. If you are not set up to use network tuners or
>>> external frontends, you would just set the ping target to be localhost
>>> (127.0.0.1). I am going to try this out as soon as I can work out how
>>> to get the mythtv packages to install correctly.
>>
>>
>> The [Unit] section of my mythbackend.service file has this:
>> [Unit]
>> Description=MythTV backend service
>> After=network-online.target mariadb.service mysqld.service time-sync.target
>> # Uncomment the following line if you will be using the mythweb plugin
>> on the
>> # same system as mythbackend.
>> Wants=httpd.service
>>
>> That set of 'After' requirements will exactly fix the startup problem
>> which it appears you are experiencing. Stephen is correct about Restart,
>> but that does not deal with the real problem, which is that your system
>> is attempting to start mythbackend when the dependencies are not yet
>> ready. The Restart settings are a parallel workaround, but not
>> guaranteed to work in all cases (such as an external mariadb box, which
>> is slow to respond, causing the mysql client on the mythbox to take a
>> long time... or even a login error between the mysql client box and the
>> mysql server...)
>> Geoff
>
> My guess would be that time-sync.target is what is fixing things. That
> means that networking has been up long enough for the ntp daemon to
> exchange packets with at least one other time service. Which means
> that external networking is up and actually working. The external
> time service could be another box on the local network, but unless you
> have set the ntp config for that, it will likely be Internet time
> services.
>
> In my Fedora 33 VM, I have mythbackend starting properly using an IPTV
> tuner. Note that mythbackend does not actually test IPTV tuners
> during startup, so mythbackend probably does not need networking to be
> fully up (external IP addresses available) in order to start with this
> config. Mythbackend does not fail and restart at all, and I have not
> changed its systemd config. The /etc/mythtv/config.xml file is
> currently:
>
> <Configuration>
> <UPnP>
> <UDN>
> <MediaRenderer></MediaRenderer>
> </UDN>
> </UPnP>
> <Database>
> <UserName>mythtv</UserName>
> <PingHost>0</PingHost>
> <Host>localhost</Host>
> <DatabaseName>mythconverg</DatabaseName>
> <Password>********</Password>
> <Port>3306</Port>
> </Database>
> <WakeOnLan>
> <Enabled>0</Enabled>
> <SQLReconnectWaitTime>0</SQLReconnectWaitTime>
> <SQLConnectRetry>5</SQLConnectRetry>
> <Command>echo 'WOLsqlServerCommand not set'</Command>
> </WakeOnLan>
> </Configuration>
>
> I had to create that manually by copying and exiting the config.xml
> from my main Ubuntu MythTV box as the MythTV packages and mythtv-setup
> failed to create it. Looking at the config.xml options, setting the
> PingHost option might also be a good idea as it means that mythbackend
> will try to ping the Host address before going any further trying to
> connect to the database.
>
> I still have to try with using the external IP address of the VM as
> the database connection - that is where I think I will get problems.

I may have missed it, but I don't think the OP ever stated whether this
was a BE/FE box, or an external FE, trying to talk to the BE. It makes a
difference in the config.xml...

Firstly, my config.xml file is basically similar to that above. I did a
thorough search to delete all copies except the one in /etc/mythtv/ so
as to avoid problems.
My config.xml differs in minor respects as it is set up to allow outside
access to the database:
<Host> is the IP address of the backend and mariadb server box.
Localhost is only useful on a BE/FE box, but NOT if you want access from
any other FE. The other items in that stanza are the login details for
the user from the particular FE, INTO the mariadb server on the <Host> box.
Of course, if that is not set, then the external FE will never connect.
Remember that although you may have a user mythtv on each box (or even
your phone!) it is a different user for the mariadb access. Each
user/computer combo has to be authorized on the mariadb server.

( OT: Gosh I've been at this for a while. I knew I posted an answer
about this "some time ago". Turns out it was 14 years ago! So I won't
repeat myself: Check it out here:

https://lists.archive.carbon60.com/mythtv/users/214422?search_string=mandamus.org%20mysql;#214422

There is a newer thread (2016) focusing on getting mysql running on a
step by step basis, and the backend installed and configured, here:

https://lists.archive.carbon60.com/mythtv/users/601218?search_string=mandamus.org%20mysql;#601218

Both are good for troubleshooting as I tried to do it step by step.
)

Since your external FE might find the backend has gone to sleep,
WakeInLAN should be enabled, and <Command> should be <Command>systemctl
restart mariadb.service</Command>
I have wait time as 2 seconds and retry is 5 to allow the mariadb
service to start.


Geoff
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Thu, 24 Dec 2020 23:44:41 -0500, you wrote:

>On 2020-12-24 10:45 p.m., Stephen Worthington wrote:
>> On Thu, 24 Dec 2020 15:58:17 -0500, you wrote:
>>
>>> On 2020-12-23 10:30 p.m., Stephen Worthington wrote:
>>>> On Wed, 23 Dec 2020 17:10:43 -0700, you wrote:
>>>>
>>>>> On Sun, Dec 20, 2020 at 02:39:17PM +1300, Stephen Worthington wrote:
>>>>>> How about giving us a bit more detail about the problem so we can
>>>>>> help? The output into mythbackend.log and the output of "journalctl
>>>>>> -eu mythbackend" would be a start. And "systemctl cat mythbackend",
>>>>>> so we can see exactly what the unit files are.
>>>>>
>>>>> I've pretty much given up trying to get systemd to work. This was a year or
>>>>> two ago and my recollection matches what Geoff had posted that mythbackend
>>>>> wasn't starting because mariadb wasn't really ready. That's your main
>>>>> guess as well. I'm not particularly concerned about that, but rather
>>>>> why systemd doesn't try to run mythbackend again.
>>>
>>> Because it was not told to try again (Restart=Always) or it tried to,
>>> the number of times it was told to try (all in a very short time,
>>> usually), and those attempts also failed.
>>>
>>> <snip>
>>>> So the above settings are not a good workaround at all. It would be
>>>> better to use something like this:
>>>>
>>>> Restart=on-failure
>>>> RestartSec=5
>>>> StartLimitBurst=5
>>>> StartLimitInterval=30
>>>>
>>>> What that does is to change the RestartSec from its default of 100 ms,
>>>> so that restarts are done only very 5 seconds. Then it increases the
>>>> StartLimitInterval to account for that change which means that 5
>>>> restarts done at 5 s intervals takes 25 seconds, then mythbackend
>>>> needs time to start.
>>>>
>>>> However, as with all race conditions, it is never a good idea to use
>>>> delays to fix them. The proper fix would be to get
>>>> mythbackend.service to actually wait for networking to be up before it
>>>> allows mythbackend to be started. One way to do that is to use my fix
>>>> that works on Ubuntu for this when people are using network tuners or
>>>> external frontends. If you are not set up to use network tuners or
>>>> external frontends, you would just set the ping target to be localhost
>>>> (127.0.0.1). I am going to try this out as soon as I can work out how
>>>> to get the mythtv packages to install correctly.
>>>
>>>
>>> The [Unit] section of my mythbackend.service file has this:
>>> [Unit]
>>> Description=MythTV backend service
>>> After=network-online.target mariadb.service mysqld.service time-sync.target
>>> # Uncomment the following line if you will be using the mythweb plugin
>>> on the
>>> # same system as mythbackend.
>>> Wants=httpd.service
>>>
>>> That set of 'After' requirements will exactly fix the startup problem
>>> which it appears you are experiencing. Stephen is correct about Restart,
>>> but that does not deal with the real problem, which is that your system
>>> is attempting to start mythbackend when the dependencies are not yet
>>> ready. The Restart settings are a parallel workaround, but not
>>> guaranteed to work in all cases (such as an external mariadb box, which
>>> is slow to respond, causing the mysql client on the mythbox to take a
>>> long time... or even a login error between the mysql client box and the
>>> mysql server...)
>>> Geoff
>>
>> My guess would be that time-sync.target is what is fixing things. That
>> means that networking has been up long enough for the ntp daemon to
>> exchange packets with at least one other time service. Which means
>> that external networking is up and actually working. The external
>> time service could be another box on the local network, but unless you
>> have set the ntp config for that, it will likely be Internet time
>> services.
>>
>> In my Fedora 33 VM, I have mythbackend starting properly using an IPTV
>> tuner. Note that mythbackend does not actually test IPTV tuners
>> during startup, so mythbackend probably does not need networking to be
>> fully up (external IP addresses available) in order to start with this
>> config. Mythbackend does not fail and restart at all, and I have not
>> changed its systemd config. The /etc/mythtv/config.xml file is
>> currently:
>>
>> <Configuration>
>> <UPnP>
>> <UDN>
>> <MediaRenderer></MediaRenderer>
>> </UDN>
>> </UPnP>
>> <Database>
>> <UserName>mythtv</UserName>
>> <PingHost>0</PingHost>
>> <Host>localhost</Host>
>> <DatabaseName>mythconverg</DatabaseName>
>> <Password>********</Password>
>> <Port>3306</Port>
>> </Database>
>> <WakeOnLan>
>> <Enabled>0</Enabled>
>> <SQLReconnectWaitTime>0</SQLReconnectWaitTime>
>> <SQLConnectRetry>5</SQLConnectRetry>
>> <Command>echo 'WOLsqlServerCommand not set'</Command>
>> </WakeOnLan>
>> </Configuration>
>>
>> I had to create that manually by copying and exiting the config.xml
>> from my main Ubuntu MythTV box as the MythTV packages and mythtv-setup
>> failed to create it. Looking at the config.xml options, setting the
>> PingHost option might also be a good idea as it means that mythbackend
>> will try to ping the Host address before going any further trying to
>> connect to the database.
>>
>> I still have to try with using the external IP address of the VM as
>> the database connection - that is where I think I will get problems.
>
>I may have missed it, but I don't think the OP ever stated whether this
>was a BE/FE box, or an external FE, trying to talk to the BE. It makes a
>difference in the config.xml...
>
>Firstly, my config.xml file is basically similar to that above. I did a
>thorough search to delete all copies except the one in /etc/mythtv/ so
>as to avoid problems.
>My config.xml differs in minor respects as it is set up to allow outside
>access to the database:
><Host> is the IP address of the backend and mariadb server box.
>Localhost is only useful on a BE/FE box, but NOT if you want access from
>any other FE. The other items in that stanza are the login details for
>the user from the particular FE, INTO the mariadb server on the <Host> box.
>Of course, if that is not set, then the external FE will never connect.
>Remember that although you may have a user mythtv on each box (or even
>your phone!) it is a different user for the mariadb access. Each
>user/computer combo has to be authorized on the mariadb server.

You have a misconception here. The Host field in config.xml is only
the address of the database server. The address of the backend is
stored in the database. So even on a backend/frontend box, it makes
sense for the config.xml file to have <Host>localhost</Host>, so that
mythbackend will connect to the database using the much faster
localhost connection. And, in fact, if you specify localhost,
mythbackend has special code that checks for that and then has it
connect via the even faster Unix socket connection to the database.

If you want to have external frontends, the address for the backend in
the database needs to be set to the external IP address of the box.
And the config.xml files on other devices running frontends need to
have the external IP address of the database box (which does not need
to be on the same box as the backend). And the database needs to be
told to bind to the external IP address of its box as well as to
localhost (bind-address=:: or bind-address=0.0.0.0).

>
>( OT: Gosh I've been at this for a while. I knew I posted an answer
>about this "some time ago". Turns out it was 14 years ago! So I won't
>repeat myself: Check it out here:
>
>https://lists.archive.carbon60.com/mythtv/users/214422?search_string=mandamus.org%20mysql;#214422
>
>There is a newer thread (2016) focusing on getting mysql running on a
>step by step basis, and the backend installed and configured, here:
>
>https://lists.archive.carbon60.com/mythtv/users/601218?search_string=mandamus.org%20mysql;#601218
>
>Both are good for troubleshooting as I tried to do it step by step.
>)
>
>Since your external FE might find the backend has gone to sleep,
>WakeInLAN should be enabled, and <Command> should be <Command>systemctl
>restart mariadb.service</Command>
>I have wait time as 2 seconds and retry is 5 to allow the mariadb
>service to start.
>
>
>Geoff

I do not like the look of "restart mariadb.service". If the backend
is up and recording, that would likely corrupt the database for that
recording. Or if it was running mythfilldatabase, the EPG would get
corrupted. Mariadb should be waking up properly automatically if the
backend box went to sleep. If that is not happening, then that is
what needs to be fixed. Or maybe using "start" instead of "restart"
would work - that will only affect mariadb if it is not already
running. Using "restart" here would only be safe if the frontend
attempted to connect before doing the WOL part and never did that
command if the database connected Ok. I have never used WOL, so I am
not sure which way it works.

The "restart" command really should be changed to "systemctl restart
mariadb.service" these days also. The "restart" command is mapped to
"systemctl restart", but that is one extra layer of mapping that is
unnecessary and possible to get broken.
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 2020-12-25 1:47 a.m., Stephen Worthington wrote:
> On Thu, 24 Dec 2020 23:44:41 -0500, you wrote:

>>>
>>> My guess would be that time-sync.target is what is fixing things.

I think that you are correct. I did not know about that target, but it
appears to do exactly what is needed.

>> I may have missed it, but I don't think the OP ever stated whether this
>> was a BE/FE box, or an external FE, trying to talk to the BE. It makes a
>> difference in the config.xml...

>> <Host> is the IP address of the backend and mariadb server box.
>> Localhost is only useful on a BE/FE box, but NOT if you want access from
>> any other FE. The other items in that stanza are the login details for
>> the user from the particular FE, INTO the mariadb server on the <Host> box.
>> Of course, if that is not set, then the external FE will never connect.
>> Remember that although you may have a user mythtv on each box (or even
>> your phone!) it is a different user for the mariadb access. Each
>> user/computer combo has to be authorized on the mariadb server.
>
> You have a misconception here. The Host field in config.xml is only
> the address of the database server. The address of the backend is
> stored in the database. So even on a backend/frontend box, it makes
> sense for the config.xml file to have <Host>localhost</Host>, so that
> mythbackend will connect to the database using the much faster
> localhost connection. And, in fact, if you specify localhost,
> mythbackend has special code that checks for that and then has it
> connect via the even faster Unix socket connection to the database.

I hope we have not confused the OP. Not a misconception on my part. I
was talking about setting up an external FE. As you point out, the
database server *can be* on a different box entirely from any BE or FE.
In that case, all boxes are treated as 'external' to the db server and
need the <Host> attribute to point to the db box. Only in the case of a
combined BE/db server can you use (and should use) 'localhost' (so that,
in that particular case, a socket is used instead of networking).


> If you want to have external frontends, the address for the backend in
> the database needs to be set to the external IP address of the box.
> And the config.xml files on other devices running frontends need to
> have the external IP address of the database box (which does not need
> to be on the same box as the backend). And the database needs to be
> told to bind to the external IP address of its box as well as to
> localhost (bind-address=:: or bind-address=0.0.0.0).

Exactly. the bind address setting is in /etc/my.cnf or equivalent.


> I do not like the look of "restart mariadb.service". If the backend
> is up and recording, that would likely corrupt the database for that
> recording. Or if it was running mythfilldatabase, the EPG would get
> corrupted. Mariadb should be waking up properly automatically if the
> backend box went to sleep. If that is not happening, then that is
> what needs to be fixed. Or maybe using "start" instead of "restart"
> would work - that will only affect mariadb if it is not already
> running. Using "restart" here would only be safe if the frontend
> attempted to connect before doing the WOL part and never did that
> command if the database connected Ok. I have never used WOL, so I am
> not sure which way it works.
You are correct: it should be 'systemctl start mariadb.service'. Did
some searching; if the service is running already, 'start' does nothing.
'restart' *stops* the service and starts it again, thus breaking
whatever was running.

Apparently 'restart' is used for any service upon which another service
depends. Mariadb.service needs networking (for the external service
case). Restarting networking will also start mariadb.service. (But this
is not transitive: restarting mariadb.service does nothing to networking
so I was using it 'backwards' by mistake.
WakeOnLAN can be used to bring a box out of "hibernation"/suspend etc,
so that your remote FE can wake the BE/db server. In that case you might
need to start the mariadb.service (and, I guess, in that case, 'restart'
would not make any difference).

As I said, I hope we have not by now, thoroughly confused the OP.
Checking this has refreshed my memory that doing a RTFM about mariadb
*ALWAYS* takes 2 hours.

Merry Christmas to all (even to those who are now enjoying Boxing Day).

Geoff


_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Fri, Dec 25, 2020 at 01:24:40PM -0500, R. G. Newbury wrote:
>
> I hope we have not confused the OP. Not a misconception on my part. I was
> talking about setting up an external FE. As you point out, the database
> server *can be* on a different box entirely from any BE or FE. In that case,
> all boxes are treated as 'external' to the db server and need the <Host>
> attribute to point to the db box. Only in the case of a combined BE/db
> server can you use (and should use) 'localhost' (so that, in that particular
> case, a socket is used instead of networking).

Not confused at all, I just was expressing exasperation at systemd, not really
looking for anything. I did add StartLimitIntervalSec = 0 to my override file
and next time things reboot, perhaps the backend will start up.

It does so happen that the database and one frontend are on the same box, but
nowadays I tend to use other frontends more than it. I am anticipating
retiring that box and building a container for it on my main server (or
maybe just running it native, I don't have any exposure to the outside world
on any mythtv ports).

It does seem like with restarts every 3 seconds, It seems that I could hit
the StartLimit of 5 restarts at 12 seconds, after the default of 10s, but
maybe that is slightly too close, although it seemed to fail startup for
3 of 3 reboots since the problem started happening...

Now, if I could just figure out how to get Plex to re-load the cover.jpg
images for music once I have changed the file... But that's certainly
fodder for a different mailing list (as well as this systemd issue, really.)

marcus hall
marcus@tuells.org
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 2020-12-25 4:21 p.m., marcus hall wrote:
> On Fri, Dec 25, 2020 at 01:24:40PM -0500, R. G. Newbury wrote:
>>
>> I hope we have not confused the OP. Not a misconception on my part. I was
>> talking about setting up an external FE. As you point out, the database
>> server *can be* on a different box entirely from any BE or FE. In that case,
>> all boxes are treated as 'external' to the db server and need the <Host>
>> attribute to point to the db box. Only in the case of a combined BE/db
>> server can you use (and should use) 'localhost' (so that, in that particular
>> case, a socket is used instead of networking).
>
> Not confused at all, I just was expressing exasperation at systemd, not really
> looking for anything. > I did add StartLimitIntervalSec = 0 to my override file
> and next time things reboot, perhaps the backend will start up.
>
> It does so happen that the database and one frontend are on the same box, but
> nowadays I tend to use other frontends more than it.

Ah HAH! You have the db server and an FE on one box. So where is the BE?
If it is on another box, then quite possibly your BE box cannot access
the dbserver and (drum-roll....) the BE will not start.
If the BE is on the same box with the db server (and an FE) there is
still the possibility that the BE 'user' is not properly authenticated
to the mariadb server.

I find it difficult to express with appropriate moderation my
disagreement with the proposition

https://lists.archive.carbon60.com/mythtv/users/214422?search_string=mandamus.org%20mysql;#214422

This link does "baby steps" through ensuring that your computers can
talk to the db server. In going through that test procedure you will
have to start on the db server to check that you can access it properly.
Then run the mc.sql script once for each user+computer combo to set up
the database accesses. Then you can check on the dbserver (as root)
using the mysql database with: 'select host, user, password from user;'
(Yes there is a field named user in a table named user.) That should
list each computer and each user on that computer. If the BE is on a
separate box, it needs that access too.

As you go around and try from each computer with each user/computer
combo you do NOT want to see 'Access denied to 'luser'@'some-computer'
but if you do, you know what needs to be fixed.

Until you are absolutely sure that each mythtv user can access the db
server from its own box, it is useless to try starting the BE.

> I am anticipating
> retiring that box and building a container for it on my main server (or
> maybe just running it native, I don't have any exposure to the outside world
> on any mythtv ports).

> It does seem like with restarts every 3 seconds, It seems that I could hit
> the StartLimit of 5 restarts at 12 seconds, after the default of 10s, but
> maybe that is slightly too close, although it seemed to fail startup for
> 3 of 3 reboots since the problem started happening...

I now strongly suspect that the BE cannot access the db server. Have you
tried starting the BE from a console? You should be able to see the
exact error(s) and that completely removes the systemctl startup script
from consideration.

Geoff

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 2020-12-25 10:59 p.m., R. G. Newbury wrote:

> If the BE is on the same box with the db server (and an FE) there is
> still the possibility that the BE 'user' is not properly authenticated
> to the mariadb server.
>
> I find it difficult to express with appropriate moderation my
> disagreement with the proposition

Well THAT wasn't supposed to be there... the clipboard gremlin strikes
again, pasting something in which did not belong.

Sorrry about that.
Geoff
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Fri, Dec 25, 2020 at 10:59:18PM -0500, R. G. Newbury wrote:
>
> Ah HAH! You have the db server and an FE on one box. So where is the BE? If
> it is on another box, then quite possibly your BE box cannot access the
> dbserver and (drum-roll....) the BE will not start.
> If the BE is on the same box with the db server (and an FE) there is still
> the possibility that the BE 'user' is not properly authenticated to the
> mariadb server.

One box has mariadb, mythbackend, and one instance of frontend. Other
frontends scattered throughout the house.

I do not believe that there is any issue with mythtv. Once the system boots
and I can ssh into it, I run systemctl restart mythbackend and all comes up
just fine. It's just annoying that systemd can't keep trying to start
mythbackend until it succeeds. I've set the StartLimitInterval to 0 now,
so we'll see if that fixes things, whenever the box needs a reboot. That
may not be for some time, which is why I was not too put out to have to
start it manually, since that is only once or twice a year.

marcus hall
marcus@tuells.org
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 26/12/2020 04:04, marcus hall wrote:
> On Fri, Dec 25, 2020 at 10:59:18PM -0500, R. G. Newbury wrote:
>>
>> Ah HAH! You have the db server and an FE on one box. So where is the BE? If
>> it is on another box, then quite possibly your BE box cannot access the
>> dbserver and (drum-roll....) the BE will not start.
>> If the BE is on the same box with the db server (and an FE) there is still
>> the possibility that the BE 'user' is not properly authenticated to the
>> mariadb server.
>
> One box has mariadb, mythbackend, and one instance of frontend. Other
> frontends scattered throughout the house.
>
> I do not believe that there is any issue with mythtv. Once the system boots
> and I can ssh into it, I run systemctl restart mythbackend and all comes up
> just fine. It's just annoying that systemd can't keep trying to start
> mythbackend until it succeeds. I've set the StartLimitInterval to 0 now,
> so we'll see if that fixes things, whenever the box needs a reboot. That
> may not be for some time, which is why I was not too put out to have to
> start it manually, since that is only once or twice a year.
>
You aren't addressing the actual problem. This is not why systemd isn't restarting the myth backend
but why it is failing to start in the first place.

Granted, this can be some set of obscure systemd interactions at startup but until you investigate
logically you will always have the same problem.

I would point out that most of us have no problems starting up the backend/host that contains the
backend. Most difficulties seem to originate at installation time, with database connections being
high on the list, network connections coming close. Check through as described above and see what
you turn up.

--

Mike Perkins

_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Sat, Dec 26, 2020 at 10:40:17AM +0000, Mike Perkins wrote:
> You aren't addressing the actual problem. This is not why systemd isn't
> restarting the myth backend but why it is failing to start in the first
> place.

Well, it's reasonable certain that the reason that mythbackend doesn't start
up is that mariadb isn't up yet. Mariadb tells systemd that it is up before
it really is (according to notes in Fedora's distributed file for mythbackend)
and so systemd tries to start mythtbackend. Since mythbackend can't connect
to mariadb (mariadb isn't quite ready yet), mythbackend fails. If systemd
would just wait the Restart interval and re-try mythbackend, eventually
mariadb would start answering and it would work. Or, if it isn't really
mariadb but something else that I don't have a dependency expressed for,
eventually that would be up and things would start. The old init (before
systemd) would normally do this from inittab entries. It was just annoying
that systemd didn't seem to be willing to do this. With very little
information about why it wasn't.

> Granted, this can be some set of obscure systemd interactions at startup but
> until you investigate logically you will always have the same problem.
>
> I would point out that most of us have no problems starting up the
> backend/host that contains the backend. Most difficulties seem to originate
> at installation time, with database connections being high on the list,
> network connections coming close. Check through as described above and see
> what you turn up.

Well, whenever I can ssh into the box, everything is up and mythbackend starts
up just fine. Everything the backend needs is on the same box, so shouldn't
be any networking issue, mariadb seems the likely culprit. The "right" way
to specify the dependency to systemd doesn't work, and it's just not worth
trying random hacks to poke at things to cover the race condition when it
involves rebooting the system to test it. Especially when the problem only
happens once or twice a year. It's "good enough".

marcus hall
marcus@tuells.org
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On Sat, 26 Dec 2020 06:27:09 -0700, you wrote:

>On Sat, Dec 26, 2020 at 10:40:17AM +0000, Mike Perkins wrote:
>> You aren't addressing the actual problem. This is not why systemd isn't
>> restarting the myth backend but why it is failing to start in the first
>> place.
>
>Well, it's reasonable certain that the reason that mythbackend doesn't start
>up is that mariadb isn't up yet. Mariadb tells systemd that it is up before
>it really is (according to notes in Fedora's distributed file for mythbackend)
>and so systemd tries to start mythtbackend. Since mythbackend can't connect
>to mariadb (mariadb isn't quite ready yet), mythbackend fails. If systemd
>would just wait the Restart interval and re-try mythbackend, eventually
>mariadb would start answering and it would work. Or, if it isn't really
>mariadb but something else that I don't have a dependency expressed for,
>eventually that would be up and things would start. The old init (before
>systemd) would normally do this from inittab entries. It was just annoying
>that systemd didn't seem to be willing to do this. With very little
>information about why it wasn't.

So what did systemd tell you went wrong? What was the output of
"journalctl -u mythbackend" immediately after boot when mythbackend
had not started? It would probably take a reboot to collect that
information now, as the journal file with that information will likely
have long since been rolled over and deleted.

>> Granted, this can be some set of obscure systemd interactions at startup but
>> until you investigate logically you will always have the same problem.
>>
>> I would point out that most of us have no problems starting up the
>> backend/host that contains the backend. Most difficulties seem to originate
>> at installation time, with database connections being high on the list,
>> network connections coming close. Check through as described above and see
>> what you turn up.
>
>Well, whenever I can ssh into the box, everything is up and mythbackend starts
>up just fine. Everything the backend needs is on the same box, so shouldn't
>be any networking issue, mariadb seems the likely culprit. The "right" way
>to specify the dependency to systemd doesn't work, and it's just not worth
>trying random hacks to poke at things to cover the race condition when it
>involves rebooting the system to test it. Especially when the problem only
>happens once or twice a year. It's "good enough".
>
>marcus hall
>marcus@tuells.org

You can easily have networking issues on one box - mythbackend is
known for having one in particular that requires extra systemd config
when using an external IP address instead of localhost. So did you do
any extra systemd config for mythbackend when you changed its config
to use the external IP address? If not, then that is the likely cause
of the problems.

We still do not know exactly what systemd and mythbackend are being
told to do. So could you please post the output of:

systemctl cat mariadb
systemctl cat mythbackend
sudo mysql mythconverg
select * from settings where value='BackendServerAddr' or
value='MasterServerIP';
quit
ifconfig
netstat -anp | grep mysql
_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org

On 2020-12-26 9:53 a.m., Stephen Worthington wrote:
> On Sat, 26 Dec 2020 06:27:09 -0700, you wrote:
>
>> On Sat, Dec 26, 2020 at 10:40:17AM +0000, Mike Perkins wrote:
>>> You aren't addressing the actual problem. This is not why systemd isn't
>>> restarting the myth backend but why it is failing to start in the first
>>> place.
>>
>> Well, it's reasonable certain that the reason that mythbackend doesn't start
>> up is that mariadb isn't up yet.

So we now (finally) know that you have a simple computer setup with a
BE, FE and db server all on the same box.

1) Have you tried:
[Unit]
Description=MythTV backend service
After=network-online.target mariadb.service mysqld.service
time-sync.target

in the mythbackend.service file?
(Note that this lists *both* mariadb.service and mysqld.service. You
should only have one of these two (possibly one and a soft link as an
alias to the one doing the heavy lifting).

Try this version, please.

2) Since you are running on a single box, try ignoring the outside FE's
for a moment, and set the config.xml to use <Host>Localhost</Host>. This
'should' avoid any network problems in access and help pinpoint things.

>> Well, whenever I can ssh into the box, everything is up and mythbackend starts
>> up just fine. Everything the backend needs is on the same box, so shouldn't
>> be any networking issue, mariadb seems the likely culprit. The "right" way
>> to specify the dependency to systemd doesn't work, and it's just not worth
>> trying random hacks to poke at things to cover the race condition when it
>> involves rebooting the system to test it. Especially when the problem only
>> happens once or twice a year. It's "good enough".

3) And now we can see another possible problem point. You can ssh in,
and start mythbackend. So what user are you when you ssh in? I always
use 'root' when ssh'ing into the mythbox, or any of my computers. I
don't think I have another user authenticated on any machine *for ssh*.
If you can start mythbackend *as root* there is a fair possibility that
the problem comes down to your 'mythtv user': either the user does not
have permission to start mythbackend *at all*, OR the 'mythtv user' is
not properly authenticated to *talk to the database*.

So: 4) Set up ssh access for your mythtv user (if the box is headless),
or sit down at the box, boot up (a cold boot for this!), and
5) try to access mysql: $ mysql-u mythtv -pmythtv -D mythconverg
to determine if it is a user access problem to the db, and

6) try to start mythbackend from a console, as the mythtv user. ( I
will admit that when I run into permissions problems, I use the bigger
hammer method and just chmod 777 everthing in sight, to start with. Then
it's down to adding the user to 'wheel' and other smaller corrections.

I have an idea that your problem is in #6. Get that figured out, and you
can then allow access to the db for users from other boxen, and set the
<Host> config on those other boxes to point to the IP of the db server
(the BE/FE/db box config.xml can remain 'localhost'. (If


> You can easily have networking issues on one box - mythbackend is
> known for having one in particular that requires extra systemd config
> when using an external IP address instead of localhost. So did you do
> any extra systemd config for mythbackend when you changed its config
> to use the external IP address? If not, then that is the likely cause
> of the problems.
>
> We still do not know exactly what systemd and mythbackend are being
> told to do. So could you please post the output of:
>
> systemctl cat mariadb
> systemctl cat mythbackend
> sudo mysql mythconverg
> select * from settings where value='BackendServerAddr' or
> value='MasterServerIP';
> quit
> ifconfig
> netstat -anp | grep mysql

I think we need the access question answered before further spelunking
as set out above, are *necessary*. The 'select... MasterServerIP' will
show where myth thinks the db server lives. In addition, we need to see
the output of "ifconfig | grep -h 'inet'" which will tell us what IP
your box is using. It now occurs to me that there could be a mis-match
between those two numbers, which would explain a lot!

Geoff







_______________________________________________
mythtv-users mailing list
mythtv-users@mythtv.org
http://lists.mythtv.org/mailman/listinfo/mythtv-users
http://wiki.mythtv.org/Mailing_List_etiquette
MythTV Forums: https://forum.mythtv.org