Mailing List Archive: How to find out all openrc dependencies?

How to find out all openrc dependencies?

Apr 10, 2024, 10:19 PM

Post #1 of 7 (62 views)

Hi all,

For a while I've been seeing the following ERROR-messages when booting 1 of my
systems:

* ERROR: cannot start multipathd as localmount would not start
* ERROR: cannot start zfs-import as localmount would not start

This isn't a big concern as these services will start correctly later:

INIT: Entering runlevel: 3
* Starting multipathd ...
[ ok ]
* Importing ZFS pool(s) ...
[ ok ]

But I am trying to find the cause of these errors as they are preventing
parallel-start from actually working correctly.

When I check with "rc-depend", I don't see an obious cause:

# /lib/rc/bin/rc-depend multipathd
sysfs devfs udev udev-trigger modules fsck root localmount multipathd

# /lib/rc/bin/rc-depend localmount
sysfs devfs udev udev-trigger modules fsck root localmount

# /lib/rc/bin/rc-depend zfs-import
multipath sysfs devfs udev udev-trigger modules fsck root localmount
multipathd zfs-import

# /lib/rc/bin/rc-depend multipath
multipath

From how I read these, it should be able to start "localmount" properly before
even trying to start "multipathd" and "zfs-import"
There is also no technical dependency for "localmount" (the root filesystem is
not on ZFS on this system)

Any help/suggestions on how to find the cause would be appreciated.

--
Joost

Re: How to find out all openrc dependencies? [ In reply to ]

confabulate at kintzios

Apr 11, 2024, 2:35 AM

Post #2 of 7 (60 views)

Permalink

On Thursday, 11 April 2024 06:19:57 BST J. Roeleveld wrote:
> Hi all,
>
> For a while I've been seeing the following ERROR-messages when booting 1 of
> my systems:
>
> * ERROR: cannot start multipathd as localmount would not start
> * ERROR: cannot start zfs-import as localmount would not start
>
> This isn't a big concern as these services will start correctly later:
>
> INIT: Entering runlevel: 3
> * Starting multipathd ...
> [ ok ]
> * Importing ZFS pool(s) ...
> [ ok ]
>
> But I am trying to find the cause of these errors as they are preventing
> parallel-start from actually working correctly.
>
> When I check with "rc-depend", I don't see an obious cause:
>
> # /lib/rc/bin/rc-depend multipathd
> sysfs devfs udev udev-trigger modules fsck root localmount multipathd
>
> # /lib/rc/bin/rc-depend localmount
> sysfs devfs udev udev-trigger modules fsck root localmount
>
> # /lib/rc/bin/rc-depend zfs-import
> multipath sysfs devfs udev udev-trigger modules fsck root localmount
> multipathd zfs-import
>
> # /lib/rc/bin/rc-depend multipath
> multipath
>
> From how I read these, it should be able to start "localmount" properly
> before even trying to start "multipathd" and "zfs-import"
> There is also no technical dependency for "localmount" (the root filesystem
> is not on ZFS on this system)
>
> Any help/suggestions on how to find the cause would be appreciated.
>
> --
> Joost

Check if hwclock is in the boot runlevel:

rc-update -s -v | grep hwclock

Re: How to find out all openrc dependencies? [ In reply to ]

joost at antarean

Apr 11, 2024, 2:48 AM

Post #3 of 7 (60 views)

Permalink

On Thursday, 11 April 2024 11:35:10 CEST Michael wrote:
> On Thursday, 11 April 2024 06:19:57 BST J. Roeleveld wrote:
> > Hi all,
> >
> > For a while I've been seeing the following ERROR-messages when booting 1
> > of
> > my systems:
> >
> > * ERROR: cannot start multipathd as localmount would not start
> >
> > * ERROR: cannot start zfs-import as localmount would not start
> >
> > This isn't a big concern as these services will start correctly later:
> >
> > INIT: Entering runlevel: 3
> >
> > * Starting multipathd ...
> > [ ok ]
> > * Importing ZFS pool(s) ...
> > [ ok ]
> >
> > But I am trying to find the cause of these errors as they are preventing
> > parallel-start from actually working correctly.
> >
> > When I check with "rc-depend", I don't see an obious cause:
> >
> > # /lib/rc/bin/rc-depend multipathd
> > sysfs devfs udev udev-trigger modules fsck root localmount multipathd
> >
> > # /lib/rc/bin/rc-depend localmount
> > sysfs devfs udev udev-trigger modules fsck root localmount
> >
> > # /lib/rc/bin/rc-depend zfs-import
> > multipath sysfs devfs udev udev-trigger modules fsck root localmount
> > multipathd zfs-import
> >
> > # /lib/rc/bin/rc-depend multipath
> > multipath
> >
> > From how I read these, it should be able to start "localmount" properly
> > before even trying to start "multipathd" and "zfs-import"
> > There is also no technical dependency for "localmount" (the root
> > filesystem
> > is not on ZFS on this system)
> >
> > Any help/suggestions on how to find the cause would be appreciated.
> >
> > --
> > Joost
>
> Check if hwclock is in the boot runlevel:
>
> rc-update -s -v | grep hwclock

What does "hwclock" got to do with this?
It has no dependency with multipathd, zfs-import, localmount or anything else
that is showing an error.

--
Joost

Re: How to find out all openrc dependencies? [ In reply to ]

confabulate at kintzios

Apr 11, 2024, 3:10 AM

Post #4 of 7 (60 views)

Permalink

On Thursday, 11 April 2024 10:48:15 BST J. Roeleveld wrote:
> On Thursday, 11 April 2024 11:35:10 CEST Michael wrote:
> > On Thursday, 11 April 2024 06:19:57 BST J. Roeleveld wrote:
> > > Hi all,
> > >
> > > For a while I've been seeing the following ERROR-messages when booting 1
> > > of
> > > my systems:
> > >
> > > * ERROR: cannot start multipathd as localmount would not start
> > >
> > > * ERROR: cannot start zfs-import as localmount would not start
> > >
> > > This isn't a big concern as these services will start correctly later:
> > >
> > > INIT: Entering runlevel: 3
> > >
> > > * Starting multipathd ...
> > > [ ok ]
> > > * Importing ZFS pool(s) ...
> > > [ ok ]
> > >
> > > But I am trying to find the cause of these errors as they are preventing
> > > parallel-start from actually working correctly.
> > >
> > > When I check with "rc-depend", I don't see an obious cause:
> > >
> > > # /lib/rc/bin/rc-depend multipathd
> > > sysfs devfs udev udev-trigger modules fsck root localmount multipathd
> > >
> > > # /lib/rc/bin/rc-depend localmount
> > > sysfs devfs udev udev-trigger modules fsck root localmount
> > >
> > > # /lib/rc/bin/rc-depend zfs-import
> > > multipath sysfs devfs udev udev-trigger modules fsck root localmount
> > > multipathd zfs-import
> > >
> > > # /lib/rc/bin/rc-depend multipath
> > > multipath
> > >
> > > From how I read these, it should be able to start "localmount" properly
> > > before even trying to start "multipathd" and "zfs-import"
> > > There is also no technical dependency for "localmount" (the root
> > > filesystem
> > > is not on ZFS on this system)
> > >
> > > Any help/suggestions on how to find the cause would be appreciated.
> > >
> > > --
> > > Joost
> >
> > Check if hwclock is in the boot runlevel:
> >
> > rc-update -s -v | grep hwclock
>
> What does "hwclock" got to do with this?
> It has no dependency with multipathd, zfs-import, localmount or anything
> else that is showing an error.
>
> --
> Joost

Our systems are certainly different, but I noticed this dependency on my
localmount which is missing on yours:

# /lib/rc/bin/rc-depend localmount
sysfs devfs udev udev-trigger hwclock modules fsck root dmcrypt localmount
^^^^^^^
Have you compared your system services which has this problem, with other
systems of yours which can startup properly?

Re: How to find out all openrc dependencies? [ In reply to ]

joost at antarean

Apr 14, 2024, 12:28 AM

Post #5 of 7 (52 views)

Permalink

On Thursday, 11 April 2024 12:10:31 CEST Michael wrote:
> On Thursday, 11 April 2024 10:48:15 BST J. Roeleveld wrote:
> > On Thursday, 11 April 2024 11:35:10 CEST Michael wrote:
> > > On Thursday, 11 April 2024 06:19:57 BST J. Roeleveld wrote:
> > > > Hi all,
> > > >
> > > > For a while I've been seeing the following ERROR-messages when booting
> > > > 1
> > > > of
> > > > my systems:
> > > >
> > > > * ERROR: cannot start multipathd as localmount would not start
> > > >
> > > > * ERROR: cannot start zfs-import as localmount would not start
> > > >
> > > > This isn't a big concern as these services will start correctly later:
> > > >
> > > > INIT: Entering runlevel: 3
> > > >
> > > > * Starting multipathd ...
> > > > [ ok ]
> > > > * Importing ZFS pool(s) ...
> > > > [ ok ]
> > > >
> > > > But I am trying to find the cause of these errors as they are
> > > > preventing
> > > > parallel-start from actually working correctly.
> > > >
> > > > When I check with "rc-depend", I don't see an obious cause:
> > > >
> > > > # /lib/rc/bin/rc-depend multipathd
> > > > sysfs devfs udev udev-trigger modules fsck root localmount multipathd
> > > >
> > > > # /lib/rc/bin/rc-depend localmount
> > > > sysfs devfs udev udev-trigger modules fsck root localmount
> > > >
> > > > # /lib/rc/bin/rc-depend zfs-import
> > > > multipath sysfs devfs udev udev-trigger modules fsck root localmount
> > > > multipathd zfs-import
> > > >
> > > > # /lib/rc/bin/rc-depend multipath
> > > > multipath
> > > >
> > > > From how I read these, it should be able to start "localmount"
> > > > properly
> > > > before even trying to start "multipathd" and "zfs-import"
> > > > There is also no technical dependency for "localmount" (the root
> > > > filesystem
> > > > is not on ZFS on this system)
> > > >
> > > > Any help/suggestions on how to find the cause would be appreciated.
> > > >
> > > > --
> > > > Joost
> > >
> > > Check if hwclock is in the boot runlevel:
> > >
> > > rc-update -s -v | grep hwclock
> >
> > What does "hwclock" got to do with this?
> > It has no dependency with multipathd, zfs-import, localmount or anything
> > else that is showing an error.
> >
> > --
> > Joost
>
> Our systems are certainly different, but I noticed this dependency on my
> localmount which is missing on yours:
>
> # /lib/rc/bin/rc-depend localmount
> sysfs devfs udev udev-trigger hwclock modules fsck root dmcrypt localmount
> ^^^^^^^
> Have you compared your system services which has this problem, with other
> systems of yours which can startup properly?

Adding additional dependencies into the tree is more likely to cause further
issues. I am actually looking for how to quickly find out which dependency is
causing a circular dependency issue as the first time it thinks it needs to
start a service it fails. But the 2nd time it starts, it goes correctly.

I removed hwclock from ALL VMs as they don't actually have a hwclock.

I did find out the actual cause of the problem through a lot of trial and
error, but this is not really useful in actually quickly finding the problem.
Being able to "simulate" the startup sequence for how OpenRC wants to do
things would have simplified and sped up the entire process.

--
Joost

Re: How to find out all openrc dependencies? [ In reply to ]

confabulate at kintzios

Apr 14, 2024, 1:24 AM

Post #6 of 7 (51 views)

Permalink

On Sunday, 14 April 2024 08:28:07 BST J. Roeleveld wrote:
> On Thursday, 11 April 2024 12:10:31 CEST Michael wrote:
> > On Thursday, 11 April 2024 10:48:15 BST J. Roeleveld wrote:
> > > On Thursday, 11 April 2024 11:35:10 CEST Michael wrote:
> > > > On Thursday, 11 April 2024 06:19:57 BST J. Roeleveld wrote:
> > > > > Hi all,
> > > > >
> > > > > For a while I've been seeing the following ERROR-messages when
> > > > > booting
> > > > > 1
> > > > > of
> > > > > my systems:
> > > > >
> > > > > * ERROR: cannot start multipathd as localmount would not start
> > > > >
> > > > > * ERROR: cannot start zfs-import as localmount would not start
> > > > >
> > > > > This isn't a big concern as these services will start correctly
> > > > > later:
> > > > >
> > > > > INIT: Entering runlevel: 3
> > > > >
> > > > > * Starting multipathd ...
> > > > > [ ok ]
> > > > > * Importing ZFS pool(s) ...
> > > > > [ ok ]
> > > > >
> > > > > But I am trying to find the cause of these errors as they are
> > > > > preventing
> > > > > parallel-start from actually working correctly.
> > > > >
> > > > > When I check with "rc-depend", I don't see an obious cause:
> > > > >
> > > > > # /lib/rc/bin/rc-depend multipathd
> > > > > sysfs devfs udev udev-trigger modules fsck root localmount
> > > > > multipathd
> > > > >
> > > > > # /lib/rc/bin/rc-depend localmount
> > > > > sysfs devfs udev udev-trigger modules fsck root localmount
> > > > >
> > > > > # /lib/rc/bin/rc-depend zfs-import
> > > > > multipath sysfs devfs udev udev-trigger modules fsck root localmount
> > > > > multipathd zfs-import
> > > > >
> > > > > # /lib/rc/bin/rc-depend multipath
> > > > > multipath
> > > > >
> > > > > From how I read these, it should be able to start "localmount"
> > > > > properly
> > > > > before even trying to start "multipathd" and "zfs-import"
> > > > > There is also no technical dependency for "localmount" (the root
> > > > > filesystem
> > > > > is not on ZFS on this system)
> > > > >
> > > > > Any help/suggestions on how to find the cause would be appreciated.
> > > > >
> > > > > --
> > > > > Joost
> > > >
> > > > Check if hwclock is in the boot runlevel:
> > > >
> > > > rc-update -s -v | grep hwclock
> > >
> > > What does "hwclock" got to do with this?
> > > It has no dependency with multipathd, zfs-import, localmount or anything
> > > else that is showing an error.
> > >
> > > --
> > > Joost
> >
> > Our systems are certainly different, but I noticed this dependency on my
> > localmount which is missing on yours:
> >
> > # /lib/rc/bin/rc-depend localmount
> > sysfs devfs udev udev-trigger hwclock modules fsck root dmcrypt localmount
> >
> > ^^^^^^^
> >
> > Have you compared your system services which has this problem, with other
> > systems of yours which can startup properly?
>
> Adding additional dependencies into the tree is more likely to cause further
> issues. I am actually looking for how to quickly find out which dependency
> is causing a circular dependency issue as the first time it thinks it needs
> to start a service it fails. But the 2nd time it starts, it goes correctly.
>
> I removed hwclock from ALL VMs as they don't actually have a hwclock.
>
> I did find out the actual cause of the problem through a lot of trial and
> error,

Out of curiosity - what was the cause of this? I have only come across
hwclock on my installations (not VMs).

> but this is not really useful in actually quickly finding the
> problem. Being able to "simulate" the startup sequence for how OpenRC wants
> to do things would have simplified and sped up the entire process.
>
> --
> Joost

I enable the rc log and check how the various services try to start up,
however the information provided is not always useful.

Re: How to find out all openrc dependencies? [ In reply to ]

joost at antarean

Apr 14, 2024, 9:40 AM

Post #7 of 7 (50 views)

Permalink

On Sunday, 14 April 2024 10:24:54 CEST Michael wrote:
> On Sunday, 14 April 2024 08:28:07 BST J. Roeleveld wrote:

<snipped>

> > >
> > > Our systems are certainly different, but I noticed this dependency on my
> > > localmount which is missing on yours:
> > >
> > > # /lib/rc/bin/rc-depend localmount
> > > sysfs devfs udev udev-trigger hwclock modules fsck root dmcrypt
> > > localmount
> > >
> > > ^^^^^^^
> > >
> > > Have you compared your system services which has this problem, with
> > > other
> > > systems of yours which can startup properly?
> >
> > Adding additional dependencies into the tree is more likely to cause
> > further issues. I am actually looking for how to quickly find out which
> > dependency is causing a circular dependency issue as the first time it
> > thinks it needs to start a service it fails. But the 2nd time it starts,
> > it goes correctly.
> >
> > I removed hwclock from ALL VMs as they don't actually have a hwclock.
> >
> > I did find out the actual cause of the problem through a lot of trial and
> > error,
>
> Out of curiosity - what was the cause of this? I have only come across
> hwclock on my installations (not VMs).

The issue is related to how openrc handles the boot process. It first starts
everything in "boot" and then switches to whatever is set as default runlevel.

At some point, Gentoo devs found it useful to add multipath and zfs-import to
the boot-runlevel. This was not originally the case.
I consider it best that the boot-runlevel is kept as empty as possible and
have the user/admin add necessary stuff to it when needed.

I removed them from the boot runlevel and the issue has been resolved.

This was the last item on the list preventing correct parallel boot and my
server now boots in 8 minutes instead of the original 20.
It's still 8 because I need to add some tricks to reduce some of the
simultaneous stuff which will lead to timeout issues which actually make it
worse if left to defaults.

> > but this is not really useful in actually quickly finding the
> > problem. Being able to "simulate" the startup sequence for how OpenRC
> > wants
> > to do things would have simplified and sped up the entire process.
> >
> > --
> > Joost
>
> I enable the rc log and check how the various services try to start up,
> however the information provided is not always useful.

The rc.log is actually less useful then one would expect. It only logs the
same output I see on the terminal.
Unless there is an option to add additional information to the logs, like how
it determines what to start next?

--
Joost

Mailing List Archive

Attached Files:

Attached Files:

Attached Files: