Mailing List Archive

On Mon, Apr 18, 2022 at 09:43:38PM +0000, Eric Robinson wrote:
> New kernel versions are released fairly often. What are the rules
> about upgrading DRBD when that happens? Is it always necessary, or are
> there certain thresholds within which it is okay to upgrade the kernel
> without rebuilding DRBD? Rebuilding DRBD every time is pretty
> disruptive.

"depends", but in general that is how things work, there are no stable
kABI guarantees from the Linux kernel itself. So you have to rebuild
external modules. This is for example why dkms exists.

In more detail it depends on the distribution. RHEL guarantees a stable
kABI for certain symbols. Unfortunately sometimes even they make
mistakes and break it, but one usually gets away with relatively few
builds. Debian tries to keep a stable kABI, Ubuntu does not seem to do
so.

Needless to say that we for example test for new RHEL kernels and compat
and provide new binary modules for our customers. They just upgrade and
don't have to worry. Same for Debian - we build when we need to - we
build every Ubuntu kernel release. For the other distros it depends on
their model (which roughly depends on how "good" their RHEL clone
actually is). Even with rotating out old kernels rather quickly, we get
3 figure kernel numbers we build for easily.

Regards, rck
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

> -----Original Message-----
> From: drbd-user-bounces@lists.linbit.com <drbd-user-
> bounces@lists.linbit.com> On Behalf Of Roland Kammerer
> Sent: Tuesday, April 19, 2022 1:55 AM
> To: drbd-user@lists.linbit.com
> Subject: Re: [DRBD-user] Do You Always Have to Upgrade/Rebuild DRBD
> Whenever You Upgrade the Kernel?
>
> On Mon, Apr 18, 2022 at 09:43:38PM +0000, Eric Robinson wrote:
> > New kernel versions are released fairly often. What are the rules
> > about upgrading DRBD when that happens? Is it always necessary, or are
> > there certain thresholds within which it is okay to upgrade the kernel
> > without rebuilding DRBD? Rebuilding DRBD every time is pretty
> > disruptive.
>
> "depends", but in general that is how things work, there are no stable kABI
> guarantees from the Linux kernel itself. So you have to rebuild external
> modules. This is for example why dkms exists.
>
> In more detail it depends on the distribution. RHEL guarantees a stable kABI
> for certain symbols. Unfortunately sometimes even they make mistakes and
> break it, but one usually gets away with relatively few builds. Debian tries to
> keep a stable kABI, Ubuntu does not seem to do so.
>
> Needless to say that we for example test for new RHEL kernels and compat
> and provide new binary modules for our customers. They just upgrade and
> don't have to worry. Same for Debian - we build when we need to - we build
> every Ubuntu kernel release. For the other distros it depends on their model
> (which roughly depends on how "good" their RHEL clone actually is). Even
> with rotating out old kernels rather quickly, we get
> 3 figure kernel numbers we build for easily.
>
> Regards, rck
> ____

Thanks for the feedback, Roland. I am actually a paying Linbit customer, but I am unaware of the process to follow when upgrading the OS. For example, I'm currently running Rocky Linux 8.5 (kernel 4.18.0-348.2.1.el8_5.x86_64) and drbd 9.19.1-1.el8. There is a kernel upgrade to 4.18.0-348.20.1.el8_5 available in the Rocky repo. If I install that, does it break drbd? If so, are you saying that I should do a rolling upgrade of drbd on my cluster?

___________________________________________
> Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list drbd-
> user@lists.linbit.com https://lists.linbit.com/mailman/listinfo/drbd-user
Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

On Fri, Apr 22, 2022 at 09:05:46PM +0000, Eric Robinson wrote:
> Thanks for the feedback, Roland. I am actually a paying Linbit
> customer, but I am unaware of the process to follow when upgrading the
> OS. For example, I'm currently running Rocky Linux 8.5 (kernel
> 4.18.0-348.2.1.el8_5.x86_64) and drbd 9.19.1-1.el8.

The name of the meta package (i.e., "drbd") is IMO pretty unfortunate,
that version refers to DRBD utils. It does not matter, can not remember
when we did a breaking change in utils. What matters is kmod-drbd. But
you most likely know that.

> There is a kernel
> upgrade to 4.18.0-348.20.1.el8_5 available in the Rocky repo. If I
> install that, does it break drbd?

In 99.9% it does not break the binary module. That would only be the
case if the vendor broke the kAbi. We usually have 1 binary module per
RHEL dot release. So as long as you don't jump from like RHEL 8.4 to 8.5
or something then the same module should be fine. Also, you might enable
a "dot repository" (one that has .../rhel8.5/... in the URL, not the
global ".../rhel8/..." one that contains all the kernel modules for all
RHEL (8.0, 8.1, 8.2,...) if you are a customer. If you enabled the "dot
repo" you should be fine taking the latest kernel + the latest
kmod-drbd. If, and that is very unlikely, there are more than 1
kmod-drbd in a dot repo, use the one closest to your kernel version,
usually the later one. Again, that usually is not the case, only if the
kAbi was broken.

Finding the best matching kernel module to a given kernel for all DRBD
modules that exist is a problem we had to solve ourselves, we even have
a public web service one can use like this:
$ cat /etc/os-release | curl -T - -X POST drbd.io:3030/api/v1/best/$(uname -r) -s

At the time of writing it answered with:
kmod-drbd-9.1.7_4.18.0_348-1.x86_64.rpm

If you are into RHEL versioning, you see that this is the first
kernel-devel package in that kernel series. So, for 4.18 there has not
been any breakage. Which again is the usual case, the people at Redhat
know what they are doing. Most of the time :).

Code for that web service and the underlying python library can be found
here:
https://github.com/LINBIT/bestdrbdmodule
https://github.com/LINBIT/python-lbdist

Regards, rck
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user

> -----Original Message-----
> From: drbd-user-bounces@lists.linbit.com <drbd-user-
> bounces@lists.linbit.com> On Behalf Of Roland Kammerer
> Sent: Monday, April 25, 2022 1:40 AM
> To: drbd-user@lists.linbit.com
> Cc: Roland Kammerer <roland.kammerer@linbit.com>
> Subject: Re: [DRBD-user] Do You Always Have to Upgrade/Rebuild DRBD
> Whenever You Upgrade the Kernel?
>
> On Fri, Apr 22, 2022 at 09:05:46PM +0000, Eric Robinson wrote:
> > Thanks for the feedback, Roland. I am actually a paying Linbit
> > customer, but I am unaware of the process to follow when upgrading the
> > OS. For example, I'm currently running Rocky Linux 8.5 (kernel
> > 4.18.0-348.2.1.el8_5.x86_64) and drbd 9.19.1-1.el8.
>
> The name of the meta package (i.e., "drbd") is IMO pretty unfortunate, that
> version refers to DRBD utils. It does not matter, can not remember when we
> did a breaking change in utils. What matters is kmod-drbd. But you most
> likely know that.
>

Indeed. We've been using DRBD since 2006. We're not experts, but we've picked up a few of the basics. ????


> > There is a kernel
> > upgrade to 4.18.0-348.20.1.el8_5 available in the Rocky repo. If I
> > install that, does it break drbd?
>
> In 99.9% it does not break the binary module. That would only be the case if
> the vendor broke the kAbi. We usually have 1 binary module per RHEL dot
> release. So as long as you don't jump from like RHEL 8.4 to 8.5 or something
> then the same module should be fine. Also, you might enable a "dot
> repository" (one that has .../rhel8.5/... in the URL, not the global
> ".../rhel8/..." one that contains all the kernel modules for all RHEL (8.0, 8.1,
> 8.2,...) if you are a customer. If you enabled the "dot repo" you should be
> fine taking the latest kernel + the latest kmod-drbd. If, and that is very
> unlikely, there are more than 1 kmod-drbd in a dot repo, use the one closest
> to your kernel version, usually the later one. Again, that usually is not the
> case, only if the kAbi was broken.
>
> Finding the best matching kernel module to a given kernel for all DRBD
> modules that exist is a problem we had to solve ourselves, we even have a
> public web service one can use like this:
> $ cat /etc/os-release | curl -T - -X POST drbd.io:3030/api/v1/best/$(uname -r)
> -s
>
> At the time of writing it answered with:
> kmod-drbd-9.1.7_4.18.0_348-1.x86_64.rpm
>
> If you are into RHEL versioning, you see that this is the first kernel-devel
> package in that kernel series. So, for 4.18 there has not been any breakage.
> Which again is the usual case, the people at Redhat know what they are
> doing. Most of the time :).
>
> Code for that web service and the underlying python library can be found
> here:
> https://github.com/LINBIT/bestdrbdmodule
> https://github.com/LINBIT/python-lbdist
>
> Regards, rck

This is all excellent information, thanks much.

> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list drbd-
> user@lists.linbit.com https://lists.linbit.com/mailman/listinfo/drbd-user
Disclaimer : This email and any files transmitted with it are confidential and intended solely for intended recipients. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physician Select Management. Warning: Although Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
_______________________________________________
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
https://lists.linbit.com/mailman/listinfo/drbd-user