Mailing List Archive: No quorum on UpToDate devices

No quorum on UpToDate devices

Mar 15, 2023, 9:27 AM

Post #1 of 4 (169 views)

Hi, any idea why drbd can lose their quorum? All the replicas are UpToDate:

root@kube-node-14:/# drbdsetup status
pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Primary
disk:Diskless
kube-node-4 role:Secondary
peer-disk:UpToDate
kube-node-6 role:Secondary
peer-disk:UpToDate

root@kube-node-6:/# drbdsetup status
pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Secondary suspended:quorum
disk:UpToDate quorum:no blocked:upper
kube-node-14 role:Primary
peer-disk:Diskless
kube-node-4 role:Secondary
peer-disk:UpToDate

root@kube-node-4:/# drbdsetup status
pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Secondary suspended:quorum
disk:UpToDate quorum:no blocked:upper
kube-node-14 role:Primary
peer-disk:Diskless
kube-node-6 role:Secondary
peer-disk:UpToDate

Best Regards,
Andrei Kvapil

Re: No quorum on UpToDate devices [ In reply to ]

bajizs at cnt

Mar 15, 2023, 10:57 AM

Post #2 of 4 (168 views)

Permalink

Helo,

blocked:upper - Your physical device (HDD/SSD) is blocked, can not be
accessed. You need to solve this problem

Or sometimes the drbd kernel process for this drbd device stop to work.
Check it with dmesg, you can find log from some frozed drbd process.
For this situation You can only restart the server.

Zsolt

2023. 03. 15. 17:27 keltezéssel, kvaps írta:

> Hi, any idea why drbd can lose their quorum? All the replicas are
> UpToDate:
>
> root@kube-node-14:/# drbdsetup status
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Primary
> disk:Diskless
> kube-node-4 role:Secondary
> peer-disk:UpToDate
> kube-node-6 role:Secondary
> peer-disk:UpToDate
>
> root@kube-node-6:/# drbdsetup status
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Secondary suspended:quorum
> disk:UpToDate quorum:no blocked:upper
> kube-node-14 role:Primary
> peer-disk:Diskless
> kube-node-4 role:Secondary
> peer-disk:UpToDate
>
> root@kube-node-4:/# drbdsetup status
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Secondary suspended:quorum
> disk:UpToDate quorum:no blocked:upper
> kube-node-14 role:Primary
> peer-disk:Diskless
> kube-node-6 role:Secondary
> peer-disk:UpToDate
>
>
> Best Regards,
> Andrei Kvapil
>
> _______________________________________________
> Star us on GITHUB:https://github.com/LINBIT
> drbd-user mailing list
> drbd-user@lists.linbit.com
> https://lists.linbit.com/mailman/listinfo/drbd-user

Re: No quorum on UpToDate devices [ In reply to ]

gianni.milo22 at gmail

Mar 15, 2023, 11:58 AM

Post #3 of 4 (168 views)

Permalink

Which drbd kmod version are you using?

There have been some changes lately in the quorum mechanism, which you may
benefit from...

https://lists.linbit.com/pipermail/drbd-user/2023-January/026354.html

On Wed, 15 Mar 2023 at 16:28, kvaps <kvapss@gmail.com> wrote:

> Hi, any idea why drbd can lose their quorum? All the replicas are UpToDate:
>
> root@kube-node-14:/# drbdsetup status
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Primary
> disk:Diskless
> kube-node-4 role:Secondary
> peer-disk:UpToDate
> kube-node-6 role:Secondary
> peer-disk:UpToDate
>
> root@kube-node-6:/# drbdsetup status
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Secondary suspended:quorum
> disk:UpToDate quorum:no blocked:upper
> kube-node-14 role:Primary
> peer-disk:Diskless
> kube-node-4 role:Secondary
> peer-disk:UpToDate
>
> root@kube-node-4:/# drbdsetup status
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Secondary suspended:quorum
> disk:UpToDate quorum:no blocked:upper
> kube-node-14 role:Primary
> peer-disk:Diskless
> kube-node-6 role:Secondary
> peer-disk:UpToDate
>
>
> Best Regards,
> Andrei Kvapil
> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT
> drbd-user mailing list
> drbd-user@lists.linbit.com
> https://lists.linbit.com/mailman/listinfo/drbd-user
>

Re: No quorum on UpToDate devices [ In reply to ]

kvapss at gmail

Mar 15, 2023, 3:45 PM

Post #4 of 4 (168 views)

Permalink

Yeah, the drbd version is 9.2.1 on all three nodes
I fixed that by adding additional diskless resource, then removing it.

Next time I'll try dump-md. Thank you!

Best Regards,
Andrei Kvapil

On Wed, Mar 15, 2023 at 7:59?PM GM <gianni.milo22@gmail.com> wrote:

> Which drbd kmod version are you using?
>
> There have been some changes lately in the quorum mechanism, which you may
> benefit from...
>
> https://lists.linbit.com/pipermail/drbd-user/2023-January/026354.html
>
>
>
> On Wed, 15 Mar 2023 at 16:28, kvaps <kvapss@gmail.com> wrote:
>
>> Hi, any idea why drbd can lose their quorum? All the replicas are
>> UpToDate:
>>
>> root@kube-node-14:/# drbdsetup status
>> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
>> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Primary
>> disk:Diskless
>> kube-node-4 role:Secondary
>> peer-disk:UpToDate
>> kube-node-6 role:Secondary
>> peer-disk:UpToDate
>>
>> root@kube-node-6:/# drbdsetup status
>> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
>> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Secondary suspended:quorum
>> disk:UpToDate quorum:no blocked:upper
>> kube-node-14 role:Primary
>> peer-disk:Diskless
>> kube-node-4 role:Secondary
>> peer-disk:UpToDate
>>
>> root@kube-node-4:/# drbdsetup status
>> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d
>> pvc-2ba7ccd7-e92c-4c92-be8e-6e55c9a0840d role:Secondary suspended:quorum
>> disk:UpToDate quorum:no blocked:upper
>> kube-node-14 role:Primary
>> peer-disk:Diskless
>> kube-node-6 role:Secondary
>> peer-disk:UpToDate
>>
>>
>> Best Regards,
>> Andrei Kvapil
>> _______________________________________________
>> Star us on GITHUB: https://github.com/LINBIT
>> drbd-user mailing list
>> drbd-user@lists.linbit.com
>> https://lists.linbit.com/mailman/listinfo/drbd-user
>>
> _______________________________________________
> Star us on GITHUB: https://github.com/LINBIT
> drbd-user mailing list
> drbd-user@lists.linbit.com
> https://lists.linbit.com/mailman/listinfo/drbd-user
>