Mailing List Archive: Encrypted hard drives on LVM and urgent power shutdowns.

Encrypted hard drives on LVM and urgent power shutdowns.

rdalek1967 at gmail

Sep 11, 2022, 6:56 PM

Post #1 of 8 (242 views)

Howdy,

Last night we had some bad weather where I live and we ended up with
some power problems. Ironically they went out a few hours after the
storm was gone. Anyway. I had all sorts of encrypted drives open. My
usual drives inside my puter plus the large 14TB external backup drive
that is still copying files over. Glad my UPS held up while I closed
all those drives and did a proper shutdown. Doing all that tho, it made
me think about if I wasn't here to do all that. Being Linux, I'd
suspect that upsmon would tell the puter to do a proper shutdown which
includes unmounting the file system, closing the encrypted drives, like
I do with cryptsetup close <name> etc and then shutting down. However,
one has to ask, is it set up to do so by default? I manage the
encrypted drives manually. I don't use the crypt services for that like
people do when all of the system drive(s) is encrypted or when just
/home is encrypted. My encrypted stuff is mounted within /home or for
the external backups, in /mnt. Thing is, some aren't open unless I'm
using them or are external. Since I do it manually, is there a tool
that sees they need unmounting and closing and does it or do I need to
do something to make sure it is done before a shutdown?

I suspect this would happen on its own but I'd like to make sure. I'd
hate to mess up the file system badly on any of my drives or in a worst
case scenario, brick a hard drive with some 1 in a million chance problem.

I thought about having a drive connected, open and mounted that I don't
really need and just do a shutdown, see what happens. Then again, why
not ask and see if anyone else has had this happen and if things turned
out OK or if there was problems. I'm lucky, most of the time I'm either
home or very close by. Still, it can happen when I'm not here. I
already wonder if upsmon will kick in correctly and do a proper
shutdown. After all, it has never had to before. I'm running on faith
that it will. I hope I'm right.

Thoughts? Default will take care of things? I need to take steps to be
sure in case I'm not here? Personal experience? A good theory? ;-)

Thanks.

Dale

:-) :-)

Re: Encrypted hard drives on LVM and urgent power shutdowns. [ In reply to ]

gentoo at tastytea

Sep 11, 2022, 8:40 PM

Post #2 of 8 (242 views)

Permalink

On 2022-09-11 20:56-0500 Dale <rdalek1967@gmail.com> wrote:

> Howdy,
>
> Last night we had some bad weather where I live and we ended up with
> some power problems. Ironically they went out a few hours after the
> storm was gone. Anyway. I had all sorts of encrypted drives open.
> My usual drives inside my puter plus the large 14TB external backup
> drive that is still copying files over. Glad my UPS held up while I
> closed all those drives and did a proper shutdown. Doing all that
> tho, it made me think about if I wasn't here to do all that. Being
> Linux, I'd suspect that upsmon would tell the puter to do a proper
> shutdown which includes unmounting the file system, closing the
> encrypted drives, like I do with cryptsetup close <name> etc and then
> shutting down. However, one has to ask, is it set up to do so by
> default? I manage the encrypted drives manually. I don't use the
> crypt services for that like people do when all of the system
> drive(s) is encrypted or when just /home is encrypted. My encrypted
> stuff is mounted within /home or for the external backups, in /mnt.
> Thing is, some aren't open unless I'm using them or are external.
> Since I do it manually, is there a tool that sees they need
> unmounting and closing and does it or do I need to do something to
> make sure it is done before a shutdown?
>
> I suspect this would happen on its own but I'd like to make sure. I'd
> hate to mess up the file system badly on any of my drives or in a
> worst case scenario, brick a hard drive with some 1 in a million
> chance problem.
>
> I thought about having a drive connected, open and mounted that I
> don't really need and just do a shutdown, see what happens. Then
> again, why not ask and see if anyone else has had this happen and if
> things turned out OK or if there was problems. I'm lucky, most of
> the time I'm either home or very close by. Still, it can happen when
> I'm not here. I already wonder if upsmon will kick in correctly and
> do a proper shutdown. After all, it has never had to before. I'm
> running on faith that it will. I hope I'm right.
>
> Thoughts? Default will take care of things? I need to take steps to
> be sure in case I'm not here? Personal experience? A good theory?
> ;-)

Yes, /etc/init.d/mount-ro will take care of that. It first calls `sync`
and then calls `umount -r` on everything. It's set up to ruin on
shutdown by default. I'm sure systemd does something similar.

I don't think `cryptsetup luksClose` is necessary on shutdown, since it
only sets up the mapping(?).

Kind regards, tastytea

Re: Encrypted hard drives on LVM and urgent power shutdowns. [ In reply to ]

rdalek1967 at gmail

Sep 11, 2022, 8:58 PM

Post #3 of 8 (241 views)

Permalink

tastytea wrote:
> On 2022-09-11 20:56-0500 Dale <rdalek1967@gmail.com> wrote:
>
>> Howdy,
>>
>> Last night we had some bad weather where I live and we ended up with
>> some power problems. Ironically they went out a few hours after the
>> storm was gone. Anyway. I had all sorts of encrypted drives open.
>> My usual drives inside my puter plus the large 14TB external backup
>> drive that is still copying files over. Glad my UPS held up while I
>> closed all those drives and did a proper shutdown. Doing all that
>> tho, it made me think about if I wasn't here to do all that. Being
>> Linux, I'd suspect that upsmon would tell the puter to do a proper
>> shutdown which includes unmounting the file system, closing the
>> encrypted drives, like I do with cryptsetup close <name> etc and then
>> shutting down. However, one has to ask, is it set up to do so by
>> default? I manage the encrypted drives manually. I don't use the
>> crypt services for that like people do when all of the system
>> drive(s) is encrypted or when just /home is encrypted. My encrypted
>> stuff is mounted within /home or for the external backups, in /mnt.
>> Thing is, some aren't open unless I'm using them or are external.
>> Since I do it manually, is there a tool that sees they need
>> unmounting and closing and does it or do I need to do something to
>> make sure it is done before a shutdown?
>>
>> I suspect this would happen on its own but I'd like to make sure. I'd
>> hate to mess up the file system badly on any of my drives or in a
>> worst case scenario, brick a hard drive with some 1 in a million
>> chance problem.
>>
>> I thought about having a drive connected, open and mounted that I
>> don't really need and just do a shutdown, see what happens. Then
>> again, why not ask and see if anyone else has had this happen and if
>> things turned out OK or if there was problems. I'm lucky, most of
>> the time I'm either home or very close by. Still, it can happen when
>> I'm not here. I already wonder if upsmon will kick in correctly and
>> do a proper shutdown. After all, it has never had to before. I'm
>> running on faith that it will. I hope I'm right.
>>
>> Thoughts? Default will take care of things? I need to take steps to
>> be sure in case I'm not here? Personal experience? A good theory?
>> ;-)
> Yes, /etc/init.d/mount-ro will take care of that. It first calls `sync`
> and then calls `umount -r` on everything. It's set up to ruin on
> shutdown by default. I'm sure systemd does something similar.
>
> I don't think `cryptsetup luksClose` is necessary on shutdown, since it
> only sets up the mapping(?).
>
> Kind regards, tastytea
>
>

Thanks much for this info. I figured there was some tool that would do
that regardless of what it was. I know regular file systems would be
and couldn't imagine that encrypted would be any different but I didn't
want to find out I was wrong the hard way. After all, this 14TB backup
has been running for a few days now. Even when it gets through, I have
to run it again because of additions and other changes I made in the
past few days. While I could just start over with a fresh backup if it
got damaged, it would be time consuming to do so. Also, it would put
data at risk if I had a failure of the running drives while that backup
was not available. Not likely but bad things happen.

Next time power fails, I'll just stop all the processes I can and then
do a shutdown, knowing that everything will close safely. That will
save me some battery time as well.

Thanks much.

Dale

:-) :-)

Re: Encrypted hard drives on LVM and urgent power shutdowns. [ In reply to ]

billk at iinet

Sep 11, 2022, 10:08 PM

Post #4 of 8 (242 views)

Permalink

If your using nut, it has to be setup - and should be regularly tested
to make sure it works.

BillK

'

On 12/9/22 09:56, Dale wrote:
> Howdy,
>
> Last night we had some bad weather where I live and we ended up with
> some power problems. Ironically they went out a few hours after the
> storm was gone. Anyway. I had all sorts of encrypted drives open. My
> usual drives inside my puter plus the large 14TB external backup drive
> that is still copying files over. Glad my UPS held up while I closed
> all those drives and did a proper shutdown. Doing all that tho, it made
> me think about if I wasn't here to do all that. Being Linux, I'd
> suspect that upsmon would tell the puter to do a proper shutdown which
> includes unmounting the file system, closing the encrypted drives, like
> I do with cryptsetup close <name> etc and then shutting down. However,
> one has to ask, is it set up to do so by default? I manage the
> encrypted drives manually. I don't use the crypt services for that like
> people do when all of the system drive(s) is encrypted or when just
> /home is encrypted. My encrypted stuff is mounted within /home or for
> the external backups, in /mnt. Thing is, some aren't open unless I'm
> using them or are external. Since I do it manually, is there a tool
> that sees they need unmounting and closing and does it or do I need to
> do something to make sure it is done before a shutdown?
>
> I suspect this would happen on its own but I'd like to make sure. I'd
> hate to mess up the file system badly on any of my drives or in a worst
> case scenario, brick a hard drive with some 1 in a million chance problem.
>
> I thought about having a drive connected, open and mounted that I don't
> really need and just do a shutdown, see what happens. Then again, why
> not ask and see if anyone else has had this happen and if things turned
> out OK or if there was problems. I'm lucky, most of the time I'm either
> home or very close by. Still, it can happen when I'm not here. I
> already wonder if upsmon will kick in correctly and do a proper
> shutdown. After all, it has never had to before. I'm running on faith
> that it will. I hope I'm right.
>
> Thoughts? Default will take care of things? I need to take steps to be
> sure in case I'm not here? Personal experience? A good theory? ;-)
>
> Thanks.
>
> Dale
>
> :-) :-)
>

Re: Encrypted hard drives on LVM and urgent power shutdowns. [ In reply to ]

rdalek1967 at gmail

Sep 11, 2022, 10:57 PM

Post #5 of 8 (241 views)

Permalink

William Kenworthy wrote:
> If your using nut, it has to be setup - and should be regularly tested
> to make sure it works.
>
> BillK

I think upsmon is part of nut. I keep forgetting that since the service
is ups something. Thing is, I've never quite figured out how to test it
without unplugging and running down the batteries. I do have everything
configured and have ever since I built this puter. I did the same on
previous puter and UPS. Pretty sure it will work. When power blinks
etc, it does notice it and logs it in messages file. Also, the upsc
command outputs the info correctly when run.

I wish I could send a command to the UPS to fake a power failure, wait
say one minute and then it tell puter to shutdown all on its own. If it
does it correctly, it should work in the event of a actual power failure
and not run down my batteries either. I try to keep the batteries
topped off at all times since we do on occasion have some crazy driver
not watching where he/she is going and hits a power pole. Doesn't do
the pole any good and the car seems to not enjoy it either. :/
Sometimes the driver is no longer caring about it.

I actually wish I had a much larger external battery. Thing is, I'm
concerned about the charging bit. The charging section in a UPS isn't
really that powerful since it mostly just keeps the batteries topped
off. I have 7Amp/hr batteries and I think it takes like 7 or 8 hours to
charge from almost dead. I'd guess it is at most a 1 to 1.5 amp
charging circuit.

If you know of a command to test without running down batteries, I have
a Cyberpower UPS and I'm certainly interested. It's only a decade or so
old so has quite a few features. There may be a way to do this but I've
yet to find it.

Thanks.

Dale

:-) :-)

Re: Encrypted hard drives on LVM and urgent power shutdowns. [ In reply to ]

confabulate at kintzios

Sep 12, 2022, 12:49 AM

Post #6 of 8 (241 views)

Permalink

On Monday, 12 September 2022 06:57:58 BST Dale wrote:
> William Kenworthy wrote:
> > If your using nut, it has to be setup - and should be regularly tested
> > to make sure it works.
> >
> > BillK
>
> I think upsmon is part of nut. I keep forgetting that since the service
> is ups something. Thing is, I've never quite figured out how to test it
> without unplugging and running down the batteries. I do have everything
> configured and have ever since I built this puter. I did the same on
> previous puter and UPS. Pretty sure it will work. When power blinks
> etc, it does notice it and logs it in messages file. Also, the upsc
> command outputs the info correctly when run.
>
> I wish I could send a command to the UPS to fake a power failure, wait
> say one minute and then it tell puter to shutdown all on its own. If it
> does it correctly, it should work in the event of a actual power failure
> and not run down my batteries either. I try to keep the batteries
> topped off at all times since we do on occasion have some crazy driver
> not watching where he/she is going and hits a power pole. Doesn't do
> the pole any good and the car seems to not enjoy it either. :/
> Sometimes the driver is no longer caring about it.
>
> I actually wish I had a much larger external battery. Thing is, I'm
> concerned about the charging bit. The charging section in a UPS isn't
> really that powerful since it mostly just keeps the batteries topped
> off. I have 7Amp/hr batteries and I think it takes like 7 or 8 hours to
> charge from almost dead. I'd guess it is at most a 1 to 1.5 amp
> charging circuit.

Many UPSs containing two batteries connected in series always end up with one
battery less charged than the other. This is because although the current
which flows through the circuit is the same, due to Ohm's Law the voltage
drops across each battery as the resistance decreases along the circuit.
Therefore the first battery never gets fully charged. The opposite happens
with a discharge cycle.

Once every three months or every time I experience a noticeably prolonged
power cut, I shutdown the loads, take the UPS off line, disconnect the
batteries and top up each battery's charge with a car battery charger. Then I
replace them but make sure I reverse their positions. I have found this is
the best way to maximise their service life.

I also use the car battery charger to top up single battery UPSs, but I only
do this twice a year. I have found the charge these single batteries need is
comparably shorter.

> If you know of a command to test without running down batteries, I have
> a Cyberpower UPS and I'm certainly interested. It's only a decade or so
> old so has quite a few features. There may be a way to do this but I've
> yet to find it.
>
> Thanks.
>
> Dale
>
> :-) :-)

Check the NUT command set and options, in particular upsmon. You'll probably
want to try issuing a 'forced shutdown' command 'upsmon -c fsd', which won't
wait for the batteries to discharge first, like it does with upssched. Before
you try this, best sync your disks first and remount them read only just in
case.

There may be some UPS specific test function too, this very much depends on
the UPS and driver. You can check what variables a UPS has and how to tweak
them to cause a shutdown by running 'upsrw -l', if any of the exposed UPS
variable are tweakable.

RE: Encrypted hard drives on LVM and urgent power shutdowns. [ In reply to ]

lperkins at openeye

Sep 12, 2022, 9:54 AM

Post #7 of 8 (241 views)

Permalink

>-----Original Message-----
>From: Dale <rdalek1967@gmail.com>
>Sent: Sunday, September 11, 2022 10:58 PM
>To: gentoo-user@lists.gentoo.org
>Subject: Re: [gentoo-user] Encrypted hard drives on LVM and urgent power shutdowns.
>
>William Kenworthy wrote:
>> If your using nut, it has to be setup - and should be regularly tested
>> to make sure it works.
>>
>> BillK
>
>
>I think upsmon is part of nut. I keep forgetting that since the service is ups something. Thing is, I've never quite figured out how to test it without unplugging and running down the batteries. I do have everything configured and have ever since I built this puter. I did the same on previous puter and UPS. Pretty sure it will work. When power blinks etc, it does notice it and logs it in messages file. Also, the upsc command outputs the info correctly when run.
>
>I wish I could send a command to the UPS to fake a power failure, wait say one minute and then it tell puter to shutdown all on its own. If it does it correctly, it should work in the event of a actual power failure and not run down my batteries either. I try to keep the batteries topped off at all times since we do on occasion have some crazy driver not watching where he/she is going and hits a power pole. Doesn't do the pole any good and the car seems to not enjoy it either. :/ Sometimes the driver is no longer caring about it.
>
>I actually wish I had a much larger external battery. Thing is, I'm concerned about the charging bit. The charging section in a UPS isn't really that powerful since it mostly just keeps the batteries topped off. I have 7Amp/hr batteries and I think it takes like 7 or 8 hours to charge from almost dead. I'd guess it is at most a 1 to 1.5 amp charging circuit.
>
>If you know of a command to test without running down batteries, I have a Cyberpower UPS and I'm certainly interested. It's only a decade or so old so has quite a few features. There may be a way to do this but I've yet to find it.
>
>Thanks.
>
>Dale
>
>:-) :-)
>

Some of the higher-end UPS models do have diagnostic modes for simulating various events to make sure the connected systems behave as desired. A very few of the consumer-grade ones do as well. But how to do it is model specific, so you'll have to dig up the documentation.

Commercial-grade units also often have a DC port on the back so you can plug in larger battery banks and/or hotswap battery banks during extended outages.

If you want an arbitrarily large battery bank, just get a decent power inverter heavy enough to run your load and a battery float charger that can push enough amps to keep up, then put as big a stack of batteries as you like between the two. The nicer inverters will even warn you when the batteries get low.

You can often get used batteries from the local automotive shop for just the core charge. Just because it can't provide 600 amps to start a car any more doesn't mean it can't provide 60 to run your computers. Obviously they'll require more regular maintenance, but it's hard to beat the price.

LMP

Re: Encrypted hard drives on LVM and urgent power shutdowns. [ In reply to ]

rich0 at gentoo

Sep 12, 2022, 10:27 AM

Post #8 of 8 (241 views)

Permalink

On Sun, Sep 11, 2022 at 9:56 PM Dale <rdalek1967@gmail.com> wrote:
>
> I suspect this would happen on its own but I'd like to make sure. I'd
> hate to mess up the file system badly on any of my drives or in a worst
> case scenario, brick a hard drive with some 1 in a million chance problem.
>

I just wanted to comment that LUKS encryption on linux is pretty-much
a block-level passthrough. So if your filesystem is journaled and
using barriers or syncing to ensure consistency, and you add LUKS to
it, then you shouldn't really see any difference in behavior if it is
interrupted uncleanly by a power loss. The encryption could add a bit
of latency but that shouldn't change much.

Of course different filesystems handle interruptions differently, and
all those caveats still apply.

As far as unmounting goes, you just need to umount the filesystem.
umount will block until all writes are synced to disk, and that
includes all layers like LVM/LUKS/mdadm/whatever that might be
underneath it. If umount returns, then all your data is written to
disk and if at that instant you lose power there will be no data loss
for that filesystem. I guess if you're using mdadm and you have
multiple filesystems not aligned to a stripe boundary, then the raid
write hole might still apply, and that is true at anytime whether the
filesystem is mounted or not - data on a stripe shared with some other
active filesystem could get lost in some situations.

Obviously if you lose the key to a LUKS filesystem or if there is some
kind of bug in LUKS the use of encryption could hinder data recovery.
Beyond that it really shouldn't have any impact on anything. I guess
it would also give you more exposure to RAM errors (since that is
another code path that stores stuff in RAM).

As already discussed, clean shutdowns triggered by NUT/etc are of
course best, but the use of LUKS shouldn't change much with the use of
a UPS otherwise.

--
Rich

Mailing List Archive

Mailing List Archive

Attached Files: