Mailing List Archive

How to unmount bind-mounted /dev?
Hi there!

I want to shrink [*] a partition that holds a 32-bit Gentoo chroot. But
I cannot unmount it, the device is busy because proc and /dev is
bind-mounted there. And I cannot unmount this /dev, again, the device is
busy:

weird ~ # umount /32/dev
umount: /32/dev: device is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))

lsof /32/dev gives a lot of stuff, the same that lsof /dev gives. lsof
/proc also has some lines, but I had no problem unmounting /32/proc.

I could remove the /32/dev entry from fstab and reboot, but I need the
machine up at the moment, and I'm also curious why this is a problem.
Maybe you know?

[*] BTW, I just wrote a script so automatize this. It shrinks the file
system, then reduces the logical volume. It also reduces the size of a
LUKS crypto volume on the LVM.
Enlarging also works, but that already is easy using lvresize,
cryptsetup resize and resize2fs. Shrinking is more tricky though, you
have to calculate the sizes, and things could break if the underlying
partition is made too small.

Wonko
Re: How to unmount bind-mounted /dev? [ In reply to ]
On Saturday 30 April 2011 Alex Schuster wrote:
[...]

Hello,

> weird ~ # umount /32/dev
> umount: /32/dev: device is busy.
> (In some cases useful info about processes that use
> the device is found by lsof(8) or fuser(1))
try 'umount -l /32/de'

Thomas
Re: How to unmount bind-mounted /dev? [ In reply to ]
Thomas Ulrich Nockmann writes:

> On Saturday 30 April 2011 Alex Schuster wrote:

>> weird ~ # umount /32/dev
>> umount: /32/dev: device is busy.
>> (In some cases useful info about processes that use
>> the device is found by lsof(8) or fuser(1))
> try 'umount -l /32/de'

Cool, this does the trick!

Thanks,

Wonko
Re: How to unmount bind-mounted /dev? [ In reply to ]
I just wrote:

> Thomas Ulrich Nockmann writes:
>
>> On Saturday 30 April 2011 Alex Schuster wrote:
>
>>> weird ~ # umount /32/dev
>>> umount: /32/dev: device is busy.
>>> (In some cases useful info about processes that use
>>> the device is found by lsof(8) or fuser(1))
>> try 'umount -l /32/de'
>
> Cool, this does the trick!

But it does not help :( After unmounting /32/dev, I can finally unmount
/32, but now the fsck fails:

weird ~ # fsck -Cf /dev/mapper/32
fsck from util-linux 2.19
e2fsck 1.41.14 (22-Dec-2010)
fsck.ext3: Device or resource busy while trying to open /dev/mapper/32
Filesystem mounted or opened exclusively by another program?

lsof and fuser report nothing. I guess I will have to reboot then.

Wonko
Re: How to unmount bind-mounted /dev? [ In reply to ]
On Sat, Apr 30, 2011 at 5:15 AM, Alex Schuster <wonko@wonkology.org> wrote:

> I just wrote:
>
> > Thomas Ulrich Nockmann writes:
> >
> >> On Saturday 30 April 2011 Alex Schuster wrote:
> >
> >>> weird ~ # umount /32/dev
> >>> umount: /32/dev: device is busy.
> >>> (In some cases useful info about processes that use
> >>> the device is found by lsof(8) or fuser(1))
> >> try 'umount -l /32/de'
> >
> > Cool, this does the trick!
>
> But it does not help :( After unmounting /32/dev, I can finally unmount
> /32, but now the fsck fails:
>
> weird ~ # fsck -Cf /dev/mapper/32
> fsck from util-linux 2.19
> e2fsck 1.41.14 (22-Dec-2010)
> fsck.ext3: Device or resource busy while trying to open /dev/mapper/32
> Filesystem mounted or opened exclusively by another program?
>
> lsof and fuser report nothing. I guess I will have to reboot then.
>
> Wonko
>
>
Try a lazy umount, or forced umount?

# umount -f
# umount -l
Re: How to unmount bind-mounted /dev? [ In reply to ]
Mark Shields writes:

> On Sat, Apr 30, 2011 at 5:15 AM, Alex Schuster <wonko@wonkology.org
> <mailto:wonko@wonkology.org>> wrote:
>
> I just wrote:
>
> > Thomas Ulrich Nockmann writes:

> >> try 'umount -l /32/de'
> >
> > Cool, this does the trick!
>
> But it does not help :( After unmounting /32/dev, I can finally unmount
> /32, but now the fsck fails:
>
> weird ~ # fsck -Cf /dev/mapper/32
> fsck from util-linux 2.19
> e2fsck 1.41.14 (22-Dec-2010)
> fsck.ext3: Device or resource busy while trying to open /dev/mapper/32
> Filesystem mounted or opened exclusively by another program?
>
> lsof and fuser report nothing. I guess I will have to reboot then.

> Try a lazy umount, or forced umount?
>
> # umount -f
> # umount -l

The lazy unmount was Thomas' hint already and worked, the partition is
no longer mounted. But I cannot fsck it, it is still in use. cryptsetup
luksClose works neither.
It's no big trouble, but still I'm curious why this is.

Wonko
Re: How to unmount bind-mounted /dev? [ In reply to ]
On Sunday 01 May 2011 00:48:38 Alex Schuster wrote:
> Mark Shields writes:
> > On Sat, Apr 30, 2011 at 5:15 AM, Alex Schuster <wonko@wonkology.org
> >
> > <mailto:wonko@wonkology.org>> wrote:
> > I just wrote:
> > > Thomas Ulrich Nockmann writes:
> > >> try 'umount -l /32/de'
> > >
> > > Cool, this does the trick!
> >
> > But it does not help :( After unmounting /32/dev, I can finally
> > unmount /32, but now the fsck fails:
> >
> > weird ~ # fsck -Cf /dev/mapper/32
> > fsck from util-linux 2.19
> > e2fsck 1.41.14 (22-Dec-2010)
> > fsck.ext3: Device or resource busy while trying to open
> > /dev/mapper/32 Filesystem mounted or opened exclusively by another
> > program?
> >
> > lsof and fuser report nothing. I guess I will have to reboot then.
> >
> > Try a lazy umount, or forced umount?
> >
> > # umount -f
> > # umount -l
>
> The lazy unmount was Thomas' hint already and worked, the partition is
> no longer mounted. But I cannot fsck it, it is still in use. cryptsetup
> luksClose works neither.
> It's no big trouble, but still I'm curious why this is.
>
> Wonko

Asking the obvious: could this message be there because this partition is
still mounted? Did you check that this partition has been unmounted from all
mount points, both original mount point and bind-mount?
--
Regards,
Mick
Re: How to unmount bind-mounted /dev? [ In reply to ]
Mick writes:

> On Sunday 01 May 2011 00:48:38 Alex Schuster wrote:

> > The lazy unmount was Thomas' hint already and worked, the partition is
> > no longer mounted. But I cannot fsck it, it is still in use. cryptsetup
> > luksClose works neither.
> > It's no big trouble, but still I'm curious why this is.

> Asking the obvious: could this message be there because this partition
> is still mounted?

I grepped /proc/mounts for it and saw no references. I'm pretty sure I did
not overlook something. And it's already the second time I tried this, one
month ago the same had happened, but I did not care about it then.

> Did you check that this partition has been unmounted
> from all mount points, both original mount point and bind-mount?

The partition only has one mount point, but others were mounted inside it:

/dev/mapper/32 on /32 type ext3 (rw,noatime)
/dev on /32/dev type none (rw,bind)
/proc on /32/proc type none (rw,bind)
/home on /32/home type none (rw,bind,noatime)
/var/portage on /32/var/portage type none (rw,bind,noatime)

/var/portage has another file system inside for the portage tree. When I
want to unmount /32/var/portage, I have to unmount /32/var/portage/tree
first. All except /32/dev could be unmounted, for /32/dev I needed the -l
option to mount. Then /32 itself could be unmounted. But things like fsck or
'cryptsetup remove' failed, /dev/mapper/32 was in use.

Then I rebooted, but I had forgotten to save my changes to fstab, so all
those things were again mounted. I tried again anyway, and this time there
was no problem. I had to use umount -l for /32/dev again, and this time also
for /32/proc (that was not necessary the last time), but after unmounting
/32, I could fsck its partition and shrink it.

I have no idea why it did not work the last times I tried. I'll try to
reproduce this from time to time, maybe after some more uptime and work on
this partition it will happen again.

Wonko
Re: How to unmount bind-mounted /dev? [ In reply to ]
On Sunday 01 May 2011 14:08:36 Alex Schuster wrote:
> Mick writes:
> > On Sunday 01 May 2011 00:48:38 Alex Schuster wrote:
> > > The lazy unmount was Thomas' hint already and worked, the partition is
> > > no longer mounted. But I cannot fsck it, it is still in use. cryptsetup
> > > luksClose works neither.
> > > It's no big trouble, but still I'm curious why this is.
> >
> > Asking the obvious: could this message be there because this partition
> > is still mounted?
>
> I grepped /proc/mounts for it and saw no references. I'm pretty sure I did
> not overlook something. And it's already the second time I tried this, one
> month ago the same had happened, but I did not care about it then.
>
> > Did you check that this partition has been unmounted
> > from all mount points, both original mount point and bind-mount?
>
> The partition only has one mount point, but others were mounted inside it:
>
> /dev/mapper/32 on /32 type ext3 (rw,noatime)
> /dev on /32/dev type none (rw,bind)
> /proc on /32/proc type none (rw,bind)
> /home on /32/home type none (rw,bind,noatime)
> /var/portage on /32/var/portage type none (rw,bind,noatime)
>
> /var/portage has another file system inside for the portage tree. When I
> want to unmount /32/var/portage, I have to unmount /32/var/portage/tree
> first. All except /32/dev could be unmounted, for /32/dev I needed the -l
> option to mount. Then /32 itself could be unmounted. But things like fsck
> or 'cryptsetup remove' failed, /dev/mapper/32 was in use.
>
> Then I rebooted, but I had forgotten to save my changes to fstab, so all
> those things were again mounted. I tried again anyway, and this time there
> was no problem. I had to use umount -l for /32/dev again, and this time
> also for /32/proc (that was not necessary the last time), but after
> unmounting /32, I could fsck its partition and shrink it.
>
> I have no idea why it did not work the last times I tried. I'll try to
> reproduce this from time to time, maybe after some more uptime and work on
> this partition it will happen again.

Yes, you've done the right thing, unmounting directories from the lower to the
higher, before you try to unmount the top of the tree. I would think that
/dev and /proc would be accessed by the OS in real time, every time you
read/write to a device/memory/acpi, etc. so trying to umount them could be
more troublesome. Perhaps immediately after rebooting there was not much
activity from previous actions and that's why you were able to unmount them
without too much trouble.

--
Regards,
Mick