Mailing List Archive

On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund <
joakim.tjernlund@transmode.se> wrote:

>
> I got the impression from docs that PKG_INSTALL_MASK would actually
> mask files out so they never get into the binary package, this doesn't
> seem to happen.
> Did I misunderstand? If I did, I think an MASK to do the above
> would be a worthy addition to quickpkg, very useful for embedded targets
> to keep the pkg size down.
> Also PKG_INSTALL_KEEP which would list files keep(rest is dropped) would
> be nice.
>
> Oh, something else I wonder about. How does pre/post install work with
> quickpkgs? Is it possible to have such scripts and then have qmerge execute
> them?
>
> Jocke
>
>
> Yes, PKG_INSTALL_MASK on time of installation will mask the files defined
in it.
Binary package do contain all files, even the masked files.
Kfir

On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
>
>
> On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
>
> I got the impression from docs that PKG_INSTALL_MASK would
> actually
> mask files out so they never get into the binary package, this
> doesn't
> seem to happen.
> Did I misunderstand? If I did, I think an MASK to do the above
> would be a worthy addition to quickpkg, very useful for
> embedded targets to keep the pkg size down.
> Also PKG_INSTALL_KEEP which would list files keep(rest is
> dropped) would be nice.
>
> Oh, something else I wonder about. How does pre/post install
> work with
> quickpkgs? Is it possible to have such scripts and then have
> qmerge execute
> them?
>
> Jocke
>
>
> Yes, PKG_INSTALL_MASK on time of installation will mask the files
> defined in it.
> Binary package do contain all files, even the masked files.
> Kfir


$PKG_INSTALL_MASK is supposed to omit the files in it's list from making
it into the binary pkgs in the first place. The idea there was to make
smaller binary pkgs for embedded devices and such (for use only with
private repos).

$INSTALL_MASK is supposed to omit the files listed in it from being
installed on the file system.

--
solar <solar@gentoo.org>
Gentoo Linux

solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
>
> On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> >
> >
> > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > <joakim.tjernlund@transmode.se> wrote:
> >
> > I got the impression from docs that PKG_INSTALL_MASK would
> > actually
> > mask files out so they never get into the binary package, this
> > doesn't
> > seem to happen.
> > Did I misunderstand? If I did, I think an MASK to do the above
> > would be a worthy addition to quickpkg, very useful for
> > embedded targets to keep the pkg size down.
> > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > dropped) would be nice.
> >
> > Oh, something else I wonder about. How does pre/post install
> > work with
> > quickpkgs? Is it possible to have such scripts and then have
> > qmerge execute
> > them?
> >
> > Jocke
> >
> >
> > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > defined in it.
> > Binary package do contain all files, even the masked files.
> > Kfir
>
>
> $PKG_INSTALL_MASK is supposed to omit the files in it's list from making
> it into the binary pkgs in the first place. The idea there was to make
> smaller binary pkgs for embedded devices and such (for use only with
> private repos).
>
> $INSTALL_MASK is supposed to omit the files listed in it from being
> installed on the file system.

Hi Solar, long time no see :)

So the current behaviour is a bug, good to know.

I am contemplating a greater problem too. Our system require we can install
multiple versions of our SW and switch between them. This is easy to do when it
comes to our own app but not when one want to upgrade core parts of the system, like
libc etc.

So I am thinking one could use --bind mounts and switch_root to solve that. Basically
one has a skeleton root FS with /bin, /lib, /usr /opt etc.
Each upgrade goes into dirs like:
bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
and --bind mounts them under the corresponding skeleton dir.

To do that one needs an way to repackage a root fs created by a bunch of different
ebuilds, some of them spanning several of the above dirs, into a package per xx_1.x.y dir.
Not sure if this can be done with the current portage/portage-utils and how, any
ideas welcome :)

Perhaps there is a better way then the above?

Jocke

On Mon, 2011-12-26 at 19:03 +0100, Joakim Tjernlund wrote:
> solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
> >
> > On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> > >
> > >
> > > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > > <joakim.tjernlund@transmode.se> wrote:
> > >
> > > I got the impression from docs that PKG_INSTALL_MASK would
> > > actually
> > > mask files out so they never get into the binary package, this
> > > doesn't
> > > seem to happen.
> > > Did I misunderstand? If I did, I think an MASK to do the above
> > > would be a worthy addition to quickpkg, very useful for
> > > embedded targets to keep the pkg size down.
> > > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > > dropped) would be nice.
> > >
> > > Oh, something else I wonder about. How does pre/post install
> > > work with
> > > quickpkgs? Is it possible to have such scripts and then have
> > > qmerge execute
> > > them?
> > >
> > > Jocke
> > >
> > >
> > > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > > defined in it.
> > > Binary package do contain all files, even the masked files.
> > > Kfir
> >
> >
> > $PKG_INSTALL_MASK is supposed to omit the files in it's list from making
> > it into the binary pkgs in the first place. The idea there was to make
> > smaller binary pkgs for embedded devices and such (for use only with
> > private repos).
> >
> > $INSTALL_MASK is supposed to omit the files listed in it from being
> > installed on the file system.
>
> Hi Solar, long time no see :)
>
> So the current behaviour is a bug, good to know.

I would not really call it a bug. *INSTALL_MASK is a portage feature
itself. quickpkg does not support all the features of portage itself.
You could/should file a feature request bug for quickpkg to add such
support. Maybe file a bug for portage-utils@ as well because I just
checked and looks like we never added support for PKG_INSTALL_MASK in
qpkg.c (c version of quickpkg)


> I am contemplating a greater problem too. Our system require we can install
> multiple versions of our SW and switch between them. This is easy to do when it
> comes to our own app but not when one want to upgrade core parts of the system, like
> libc etc.
>
> So I am thinking one could use --bind mounts and switch_root to solve that. Basically
> one has a skeleton root FS with /bin, /lib, /usr /opt etc.
> Each upgrade goes into dirs like:
> bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
> Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
> and --bind mounts them under the corresponding skeleton dir.
>
> To do that one needs an way to repackage a root fs created by a bunch of different
> ebuilds, some of them spanning several of the above dirs, into a package per xx_1.x.y dir.
> Not sure if this can be done with the current portage/portage-utils and how, any
> ideas welcome :)
>
> Perhaps there is a better way then the above?
>
> Jocke

Not sure about all this.


--
solar <solar@gentoo.org>
Gentoo Linux

solar <solar@gentoo.org> wrote on 2011/12/26 21:38:52:
>
> On Mon, 2011-12-26 at 19:03 +0100, Joakim Tjernlund wrote:
> > solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
> > >
> > > On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> > > >
> > > >
> > > > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > > > <joakim.tjernlund@transmode.se> wrote:
> > > >
> > > > I got the impression from docs that PKG_INSTALL_MASK would
> > > > actually
> > > > mask files out so they never get into the binary package, this
> > > > doesn't
> > > > seem to happen.
> > > > Did I misunderstand? If I did, I think an MASK to do the above
> > > > would be a worthy addition to quickpkg, very useful for
> > > > embedded targets to keep the pkg size down.
> > > > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > > > dropped) would be nice.
> > > >
> > > > Oh, something else I wonder about. How does pre/post install
> > > > work with
> > > > quickpkgs? Is it possible to have such scripts and then have
> > > > qmerge execute
> > > > them?
> > > >
> > > > Jocke
> > > >
> > > >
> > > > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > > > defined in it.
> > > > Binary package do contain all files, even the masked files.
> > > > Kfir
> > >
> > >
> > > $PKG_INSTALL_MASK is supposed to omit the files in it's list from making
> > > it into the binary pkgs in the first place. The idea there was to make
> > > smaller binary pkgs for embedded devices and such (for use only with
> > > private repos).
> > >
> > > $INSTALL_MASK is supposed to omit the files listed in it from being
> > > installed on the file system.
> >
> > Hi Solar, long time no see :)
> >
> > So the current behaviour is a bug, good to know.
>
> I would not really call it a bug. *INSTALL_MASK is a portage feature
> itself. quickpkg does not support all the features of portage itself.
> You could/should file a feature request bug for quickpkg to add such
> support. Maybe file a bug for portage-utils@ as well because I just
> checked and looks like we never added support for PKG_INSTALL_MASK in
> qpkg.c (c version of quickpkg)

I don't really get the difference between the two as impl. today.
What can PKG_INSTALL_MASK do that INSTALL_MASK can't(or vice versa)?

>
>
> > I am contemplating a greater problem too. Our system require we can install
> > multiple versions of our SW and switch between them. This is easy to do when it
> > comes to our own app but not when one want to upgrade core parts of the system, like
> > libc etc.
> >
> > So I am thinking one could use --bind mounts and switch_root to solve that. Basically
> > one has a skeleton root FS with /bin, /lib, /usr /opt etc.
> > Each upgrade goes into dirs like:
> > bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
> > Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
> > and --bind mounts them under the corresponding skeleton dir.
> >
> > To do that one needs an way to repackage a root fs created by a bunch of different
> > ebuilds, some of them spanning several of the above dirs, into a package per xx_1.x.y dir.
> > Not sure if this can be done with the current portage/portage-utils and how, any
> > ideas welcome :)
> >
> > Perhaps there is a better way then the above?
> >
> > Jocke
>
> Not sure about all this.

Yeah, it is a bit much :) I either case it would be nice if one
could group several bianry pkgs into one that qmerge can install. Is that possible?

Jocke

On Tue, Dec 27, 2011 at 11:40 AM, Joakim Tjernlund <
joakim.tjernlund@transmode.se> wrote:

> solar <solar@gentoo.org> wrote on 2011/12/26 21:38:52:
> >
> > On Mon, 2011-12-26 at 19:03 +0100, Joakim Tjernlund wrote:
> > > solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
> > > >
> > > > On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> > > > >
> > > > >
> > > > > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > > > > <joakim.tjernlund@transmode.se> wrote:
> > > > >
> > > > > I got the impression from docs that PKG_INSTALL_MASK would
> > > > > actually
> > > > > mask files out so they never get into the binary package,
> this
> > > > > doesn't
> > > > > seem to happen.
> > > > > Did I misunderstand? If I did, I think an MASK to do the
> above
> > > > > would be a worthy addition to quickpkg, very useful for
> > > > > embedded targets to keep the pkg size down.
> > > > > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > > > > dropped) would be nice.
> > > > >
> > > > > Oh, something else I wonder about. How does pre/post
> install
> > > > > work with
> > > > > quickpkgs? Is it possible to have such scripts and then
> have
> > > > > qmerge execute
> > > > > them?
> > > > >
> > > > > Jocke
> > > > >
> > > > >
> > > > > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > > > > defined in it.
> > > > > Binary package do contain all files, even the masked files.
> > > > > Kfir
> > > >
> > > >
> > > > $PKG_INSTALL_MASK is supposed to omit the files in it's list from
> making
> > > > it into the binary pkgs in the first place. The idea there was to
> make
> > > > smaller binary pkgs for embedded devices and such (for use only with
> > > > private repos).
> > > >
> > > > $INSTALL_MASK is supposed to omit the files listed in it from being
> > > > installed on the file system.
> > >
> > > Hi Solar, long time no see :)
> > >
> > > So the current behaviour is a bug, good to know.
> >
> > I would not really call it a bug. *INSTALL_MASK is a portage feature
> > itself. quickpkg does not support all the features of portage itself.
> > You could/should file a feature request bug for quickpkg to add such
> > support. Maybe file a bug for portage-utils@ as well because I just
> > checked and looks like we never added support for PKG_INSTALL_MASK in
> > qpkg.c (c version of quickpkg)
>
> I don't really get the difference between the two as impl. today.
> What can PKG_INSTALL_MASK do that INSTALL_MASK can't(or vice versa)?
>
> What I undersood is that PKG_INSTALL_MASK will make smaller binary tbz
file, and
INSTALL_MASK will at install time, mask files, so the tbz will have all
files in it.

Kfir

> >
> >
> > > I am contemplating a greater problem too. Our system require we can
> install
> > > multiple versions of our SW and switch between them. This is easy to
> do when it
> > > comes to our own app but not when one want to upgrade core parts of
> the system, like
> > > libc etc.
> > >
> > > So I am thinking one could use --bind mounts and switch_root to solve
> that. Basically
> > > one has a skeleton root FS with /bin, /lib, /usr /opt etc.
> > > Each upgrade goes into dirs like:
> > > bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
> > > Then, from an initramfs, one selects which of xxx_1.x.y dir one wants
> to use
> > > and --bind mounts them under the corresponding skeleton dir.
> > >
> > > To do that one needs an way to repackage a root fs created by a bunch
> of different
> > > ebuilds, some of them spanning several of the above dirs, into a
> package per xx_1.x.y dir.
> > > Not sure if this can be done with the current portage/portage-utils
> and how, any
> > > ideas welcome :)
> > >
> > > Perhaps there is a better way then the above?
> > >
> > > Jocke
> >
> > Not sure about all this.
>
> Yeah, it is a bit much :) I either case it would be nice if one
> could group several bianry pkgs into one that qmerge can install. Is that
> possible?
>
> Jocke
>
>
>

solar <solar@gentoo.org> wrote on 2011/12/26 21:38:52:
>
> On Mon, 2011-12-26 at 19:03 +0100, Joakim Tjernlund wrote:
> > solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
> > >
> > > On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> > > >
> > > >
> > > > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > > > <joakim.tjernlund@transmode.se> wrote:
> > > >
> > > > I got the impression from docs that PKG_INSTALL_MASK would
> > > > actually
> > > > mask files out so they never get into the binary package, this
> > > > doesn't
> > > > seem to happen.
> > > > Did I misunderstand? If I did, I think an MASK to do the above
> > > > would be a worthy addition to quickpkg, very useful for
> > > > embedded targets to keep the pkg size down.
> > > > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > > > dropped) would be nice.
> > > >
> > > > Oh, something else I wonder about. How does pre/post install
> > > > work with
> > > > quickpkgs? Is it possible to have such scripts and then have
> > > > qmerge execute
> > > > them?
> > > >
> > > > Jocke
> > > >
> > > >
> > > > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > > > defined in it.
> > > > Binary package do contain all files, even the masked files.
> > > > Kfir
> > >
> > >
> > > $PKG_INSTALL_MASK is supposed to omit the files in it's list from making
> > > it into the binary pkgs in the first place. The idea there was to make
> > > smaller binary pkgs for embedded devices and such (for use only with
> > > private repos).
> > >
> > > $INSTALL_MASK is supposed to omit the files listed in it from being
> > > installed on the file system.
> >
> > Hi Solar, long time no see :)
> >
> > So the current behaviour is a bug, good to know.
>
> I would not really call it a bug. *INSTALL_MASK is a portage feature
> itself. quickpkg does not support all the features of portage itself.
> You could/should file a feature request bug for quickpkg to add such
> support. Maybe file a bug for portage-utils@ as well because I just
> checked and looks like we never added support for PKG_INSTALL_MASK in
> qpkg.c (c version of quickpkg)

I did file a bug, got some answer too. See bug 396315

Jocke

On 26/12/2011 18:03, Joakim Tjernlund wrote:
> I am contemplating a greater problem too. Our system require we can install
> multiple versions of our SW and switch between them. This is easy to do when it
> comes to our own app but not when one want to upgrade core parts of the system, like
> libc etc.
>
> So I am thinking one could use --bind mounts and switch_root to solve that. Basically
> one has a skeleton root FS with /bin, /lib, /usr /opt etc.
> Each upgrade goes into dirs like:
> bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
> Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
> and --bind mounts them under the corresponding skeleton dir.
>

I'm doing something like this using aufs. The performance seems "not
bad", but you get a couple MB or so memory hit. (I'm using squashfs as
well, so unsure which causes the main memory increase).

I think if the device is reasonably beefy then you should have no
problem with aufs or some similar overlay filesystem (I think some new
one got added to kernel recently?). Failing that, what about good old
"cp -a" to merge all the input dirs? Takes some time and disk space,
but nice and simple?

Good luck

Ed W

Ed W <lists@wildgooses.com> wrote on 2012/01/23 19:43:49:

> From: Ed W <lists@wildgooses.com>
> To: gentoo-embedded@lists.gentoo.org
> Date: 2012/01/23 19:44
> Subject: Re: [gentoo-embedded] quickpkg and PKG_INSTALL_MASK
>
> On 26/12/2011 18:03, Joakim Tjernlund wrote:
> > I am contemplating a greater problem too. Our system require we can install
> > multiple versions of our SW and switch between them. This is easy to do when it
> > comes to our own app but not when one want to upgrade core parts of the system, like
> > libc etc.
> >
> > So I am thinking one could use --bind mounts and switch_root to solve that. Basically
> > one has a skeleton root FS with /bin, /lib, /usr /opt etc.
> > Each upgrade goes into dirs like:
> > bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
> > Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
> > and --bind mounts them under the corresponding skeleton dir.
> >
>
> I'm doing something like this using aufs. The performance seems "not
> bad", but you get a couple MB or so memory hit. (I'm using squashfs as
> well, so unsure which causes the main memory increase).

hmm, not sure how aufs would work out. One would like to permanently delete the
old sw at some point and I don't see how, perhaps I misunderstand something?

How does aufs do w.r.t power cuts? We need a reliable FS w.r.t power cuts. In the
past we have used JFFS2 on NOR flash and that has worked really well. Now
we need to switch to NAND due to size requirements.

>
> I think if the device is reasonably beefy then you should have no
> problem with aufs or some similar overlay filesystem (I think some new
> one got added to kernel recently?). Failing that, what about good old
> "cp -a" to merge all the input dirs? Takes some time and disk space,
> but nice and simple?

Not sure I get the "cp -a" idea. You are suggesting to create a new root FS by
copying the non changed dirs from the previous version and combine these with
the new SW?

Jocke

* Joakim Tjernlund <joakim.tjernlund@transmode.se> [120124 02:18]:
> Ed W <lists@wildgooses.com> wrote on 2012/01/23 19:43:49:
[ .. ]
> > I'm doing something like this using aufs. The performance seems "not
> > bad", but you get a couple MB or so memory hit. (I'm using squashfs as
> > well, so unsure which causes the main memory increase).
>
> hmm, not sure how aufs would work out. One would like to permanently delete the

aufs is A Union FileSystem. You can have a RO "branch" (say SquashFS)
and a r/w "branch" (say JFFS2 or UBIFS.) When you write it will write
to the r/w branch and when you read it will read from the r/w branch if
it exists, else the r/o.

So, if you want to permanently delete the old software. Write it to the
filesystem r/w filesystem (initially or after you have the aufs mounts
mounted.) Then you can delete from the r/w filesystem and it's gone for
good.

> old sw at some point and I don't see how, perhaps I misunderstand something?
>
> How does aufs do w.r.t power cuts? We need a reliable FS w.r.t power cuts. In the
> past we have used JFFS2 on NOR flash and that has worked really well. Now
> we need to switch to NAND due to size requirements.

You still want to pick a r/w branch with a filesystem that handles power
cuts well. You can continue to use JFFS2.

[..]

Todd

On 24/01/2012 14:06, Todd Goodman wrote:
> * Joakim Tjernlund<joakim.tjernlund@transmode.se> [120124 02:18]:
>> Ed W<lists@wildgooses.com> wrote on 2012/01/23 19:43:49:
> [ .. ]
>>> I'm doing something like this using aufs. The performance seems "not
>>> bad", but you get a couple MB or so memory hit. (I'm using squashfs as
>>> well, so unsure which causes the main memory increase).
>> hmm, not sure how aufs would work out. One would like to permanently delete the
> aufs is A Union FileSystem. You can have a RO "branch" (say SquashFS)
> and a r/w "branch" (say JFFS2 or UBIFS.) When you write it will write
> to the r/w branch and when you read it will read from the r/w branch if
> it exists, else the r/o.
>
> So, if you want to permanently delete the old software. Write it to the
> filesystem r/w filesystem (initially or after you have the aufs mounts
> mounted.) Then you can delete from the r/w filesystem and it's gone for
> good.

Agreed. Just to augment that explanation:

- The RO branch would be at the bottom (can be multiple of these)
- The RW branch goes over the top (*can* be multiple of these, but one
is more normal)
- You can effectively delete stuff from the RO branches because Aufs has
the concept of "white out" files. So for all intents and purposes the
top RW layer can create any end result you like, including that of
completely masking out some lower layer
- With some knowledge of how the whiteout files work you can also "undo"
changes to the RO files. Eg directly mounting the RW layer and erasing
all files (from the RW layer) leaves you back with just the stacked RO
layers again. I find this helpful for development where I can basically
work live on the last released build and then by inspection the RW layer
has all the changes needed to apply to the next RO layer release!

I believe recent kernels also have a much simpler "Overlay Filesystem"
that has fewer features. Also the big alternative to aufs is Unionfs 2
- most distros use aufs, but both seem viable?


Finally note that you don't need to use aufs for the entire filesystem.
A common setup might be to use a bunch of bind mounts where you know you
don't need overlay features, eg /usr might be a bunch of overlays, /home
might be a bind mount to writeable storage, /var might be a ram drive
which is initialised from some fixed template, etc? In my case I have
an overlay over most things, but /usr/lib/modules is a bind mount to a
RO filesystem (you can't write to it), /home is mounted to my writeable
storage (not layered), the main OS dirs are layered and /var is a mess...

> You still want to pick a r/w branch with a filesystem that handles power
> cuts well. You can continue to use JFFS2.

Thought: Is there any evidence that one modern filesystem is better than
another with regards to sudden power removal? You probably need to
speak to filesystem experts at this point and define the kind of thing
you are trying to protect against? Sounds like you have raw flash
storage here, so that constrains your choices somewhat?

Just note that with aufs you can use quite a few filesystems for the
different layers. So for example you could have a base RW layer which
is a DM snapshot, overlaid with a loopback mount to a DVD iso, overlay
that with a squashfs, and finally overlay an Ext4 RW mount... (And of
course each of the RO layers might be stored on varied filesystems
themselves - check distributions such as Slax which allow you to overlay
a squashfs that is itself inside some loopback mounted file...)
http://aufs.sourceforge.net/aufs2/report/sq/

http://aufs.sourceforge.net/

I believe for most cheapo consumer flash storage where the underlying
flash filesystem isn't exposed, its quite susceptable to *complete*
failure with sudden poweroff? The issue is the invisible, underlying
flash filesystem gets corrupted during a partial write and that can be
the end of your flash drive - you don't even get to see it again to
recover from it... I don't believe partitioning protects you from this,
but of course separating read/write concerns to physically separate
devices would help? I presume this isn't what you are using though?


Good luck

Ed W

Ed W <lists@wildgooses.com> wrote on 2012/01/25 14:21:25:

>
> On 24/01/2012 14:06, Todd Goodman wrote:
> > * Joakim Tjernlund<joakim.tjernlund@transmode.se> [120124 02:18]:
> >> Ed W<lists@wildgooses.com> wrote on 2012/01/23 19:43:49:
> > [ .. ]
> >>> I'm doing something like this using aufs. The performance seems "not
> >>> bad", but you get a couple MB or so memory hit. (I'm using squashfs as
> >>> well, so unsure which causes the main memory increase).
> >> hmm, not sure how aufs would work out. One would like to permanently delete the
> > aufs is A Union FileSystem. You can have a RO "branch" (say SquashFS)
> > and a r/w "branch" (say JFFS2 or UBIFS.) When you write it will write
> > to the r/w branch and when you read it will read from the r/w branch if
> > it exists, else the r/o.
> >
> > So, if you want to permanently delete the old software. Write it to the
> > filesystem r/w filesystem (initially or after you have the aufs mounts
> > mounted.) Then you can delete from the r/w filesystem and it's gone for
> > good.
>
> Agreed. Just to augment that explanation:

Sorry for the late reply, got sidetracked with high prio stuff.

>
> - The RO branch would be at the bottom (can be multiple of these)
> - The RW branch goes over the top (*can* be multiple of these, but one
> is more normal)
> - You can effectively delete stuff from the RO branches because Aufs has
> the concept of "white out" files. So for all intents and purposes the
> top RW layer can create any end result you like, including that of
> completely masking out some lower layer
> - With some knowledge of how the whiteout files work you can also "undo"
> changes to the RO files. Eg directly mounting the RW layer and erasing
> all files (from the RW layer) leaves you back with just the stacked RO
> layers again. I find this helpful for development where I can basically
> work live on the last released build and then by inspection the RW layer
> has all the changes needed to apply to the next RO layer release!

When you update the RO layer you will be back to a single copy, the previous
release which is only in the overlay is gone, right?

It also implies 2 different SW update methods, one for updating the overlay
copy and one for updating the RO layer.

For development it works well though.

>
> I believe recent kernels also have a much simpler "Overlay Filesystem"
> that has fewer features. Also the big alternative to aufs is Unionfs 2
> - most distros use aufs, but both seem viable?

Dunno, I haven't tried any overlay FS yet.

>
>
> Finally note that you don't need to use aufs for the entire filesystem.
> A common setup might be to use a bunch of bind mounts where you know you
> don't need overlay features, eg /usr might be a bunch of overlays, /home
> might be a bind mount to writeable storage, /var might be a ram drive
> which is initialised from some fixed template, etc? In my case I have
> an overlay over most things, but /usr/lib/modules is a bind mount to a
> RO filesystem (you can't write to it), /home is mounted to my writeable
> storage (not layered), the main OS dirs are layered and /var is a mess...
>
> > You still want to pick a r/w branch with a filesystem that handles power
> > cuts well. You can continue to use JFFS2.
>
> Thought: Is there any evidence that one modern filesystem is better than
> another with regards to sudden power removal? You probably need to
> speak to filesystem experts at this point and define the kind of thing
> you are trying to protect against? Sounds like you have raw flash
> storage here, so that constrains your choices somewhat?

AFAIK, only FSes which are designed for flash(JFFS2, UBIFS, YAFFS etc. )
are safe w.r.t power cuts.

>
> Just note that with aufs you can use quite a few filesystems for the
> different layers. So for example you could have a base RW layer which
> is a DM snapshot, overlaid with a loopback mount to a DVD iso, overlay
> that with a squashfs, and finally overlay an Ext4 RW mount... (And of
> course each of the RO layers might be stored on varied filesystems
> themselves - check distributions such as Slax which allow you to overlay
> a squashfs that is itself inside some loopback mounted file...)

hmm, DM snapshots might be something. Have to look closer on that.

> http://aufs.sourceforge.net/aufs2/report/sq/
>
> http://aufs.sourceforge.net/
>
> I believe for most cheapo consumer flash storage where the underlying
> flash filesystem isn't exposed, its quite susceptable to *complete*
> failure with sudden poweroff? The issue is the invisible, underlying
> flash filesystem gets corrupted during a partial write and that can be
> the end of your flash drive - you don't even get to see it again to
> recover from it... I don't believe partitioning protects you from this,
> but of course separating read/write concerns to physically separate
> devices would help? I presume this isn't what you are using though?

We are not using NAND flash yet but our next product will. I do have the
impression that any block emulating device such as SSD are unreliable
w.r.t power cuts. I would love to be proven wrong though :)

Jocke

* Joakim Tjernlund <joakim.tjernlund@transmode.se> [120126 05:38]:
[..]
>
> When you update the RO layer you will be back to a single copy, the previous
> release which is only in the overlay is gone, right?

That's up to you. You can manage your updates however you want. For
example, you can have your RO layer loop mounted out of a release image
and have two (or more) different RW areas depending on which image
you're running. That can allow you to fallback from an update fairly
easily.

>
> It also implies 2 different SW update methods, one for updating the overlay
> copy and one for updating the RO layer.

Well, you probably will have an initramfs to set up the aufs. As part
of the initramfs you can do whatever you have to for the update. If you
have things on the RW layer that should be carried over you can leave
them if you mount the same RW filesystem with the new RO layer or you
can copy them to a new RW filesystem if you want them to move forwards
but still want a separate RW filesystem for each image to allow easy
fallback. It's really up to how you want to manage it and how much RW
you need to go forwards with the update (and where it is on the
filesystem too.)

[..]
> AFAIK, only FSes which are designed for flash(JFFS2, UBIFS, YAFFS etc. )
> are safe w.r.t power cuts.

Yes, that's certainly been my experience.

[..]
>
> We are not using NAND flash yet but our next product will. I do have the
> impression that any block emulating device such as SSD are unreliable
> w.r.t power cuts. I would love to be proven wrong though :)
>
> Jocke

I've used NOR flash with aufs and a RO loop mounted root filesystem from
within a release image and UBIFS RW filesystem.

It worked well.

I've used NOR with JFFS2 before as well and had some issues. Those may
have been bugs in the JFFS2 implementation I used though.

Todd

Joakim Tjernlund wrote:
> > > You still want to pick a r/w branch with a filesystem that
> > > handles power cuts well. You can continue to use JFFS2.
> >
> > Thought: Is there any evidence that one modern filesystem is better
> > than another with regards to sudden power removal? You probably
> > need to speak to filesystem experts at this point and define the
> > kind of thing you are trying to protect against? Sounds like you
> > have raw flash storage here, so that constrains your choices
> > somewhat?

In fact, having "raw" flash storage is the *only* way to have *any*
chance of completely reliable handling of power outage.


> AFAIK, only FSes which are designed for flash(JFFS2, UBIFS, YAFFS etc. )
> are safe w.r.t power cuts.

It's not only about the filesystem, it is also critical that the
kernel is actually in charge of writing to flash. This means that
if you are not using an mtd based device, you can never have perfect
reliability, regardless of what your filesystem promises.


> We are not using NAND flash yet but our next product will.

NAND or NOR is not important - mtd block device is.


> I do have the impression that any block emulating device such as
> SSD are unreliable w.r.t power cuts. I would love to be proven
> wrong though :)

Supposedly, Intel SSDs never report write completed before flash has
actually been updated. Good luck verifying that.

Everyone else buffers writes. The Samsung 830 series has 256MB of
SDRAM, and their multicore ARM controller handles the magic
buffering. Samsung 830 is built entirely by Samsung from Samsung
parts. All manufacturers besides Intel and Samsung use a mishmash
of components and firmware in their SSD products, and all do magic
buffering.


//Peter

On 26/01/2012 13:30, Todd Goodman wrote:
> * Joakim Tjernlund<joakim.tjernlund@transmode.se> [120126 05:38]:
> [..]
>> When you update the RO layer you will be back to a single copy, the previous
>> release which is only in the overlay is gone, right?
> That's up to you. You can manage your updates however you want. For
> example, you can have your RO layer loop mounted out of a release image
> and have two (or more) different RW areas depending on which image
> you're running. That can allow you to fallback from an update fairly
> easily.

Indeed.

So for example I am trying something a bit like the Slax distro:
- I produce a base distro that will hopefully change very slowly. This
has the minimal software for any other base (uclibc, busybox, few tools ...)
- I produce a faster moving update that has real software on it (eg I
need Dovecot/Postfix, some other stuff)
- I have a final overlay with "my" software on, ie integration stuff
(this is the web apps, very specific stuff to this build, etc. note
this may turn out to be a wrong move - considering combining this with
the step above...)
- I bind mount some other stuff over the top of this, eg my kernel
/lib/modules is in a squashfs and bind mounted into this. Therefore I
can update the kernel separately to all of the above.


The goal of this chain was to minimise the *size* of updates. Your goal
might be different and hence have fewer/more layers. I also considered
the possibility of "shims" between the faster moving layers above. The
motivation was to be able to do little micro updates without rebuilding
a whole layer (my layers are perhaps 10MB+ and the users have expensive
and slow bandwidth, so being able to ship a 50KB shim to fix something
is better than a 10MB module - not convinced yet, but sharing the idea...)

Don't feel this is right for you. Just highlighting how you could use
an overlay filesystem. Also see the Slax distribution for an example
which takes this to extremes (software installs are just overlay modules
to a basic core system - very clever)

Good luck

Ed W

On 26/01/2012 14:59, Peter Stuge wrote:
>> I do have the impression that any block emulating device such as
>> SSD are unreliable w.r.t power cuts. I would love to be proven
>> wrong though :)
> Supposedly, Intel SSDs never report write completed before flash has
> actually been updated. Good luck verifying that.
>
> Everyone else buffers writes. The Samsung 830 series has 256MB of
> SDRAM, and their multicore ARM controller handles the magic
> buffering. Samsung 830 is built entirely by Samsung from Samsung
> parts. All manufacturers besides Intel and Samsung use a mishmash
> of components and firmware in their SSD products, and all do magic
> buffering.

I can't find a reference right now, but I am under the impression that
the more expensive "enterprise" SSD modules now have some battery backup
onboard to allow writes to complete in sudden power down situations? I
read recently of one consumer report (probably on Anand tech, but can't
find it now...) which had also been fitted with some kind of supercap to
handle unexpected powerloss

(Update, google "SSD supercap" - seems like some of the higher end
Sandforce boards have such a thing, and on skimming it is implied that
the Vertex3 might have such a thing?)

I also found this paper which seemed interesting:
http://cseweb.ucsd.edu/users/swanson/papers/DAC2011PowerCut.pdf


Seems like it's a rocky road ahead...

Cheers

Ed W


P.S. I think this phenomena explains what surely everyone here has
experienced, that if you rip out a USB flash drive while it's still
writing then every so often the whole drive just disappears (and stays
dead)? Also I have killed one CF almost certainly due to a sudden
reboot (but it was the design that has since been recalled by
PCEngines), and a few others that on reflection might have had a sudden
powerdown in a camera or while removing pictures from the camera flash
card... Read random amazon reviews of flash drives and there are always
a bunch of people who post when their drive totally fails while it's new
- again my bet would be many of these would be explained by sudden
unplugs during write...