Mailing List Archive

Handbook btrfs support
I was looking at the handbook. File system support seems limited in the
documentation of the handbook, some quite antiquated (reiserfs?).

Looking at the handbook it would seem to be quite extensible to support
one of the newer, extremely attractive, files systems (ZFS or btrfs).
My guess is that, license_wise, it would be easier to document the
use of btrfs in the handbook?

So, these are the sections that would need only limited prose to support
btrfs in the handbook:

2.b
4.b/4.d 4.e
6.a 6.b
7.b
8.a
9.e
10.e ?

The idea would be to use a single 2T drive, with the standard
(handbook) 3 partition scheme, with just the simplest gentoo system
set up with BTRFS.

Your comments and suggestions are most welcome. If this idea is deemed
meritorious, I'll take a crack at it and post what I discover after
performing a few of this proposed btrfs handbook installs?


your thoughts?
James
Re: Handbook btrfs support [ In reply to ]
wireless posted on Fri, 06 Dec 2013 14:31:52 -0500 as excerpted:

> I was looking at the handbook. File system support seems limited in the
> documentation of the handbook, some quite antiquated (reiserfs?).
>
> Looking at the handbook it would seem to be quite extensible to support
> one of the newer, extremely attractive, files systems (ZFS or btrfs). My
> guess is that, license_wise, it would be easier to document the use of
> btrfs in the handbook?
>
> So, these are the sections that would need only limited prose to support
> btrfs in the handbook:
>
> 2.b 4.b/4.d 4.e 6.a 6.b 7.b 8.a 9.e 10.e ?
>
> The idea would be to use a single 2T drive, with the standard (handbook)
> 3 partition scheme, with just the simplest gentoo system set up with
> BTRFS.
>
> Your comments and suggestions are most welcome. If this idea is deemed
> meritorious, I'll take a crack at it and post what I discover after
> performing a few of this proposed btrfs handbook installs?

As you'll know if you follow the btrfs list, have read the wiki, or even
simply the btrfs kernel config option, btrfs isn't exactly stable yet.
While a patch in 3.13 does tone down the warning a bit, matching a
slightly toned down warning on the wiki of late, and I /do/ run btrfs
here, it's still far enough from stable that I'd hesitate to put it in
the handbook just yet.

Basically, current btrfs status for single-device or multi-device
raid0/1/10 modes is "semi-stable"; it generally works reasonably well,
but be *SURE* and keep *TESTED* backups and be prepared to use them if
necessary, AND try to keep on latest (upstream Linus) stable kernel if
not the development kernel RCs. Following current discussions and status
on the btrfs list is recommended as well.

And bugs do still often affect a reasonable cross-section of users, too.
One fixed in 3.12 was hitting a lot of systemd users as well as others
using pre-allocated files that are then written into (torrent clients
often do this too and someone reported getting hit by that, too). For
3.13, a host of kernel memory leaks have been fixed, as well as a
concurrency bug hit when people tried to run a balance and a snapshot (as
often done via cronjob, so the admin may have only been thinking about
and run the balance manually) at the same time.

Until very recently, live-git master-branch btrfs-progs was strongly
recommended too (development happens in branches and the policy for
master is that it's always run-ready, as stable as btrfs itself is at
this point), but with kernel 3.12 the btrfs-progs versioning policy
changed, with releases now generally synced with the kernel and versioned
similarly, thus making btrfs-progs-3.12 as current as the 3.12 series
kernel. Of course that's ~arch, while live-git 9999 is naturally masked.

Run older than that as a btrfs tester (which is what anyone running btrfs
is at this point, a tester) and you're not only needlessly risking having
to use those backups you're keeping as a not-yet-fully-stable filesystem
tester due to running code with known and now fixed bugs, but any testing
reports filed aren't going to be as useful either, because you're testing
old code on a fast-moving project that has moved on from it.

But a lot of gentoo users reading the handbook won't want to be upgrading
that fast nor will they be that faithful in keeping current and tested
backups, nor will they wish to bother with following the btrfs list, as
recommended for anyone wishing to test btrfs at this point.

Oh, and neither btrfs nor zfs mount options are listed in the mount
manpage yet. I don't know about zfs, but one indication of btrfs
maturing will be when it appears in the mount manpage.

So IMO btrfs is inappropriate for the handbook at this point. Or if it's
included, stress its testing aspect and that those choosing to test it
should be prepared to use their TESTED backups, keep current on the
kernel and btrfs-progs, and follow the btrfs list to keep up with current
issues with what they've chosen to test.

Tho the thinking is that btrfs should really start to stabilize in 2014,
at least for the "basic" functionality comparable to most filesystems.
Of course in some ways every year seems to be the year btrfs will
stabilize, rather reminding me of the year of the Linux desktop.
However, I know it's getting closer, as I'm actually running it this
year, something I tried but gave up on last year. And after leaving it
to mature a bit after the first bit, I could DEFINITELY see the
difference when I came back. It really is getting there, and 2014 could
very well be the year.

Meanwhile, zfs is a bit more mature, but as you point out, has licensing
issues, thus making it not particularly handbook appropriate, tho of
course people can choose to run it if they wish.

Thus at this point, ext4 really remains the best "mainline" choice, with
xfs also a reasonable choice these days. And reiserfs certainly remains
usable. I'm still using it on spinning rust here, tho it's not so good
for SSDs, which is where I'm testing btrfs. But ext4 has an ssd mode as
well, I believe.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
Re: Re: Handbook btrfs support [ In reply to ]
On 12/06/13 16:23, Duncan wrote:
> wireless posted on Fri, 06 Dec 2013 14:31:52 -0500 as excerpted:

>> btrfs in the handbook?
>>
>> So, these are the sections that would need only limited prose to support
>> btrfs in the handbook:
>>
>> 2.b 4.b/4.d 4.e 6.a 6.b 7.b 8.a 9.e 10.e ?
>>
>> The idea would be to use a single 2T drive, with the standard (handbook)
>> 3 partition scheme, with just the simplest gentoo system set up with
>> BTRFS.

What I'm thinking is just "cut and paste" the handbook somewhere
and clearly mark the sections where the handbook has been modified
(deviated from) for this btrfs experiment.


>> Your comments and suggestions are most welcome. If this idea is deemed
>> meritorious, I'll take a crack at it and post what I discover after
>> performing a few of this proposed btrfs handbook installs?
>
> As you'll know if you follow the btrfs list, have read the wiki, or even
> simply the btrfs kernel config option, btrfs isn't exactly stable yet.
> While a patch in 3.13 does tone down the warning a bit, matching a
> slightly toned down warning on the wiki of late, and I /do/ run btrfs
> here, it's still far enough from stable that I'd hesitate to put it in
> the handbook just yet.

Yes, I have a hack of btrfs running on old hardware. I'm going to
hack out a handbook_similar doc and try to explicitly follow it
noting in detail any discussion or other reference sources where
warranted.


> Basically, current btrfs status for single-device or multi-device
> raid0/1/10 modes is "semi-stable"; it generally works reasonably well,
> but be *SURE* and keep *TESTED* backups and be prepared to use them if
> necessary, AND try to keep on latest (upstream Linus) stable kernel if
> not the development kernel RCs. Following current discussions and status
> on the btrfs list is recommended as well.
>
> And bugs do still often affect a reasonable cross-section of users, too.
> One fixed in 3.12 was hitting a lot of systemd users as well as others
> using pre-allocated files that are then written into (torrent clients
> often do this too and someone reported getting hit by that, too). For
> 3.13, a host of kernel memory leaks have been fixed, as well as a
> concurrency bug hit when people tried to run a balance and a snapshot (as
> often done via cronjob, so the admin may have only been thinking about
> and run the balance manually) at the same time.
>
> Until very recently, live-git master-branch btrfs-progs was strongly
> recommended too (development happens in branches and the policy for
> master is that it's always run-ready, as stable as btrfs itself is at
> this point), but with kernel 3.12 the btrfs-progs versioning policy
> changed, with releases now generally synced with the kernel and versioned
> similarly, thus making btrfs-progs-3.12 as current as the 3.12 series
> kernel. Of course that's ~arch, while live-git 9999 is naturally masked.
>
> Run older than that as a btrfs tester (which is what anyone running btrfs
> is at this point, a tester) and you're not only needlessly risking having
> to use those backups you're keeping as a not-yet-fully-stable filesystem
> tester due to running code with known and now fixed bugs, but any testing
> reports filed aren't going to be as useful either, because you're testing
> old code on a fast-moving project that has moved on from it.

Agreed with all of the above.

> But a lot of gentoo users reading the handbook won't want to be upgrading
> that fast nor will they be that faithful in keeping current and tested
> backups, nor will they wish to bother with following the btrfs list, as
> recommended for anyone wishing to test btrfs at this point.

Hmmmmm,

I understand your concerns. The goals is to do this now, to flesh out,
the issues, so when btrfs becomes "stable" there is a simple guide
for adventurous, but ordinary users to follow. Also, a dual hope that
supporting btrfs, formally in the handbook, become a reality.

I think most folks are ready for a file system beyond ext4, particularly
for a second (gentoo) system. Some distros are actively supporting btrfs.



> Oh, and neither btrfs nor zfs mount options are listed in the mount
> manpage yet. I don't know about zfs, but one indication of btrfs
> maturing will be when it appears in the mount manpage.

I disagree. When btrfs is found in SystemRescue and many, many places
on the net, it became serious. Waiting for something to appear properly
in the manpages, is a very odd semantic as an acceptable standard for
use, imho:
http://www.phoronix.com/scan.php?page=news_item&px=MTQ2NTY

> So IMO btrfs is inappropriate for the handbook at this point. Or if it's
> included, stress its testing aspect and that those choosing to test it
> should be prepared to use their TESTED backups, keep current on the
> kernel and btrfs-progs, and follow the btrfs list to keep up with current
> issues with what they've chosen to test.

YES, it's about being ready; not about being a laggard, imho:
https://help.ubuntu.com/community/btrfs

> Tho the thinking is that btrfs should really start to stabilize in 2014,
> at least for the "basic" functionality comparable to most filesystems.
> Of course in some ways every year seems to be the year btrfs will
> stabilize, rather reminding me of the year of the Linux desktop.
> However, I know it's getting closer, as I'm actually running it this
> year, something I tried but gave up on last year. And after leaving it
> to mature a bit after the first bit, I could DEFINITELY see the
> difference when I came back. It really is getting there, and 2014 could
> very well be the year.
>
> Meanwhile, zfs is a bit more mature, but as you point out, has licensing
> issues, thus making it not particularly handbook appropriate, tho of
> course people can choose to run it if they wish.
>
> Thus at this point, ext4 really remains the best "mainline" choice, with
> xfs also a reasonable choice these days. And reiserfs certainly remains
> usable. I'm still using it on spinning rust here, tho it's not so good
> for SSDs, which is where I'm testing btrfs. But ext4 has an ssd mode as
> well, I believe.


Nothing in this effort will replace ext4, except most gentoo user, are
by the nature of being gentoo users, way_ready for a new, feature rich
file system. Ext4 is ARCANE, but very stable.

So what I'm thinking is getting some space on wiki.gentoo.org for this
effort? I'd do most of the work and have a few_folks look over the
original content enhancements ensuring that my
discoveries/ideas/workarounds are reasonable. I do not wish to imply
that it is any sort of official handbook effort, but having some of the
gentoo-doc folks look in, time to time, would be keen.

One thing I'm not sure of, is which section should it go under?

Project & Community ?

Surely not core?
Re: Handbook btrfs support [ In reply to ]
wireless posted on Fri, 06 Dec 2013 17:12:35 -0500 as excerpted:

> I understand your concerns. The goals is to do this now, to flesh out,
> the issues, so when btrfs becomes "stable" there is a simple guide for
> adventurous, but ordinary users to follow. Also, a dual hope that
> supporting btrfs, formally in the handbook, become a reality.

That's a worthwhile goal, I'll agree. =:^)

And I really do expect btrfs to be calming down (it is already, it's just
not /calm/ yet) next year, such that by this time next year, I believe/
hope my opinion will be different, that it /is/ ready. And having the
proposed changes hammered out and real deployment tested when that
happens is indeed a good thing. Thanks! =:^)

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
Re: Re: Handbook btrfs support [ In reply to ]
On Sat, Dec 07, 2013 at 11:48:52AM +0000, Duncan wrote:
> wireless posted on Fri, 06 Dec 2013 17:12:35 -0500 as excerpted:
>
> > I understand your concerns. The goals is to do this now, to flesh out,
> > the issues, so when btrfs becomes "stable" there is a simple guide for
> > adventurous, but ordinary users to follow. Also, a dual hope that
> > supporting btrfs, formally in the handbook, become a reality.
>
> That's a worthwhile goal, I'll agree. =:^)
>
> And I really do expect btrfs to be calming down (it is already, it's just
> not /calm/ yet) next year, such that by this time next year, I believe/
> hope my opinion will be different, that it /is/ ready. And having the
> proposed changes hammered out and real deployment tested when that
> happens is indeed a good thing. Thanks! =:^)

If I'm not mistaken, RedHat is hoping to include Btrfs as a stable file
system for RHEL7. Although I'm not going to use Btrfs for our databases soon
;-)

Anyway, there is already some documentation on Btrfs on
http://wiki.gentoo.org/wiki/Btrfs so it might make sense to improve that
documentation.

Wkr,
Sven Vermeulen