Mailing List Archive

1 2 3 4 5  View All
Re: NAS and replacing with larger drives [ In reply to ]
Am Wed, Dec 21, 2022 at 05:53:03AM +0000 schrieb Wols Lists:

> On 21/12/2022 02:47, Dale wrote:
> > I think if I can hold out a little while, something really nice is going
> > to come along.  It seems there is a good bit of interest in having a
> > Raspberry Pi NAS that gives really good performance.  I'm talking a NAS
> > that is about the same speed as a internal drive.  Plus the ability to
> > use RAID and such.  I'd like to have a 6 bay with 6 drives setup in
> > pairs for redundancy.  I can't recall what number RAID that is.
> > Basically, if one drive fails, another copy still exists.  Of course,
> > two independent NASs would be better in my opinion.  Still, any of this
> > is progress.
>
> That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 1+0
> is often called 10, but linux-10 is slightly different.

In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
pool. So mirror + pool = mirrorpool, hence the 1+0 ? 10.

> I'd personally be inclined to go for raid-6. That's 4 data drives, 2 parity
> (so you could have an "any two" drive failure and still recover).
> A two-copy 10 or 1+0 is vulnerable to a two-drive failure. A three-copy is
> vulnerable to a three-drive failure.

At first, I had only two drives in my 4-bay NAS, which were of course set up
as a mirror. After a year, when it became full, I bought the second pair of
drives and had long deliberations by then, what to choose. I went for raid-6
(or RaidZ2 in ZFS parlance). With only four disks, it has the same net
capacity as a pair of mirrors, but at the advantage that *any* two drives
may fail, not just two particular ones. A raid of mirrors has performance
benefits over a parity raid, but who cares for a simple Gbit storage device.

With increasing number of disks, a mirror setup is at a disadvantage with
storage efficiency – it’s always 50 % or less, if you mirror over more than
two disks. But with only four disks, that was irrelevant in my case. On the
plus-side, each mirror can have a different physical disk size, so you can
more easily mix’n’match what you got lying around, or do upgrades in smaller
increments.

If I wanted to increase my capacity, I’d have to replace *all* drives with
bigger ones. With a mirror, only the drives in one of the mirrors need
replacing. And the rebuild process would be quicker and less painful, as
each drive will only be read once to rebuild its partner, and there is no
parity calculation involved. In a RAID, each drive is replaced one by one,
and each replacement requires a full read of all drives’ payload. With older
drives, this is cause for some concern whether the disks may survive that.
That’s why, with increasing disk capacities, raid-5 is said to be obsolete.
Because if another drive fails during rebuild, you are officially screwed.

Fun, innit?

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The boss is a human just like everyone else, he just doesn’t know.
Re: NAS and replacing with larger drives [ In reply to ]
Wols Lists wrote:
> On 21/12/2022 02:47, Dale wrote:
>> I think if I can hold out a little while, something really nice is going
>> to come along.  It seems there is a good bit of interest in having a
>> Raspberry Pi NAS that gives really good performance.  I'm talking a NAS
>> that is about the same speed as a internal drive.  Plus the ability to
>> use RAID and such.  I'd like to have a 6 bay with 6 drives setup in
>> pairs for redundancy.  I can't recall what number RAID that is.
>> Basically, if one drive fails, another copy still exists.  Of course,
>> two independent NASs would be better in my opinion.  Still, any of this
>> is progress.
>
> That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note
> that 1+0 is often called 10, but linux-10 is slightly different.
>
> I'd personally be inclined to go for raid-6. That's 4 data drives, 2
> parity (so you could have an "any two" drive failure and still recover).
>
> A two-copy 10 or 1+0 is vulnerable to a two-drive failure. A
> three-copy is vulnerable to a three-drive failure.
>
> In other words, a two-copy raid-10 might be taken out by a failure
> that a raid-6 will survive. A three-copy raid-10 might be taken out by
> a failure that will take out a raid-6. Choose your poison :-)
>
> Cheers,
> Wol
>
> .
>


This is why at some point, I'd like to have two sets of backups.  RAID
or not.  If I alternate between the two or update one weekly and the
other monthly, odds of both being lost for some reason is slim.  Nothing
is ever impossible but not very likely.  I could even keep the monthly
backup, or second backup, in a outbuilding.  That would protect from a
house fire or something.  I have a fire safe but still. 

I'm hoping that this Raspberry thing will work out.  My reasons, I can
upgrade easily, change OS and/or software if I want plus I may can even
expand to a larger number of drives if needed.  Buying a pre-built NAS,
I think I'd be stuck with whatever I bought.  Most likely no path for
upgrade, changing OS and/or software or much of anything else either. 
Basically, it is what it is when I buy it.  Given the info in the video
you shared, I think something good is coming as soon as things get back
to going again. 

Thanks again for the link.  It lead to some other good videos too. 

Dale

:-)  :-)
Re: NAS and replacing with larger drives [ In reply to ]
On 21/12/22 14:19, Frank Steinmetzger wrote:
> Am Wed, Dec 21, 2022 at 05:53:03AM +0000 schrieb Wols Lists:
>
>> On 21/12/2022 02:47, Dale wrote:
>> ...
> In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
> pool. So mirror + pool = mirrorpool, hence the 1+0 ? 10.
>
> ...

I tend to use older drives that have led a hard life - so failure
happens and I have to be prepared for it (by having good backups!)

I have found mirrors to be problematic  - sometimes when one drive
fails, it causes a cascade of fails that includes the data on the
mirror.  With raid-10, its worse (even more fragile). When I eventually
moved away from raid for my main data store it was because of a
catastrophic failure of a bcache ssd fronting one of the mirrors causing
all data to be lost - somewhat self-caused by using bcache to try and
get some more speed out of the system, but as a RAID 10 with 4 HDD
fronted by 4x SSD it should have survived ...  In the end, I realised
that raided data gave me a small speedup with little or no benefit as
regards reliable data storage.  I currently have one linux raid 10 using
4xSSD's that has suffered one SSD abrupt failure and survived - which I
regard as "being lucky".  SSD's are an issue as they usually fail
abruptly without warning whereas spinning rust usually gives some warning.

I've never tried RAID-6 as it was still considered buggy/risky at the time.

No matter what storage system you use, offline backups are better - raid
is NOT a viable backup.

> Fun, innit?
>
YEP!

BillK
Re: NAS and replacing with larger drives [ In reply to ]
On Tue, Dec 20, 2022 at 11:52 PM Dale <rdalek1967@gmail.com> wrote:
<SNIP>
> This is why at some point, I'd like to have two sets of backups. RAID
> or not.

Amazon Snowball? :-) ;-)

MArk
Re: NAS and replacing with larger drives [ In reply to ]
On 21/12/2022 06:19, Frank Steinmetzger wrote:
> Am Wed, Dec 21, 2022 at 05:53:03AM +0000 schrieb Wols Lists:
>
>> On 21/12/2022 02:47, Dale wrote:
>>> I think if I can hold out a little while, something really nice is going
>>> to come along.  It seems there is a good bit of interest in having a
>>> Raspberry Pi NAS that gives really good performance.  I'm talking a NAS
>>> that is about the same speed as a internal drive.  Plus the ability to
>>> use RAID and such.  I'd like to have a 6 bay with 6 drives setup in
>>> pairs for redundancy.  I can't recall what number RAID that is.
>>> Basically, if one drive fails, another copy still exists.  Of course,
>>> two independent NASs would be better in my opinion.  Still, any of this
>>> is progress.
>>
>> That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 1+0
>> is often called 10, but linux-10 is slightly different.
>
> In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
> pool. So mirror + pool = mirrorpool, hence the 1+0 ? 10.

Except raid-10 is not a stripe of mirrors. It's each block is saved to
two different drives. (Or 3, or more, so long as you have more drives
than mirrors.)

Linux will happily give you a 2-copy mirror across 3 drives - 3x6TB
drives will give you 9TB useful storage ...
>
>> I'd personally be inclined to go for raid-6. That's 4 data drives, 2 parity
>> (so you could have an "any two" drive failure and still recover).
>> A two-copy 10 or 1+0 is vulnerable to a two-drive failure. A three-copy is
>> vulnerable to a three-drive failure.
>
> At first, I had only two drives in my 4-bay NAS, which were of course set up
> as a mirror. After a year, when it became full, I bought the second pair of
> drives and had long deliberations by then, what to choose. I went for raid-6
> (or RaidZ2 in ZFS parlance). With only four disks, it has the same net
> capacity as a pair of mirrors, but at the advantage that *any* two drives
> may fail, not just two particular ones. A raid of mirrors has performance
> benefits over a parity raid, but who cares for a simple Gbit storage device.
>
> With increasing number of disks, a mirror setup is at a disadvantage with
> storage efficiency – it’s always 50 % or less, if you mirror over more than
> two disks. But with only four disks, that was irrelevant in my case. On the
> plus-side, each mirror can have a different physical disk size, so you can
> more easily mix’n’match what you got lying around, or do upgrades in smaller
> increments.
>
> If I wanted to increase my capacity, I’d have to replace *all* drives with
> bigger ones. With a mirror, only the drives in one of the mirrors need
> replacing. And the rebuild process would be quicker and less painful, as
> each drive will only be read once to rebuild its partner, and there is no
> parity calculation involved. In a RAID, each drive is replaced one by one,
> and each replacement requires a full read of all drives’ payload.

If you've got a spare SATA connection or whatever, each replacement does
not need a full read of all drives. "mdadm --add /dev/sdx --replace
/dev/sdy". That'll stream sdy on to sdx, and only hammer the other
drives if sdy complains ...

> With older
> drives, this is cause for some concern whether the disks may survive that.
> That’s why, with increasing disk capacities, raid-5 is said to be obsolete.
> Because if another drive fails during rebuild, you are officially screwed.
>
> Fun, innit?
>
They've always said that. Just make sure you don't have multiple drives
from the same batch, then they're less likely statistically to fail at
the same time. I'm running raid-5 over 3TB partitions ...

Cheers,
Wol
Re: NAS and replacing with larger drives [ In reply to ]
Am Wed, Dec 21, 2022 at 08:03:36PM +0000 schrieb Wol:
> On 21/12/2022 06:19, Frank Steinmetzger wrote:
> > Am Wed, Dec 21, 2022 at 05:53:03AM +0000 schrieb Wols Lists:
> >
> > > On 21/12/2022 02:47, Dale wrote:
> > > > I think if I can hold out a little while, something really nice is going
> > > > to come along.  It seems there is a good bit of interest in having a
> > > > Raspberry Pi NAS that gives really good performance.  I'm talking a NAS
> > > > that is about the same speed as a internal drive.  Plus the ability to
> > > > use RAID and such.  I'd like to have a 6 bay with 6 drives setup in
> > > > pairs for redundancy.  I can't recall what number RAID that is.
> > > > Basically, if one drive fails, another copy still exists.  Of course,
> > > > two independent NASs would be better in my opinion.  Still, any of this
> > > > is progress.
> > >
> > > That's called either Raid-10 (linux), or Raid-1+0 (elsewhere). Note that 1+0
> > > is often called 10, but linux-10 is slightly different.
> >

> > In layman’s term, a stripe of mirrors. Raid-1 is the mirror, Raid-0 a (JBOD)
> > pool. So mirror + pool = mirrorpool, hence the 1+0 ? 10.
>
> Except raid-10 is not a stripe of mirrors.
> It's each block is saved to two different drives. (Or 3, or more, so long
> as you have more drives than mirrors.)

Yes? In a mirror setup, all member drives of a mirror have the same content
(at least in ZFS).

Raid 10 distributes its content across several mirrors. This is the cause
for its increased performance. So when one of the mirrors (not single drive,
but a whole set of mirrored drives) fails, the pool is gone.

> Linux will happily give you a 2-copy mirror across 3 drives - 3x6TB drives
> will give you 9TB useful storage ...

I admit, I’ve never head of that. (Though it sounds like raid-5 to me.)

> > If I wanted to increase my capacity, I’d have to replace *all* drives with
> > bigger ones. With a mirror, only the drives in one of the mirrors need
> > replacing. And the rebuild process would be quicker and less painful, as
> > each drive will only be read once to rebuild its partner, and there is no
> > parity calculation involved. In a RAID, each drive is replaced one by one,
> > and each replacement requires a full read of all drives’ payload.
>
> If you've got a spare SATA connection or whatever, each replacement does not
> need a full read of all drives. "mdadm --add /dev/sdx --replace /dev/sdy".
> That'll stream sdy on to sdx, and only hammer the other drives if sdy
> complains ...

Strange that I didn’t think of that, even though it’s a perfectly clear
concept. In ZFS there is also a replace function which would do just that.
Currently I plan on keeping my old drives (who would want to buy them off of
me anyways) and just reorganise them in Z1 over Z2. I’ll just have to move
all data off to temprary external drives.

> > With older
> > drives, this is cause for some concern whether the disks may survive that.
> > That’s why, with increasing disk capacities, raid-5 is said to be obsolete.
> > Because if another drive fails during rebuild, you are officially screwed.
> >
> > Fun, innit?
> >
> They've always said that. Just make sure you don't have multiple drives from
> the same batch, then they're less likely statistically to fail at the same
> time. I'm running raid-5 over 3TB partitions ...

Yeah, I bought my drives from different shops back then for that reason.

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

When the going gets tough, the tough get going.
... and so do I. – Alf
Re: NAS and replacing with larger drives [ In reply to ]
On 21/12/2022 20:40, Frank Steinmetzger wrote:
> Yes? In a mirror setup, all member drives of a mirror have the same content
> (at least in ZFS).
>
> Raid 10 distributes its content across several mirrors. This is the cause
> for its increased performance. So when one of the mirrors (not single drive,
> but a whole set of mirrored drives) fails, the pool is gone.
>
>> Linux will happily give you a 2-copy mirror across 3 drives - 3x6TB drives
>> will give you 9TB useful storage ...

> I admit, I’ve never head of that. (Though it sounds like raid-5 to me.)

Raid 5 has a parity drive (or rather, raid 4 has a parity drive. Raid 5
smears parity across all disks). It does not store duplicate copies.
Raid 10 has duplicate data and no parity.
>
Read up on linux raid-10. It is NOT raid-1+0.

Drive sda sdb sdc

Blocks 1 1 2
2 3 3
4 4 5
5 6 6

etc ...

https://raid.wiki.kernel.org/index.php/What_is_RAID_and_why_should_you_want_it%3F

(Disclaimer - I either wrote or heavily edited it.)

Cheers,
Wol
Re: NAS and replacing with larger drives [ In reply to ]
Dave

On Sun, Dec 18, 2022, 2:54 PM Wol <antlists@youngman.org.uk> wrote:

> On 18/12/2022 18:59, Dale wrote:
> > Since this is local, I just use rsync to do my backups. I did have to
> > change the options a bit. It seems TrueNAS doesn't like some of the
> > permissions or something.
>
> Are you running the rsync daemon on the NAS? I'm probably teaching
> grandma to suck eggs, but that massively reduces the need for network
> traffic.
>
> Cheers,
> Wol
>
>
Re: NAS and replacing with larger drives [ In reply to ]
Need link

Dave

On Tue, Dec 20, 2022, 1:52 AM Dale <rdalek1967@gmail.com> wrote:

> Wols Lists wrote:
> > On 18/12/2022 22:11, Dale wrote:
> >> Wol wrote:
> >>> On 18/12/2022 18:59, Dale wrote:
> >>>> Since this is local, I just use rsync to do my backups. I did have to
> >>>> change the options a bit. It seems TrueNAS doesn't like some of the
> >>>> permissions or something.
> >>>
> >>> Are you running the rsync daemon on the NAS? I'm probably teaching
> >>> grandma to suck eggs, but that massively reduces the need for network
> >>> traffic.
> >>>
> >>> Cheers,
> >>> Wol
> >>>
> >>>
> >>
> >>
> >> I mount the NAS on my Gentoo rig. I mount it under /mnt. Then I run
> >> rsync and copy from the source to the mount point for the NAS. I may
> >> could go the other way but never thought about doing it that way. Kinda
> >> sounds backwards to me but I dunno. ;-)
> >>
> > Sounds to me like you're doing it all wrong either way ...
> >
> > What is *supposed* to happen is that you have the daemon running on
> > one machine and the client on the other - doesn't matter which.
> >
> > Then the client tells the daemon what files are to be copied, THE TWO
> > COMPARE CHECKSUMS, and only the stuff that fails the checksum is
> > copied. So if you're doing an incremental backup, network usage and
> > writes are kept to a minimum.
> >
> > I tell people to an in-place backup if they're running on a snapshot
> > setup, because again it only writes stuff that has actually changed.
> >
> > Cheers,
> > Wol
> >
> >
>
>
> Do you have a link to the proper way to do it? I don't copy to a
> different machine often so my current method may be the problem. Maybe
> the way you mention will work much better, even a little better would be
> nice. ;-)
>
> Dale
>
> :-) :-)
>
>

1 2 3 4 5  View All