Mailing List Archive

On 09/12/2022 00:45, Dale wrote:
> Peter Humphrey wrote:
>> On Friday, 9 December 2022 00:03:29 GMT Dale wrote:
>>
>>> I think back sometimes, I started out with a 30GB hard drive waaaay back
>>> in 2003. I thought I had problems then.
>> Then you won't want to know that I paid extra in 1990 for an 85MB drive in my
>> first PC. No, not GB: MB.
>>
>
> I worked at a puter place in the late 80's.  They had old hard drives
> that were only a few MBs and had 14" platters.  Yes, 14" platters.
> Funny thing is, you could replace the platters in those.  You open the
> drive, replace platter, reassemble drive, turn on fan which had a hefty
> filter on the intake.  Once it ran long enough to have clean air inside,
> spin up the drive and go back to work.
>
> I even remember when 512KBs of ram was a big deal.  I also remember
> having expansion cards that would add a few MBs of ram.  Jeez I'm
> getting old.  o_O  We talk about TBs like they are nothing.  My first
> puter was a old Vic-20.  4Kbs of ram it had.  I played music on that
> thing and freaked my Dad out.  ROFL
>
I remember those things. About 16 MB per platter. I remember my work
buying a 300MB drive (19 platters in a disk pack, the size of a washing
machine) for our multi-user mini that served the entire company with
256KB of ram ...

Cheers,
Wol

Wol wrote:
> On 09/12/2022 01:15, Dale wrote:
>> Given the size of one of the directories I have, it takes two drives, or
>> soon will, and the use of LVM or something similar.  I can't do that as
>> it is now.  I've even wondered if I hooked two eSATA drives up and gave
>> both plenty of time to spin up if LVM would see them both and me be able
>> to use two drives as one that way.  Thing is, I don't know how LVM
>> reacts if the two drives become available at separate times, maybe even
>> many seconds or a minute or so apart.
>
> If you're using LVM to link them together, it will wait until they
> become available. Okay, not quite the same, but I run raid over
> dm-integrity, and it always unnerves me when systemd fires up this job
> and it says "waiting for lvm/home". But the system just sits there
> while dm-integrity checks its drives, makes them available, raid spots
> and loads them, and then the raid is there, lvm spots it, makes
> lvm/home available, and the system is up and running ...

Really?  Oh that just may start something.  I could easily setup two
drives and use LVM on them.  I just didn't know that it would work.  One
added benefit, they are encrypted with cryptsetup which puts everything
on top of LVM.  So, the data isn't available until I type in the
password and then mount it.  It doesn't try to mount automatically or
anything because of that.  Oh, this could be the start of something. 

>
> Cheers,
> Wol
>
>

Dale

:-)  :-) 

On Friday, 9 December 2022 08:27:18 GMT Wol wrote:
> On 09/12/2022 00:45, Dale wrote:
> > I even remember when 512KBs of ram was a big deal. I also remember
> > having expansion cards that would add a few MBs of ram. Jeez I'm
> > getting old. o_O We talk about TBs like they are nothing. My first
> > puter was a old Vic-20. 4Kbs of ram it had. I played music on that
> > thing and freaked my Dad out. ROFL
>
> I remember those things. About 16 MB per platter. I remember my work
> buying a 300MB drive (19 platters in a disk pack, the size of a washing
> machine) for our multi-user mini that served the entire company with
> 256KB of ram ...

Latest in the willy-waving contest: in the 1970s the national grid was
monitored and analysed with a Ferranti Argus 500 machine with 24KB RAM and a
2MB disk. It was common for American visitors to believe that was just driving
the control engineers' displays, and where was the main computer?

24-bit assembler code. Those were the days - some of my very best. No concept
of a file or a file-system.

--
Regards,
Peter.

On Thursday, 8 December 2022 20:44:56 GMT Frank Steinmetzger wrote:
> Am Thu, Dec 08, 2022 at 06:36:14PM +0000 schrieb Wols Lists:
> > > > I've pretty much reached a limit on my backups. I'm up to a 16TB
> > > > hard
> > > > drive for one and even that won't last long. Larger drives are much
> > > > more costly. A must have NAS is quickly approaching. I've been
> > > > searching around and find some things confusing. I'm hoping someone
> > > > can
> > > > clear up that confusion. I'm also debating what path to travel down.
> > > > I'd also like to keep costs down as well. That said, I don't mind
> > > > paying a little more for one that would offer a much better option.
> > > >
> > > > Path one, buy a NAS, possibly used, that has no drives. If possible,
> > > > I
> > > > may even replace the OS that comes on it or upgrade if I can. I'm
> > > > not
> > > > looking for fancy, or even RAID. Just looking for a two bay NAS that
> > > > will work. First, what is a DAS? Is that totally different than a
> > > > NAS? From what I've found, a DAS is not what I'm looking for since I
> > > > want a ethernet connection and the ability to control things over the
> > > > network. It seems DAS lacks that feature but not real sure. I'm not
> > > > sure I can upgrade the software/OS on a DAS either.
> > > > […]
> > >
> > > DAS is direct-attached-storage. I don't think you want that.
> >
> > Depends. If it fits in the safe, and can be connected using one of these
> > eSATA thingy connectors, it might be a very good choice.
> >
> > […]
> >
> > I get the impression Dale isn't actually PLANNING his disk storage. It's
> > just a case of "help I'm downloading all this stuff where do I put it!!!"
>
> Haha, thanks for the laugh.

Actually this had me thinking what is the need to back up the ... Internet?
If all this never-ending and recently accelerated download activity by Dale
will continue and most of these video/audio files are available on some
streaming server on the Internet, WHY do they need to be backed up locally?

I appreciate some of these video files may be rare finds, or there may be a
risk some of these may be taken off the interwebs sooner or later. This
should leave a rather small subset of all downloads, which may merit a local
backup, just in case. I'd thought the availability of higher fiber download
speeds negates the need for local backups, of readily downloadable media.

Of course, with personal and private data, plus configuration files, the
backup need is clearer and the strategy simpler.

Perhaps the whole backup strategy for files downloaded from the Internet, Vs
personal files, needs some critical (re)thinking.


> > Get yourself a basic 4-way DAS/JBOD setup, PLAN where you're putting all
> > this stuff, and plug in and remove drives as required. You don't need all
> > these huge drives if you think about what you're going to do with it all.
>
> That’s actually a good idea. Either use a hot swap frame for an internal 5¼?
> PC bay, a desktop dock for bare drives or a multi-bay enclosure. The market
> is big, you have lots of choices. USB (with or without integrated hub),
> eSATA, one or two bays, etc: https://skinflint.co.uk/?cat=hddocks
>
> Advantages:
> - no separate system to maintain just for storage: save $$$, time and power
> - very flexible: no chassis limitation on number of disks
> - no bulky external enclosures, each using a different power brick and cable
> - minimum volume to put into a safe (just get or make a bulk storage case)
>
> Disadvantages:
> - not as “fancy” as a NAS
> - possibly not all disks can be used at the same time
> - physical handling of naked disks takes more care
> - LVM is not practical, so use each disk separately
> - you gotta remember which files are where¹
> - SATA connectors aren’t made for very many insertion cycles (I think the
> spec says 50?), which doesn’t mean they endure much more, but still …
>
> > (And while it takes time and hammers the system, I regularly record off
> > the
> > TV getting a 2GB .ts file, convert it to mp4 - same resolution - and
> > reduce
> > the size by an order of magnitude - maybe more.
>
> Well, ts uses mpeg2 encoding, just like old video DVDs, which is very
> inefficient when compared with modern h264/h265. Modern digital TV broadcast
> uses h264 by now.

Depending on the PVR make/model I've seen 1080p resolution recordings with
.m2ts and .ts file extensions, while the codecs inside them are the same.
Here's the ffprobe output of a .ts recording containing a h264 video I
captured recently off terrestrial TV transmission:

Input #0, mpegts, from '20221209 0147 - BBC ONE HD - Weather for the Week
Ahead.ts':
Duration: 00:13:01.44, start: 48999.919856, bitrate: 3744 kb/s
Program 17540
Stream #0:0[0x19c9]: Video: h264 (High) ([27][0][0][0] / 0x001B),
yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr,
90k tbn, 50 tbc
Stream #0:1[0x19ca](eng): Audio: aac_latm (LC) ([17][0][0][0] / 0x0011),
48000 Hz, 5.1, fltp
Stream #0:2[0x19ce](eng): Unknown: none ([17][0][0][0] / 0x0011) (visual
impaired) (descriptions) (dependent)
Stream #0:3[0x19cd](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)

The h264 codec means converting the .ts file to mp4 will not change much at
all the size of the file. If anything the mp4 file could increase in size if
the audio stream and subtitles need to be transcoded.


> Incidentally, I got myself a new HDD today: an external 2.5? WD Passport
> Ultra 5 TB with USB-C 3.0. Just because I like portable storage and also
> because I need temporary space if I want to convert my NAS RAID-Z2 to Z1.
>
>
> ¹ I do have several external USB disks, plus the big NAS. All of which don’t
> run very often. And I don’t want to turn them on just to look for a certain
> file. That’s why I have another little script. ;-) It uses the `tree`
> command to save the complete content listing of a directory into a text
> file and names the file automatically by the name of the directory it
> crawls. So if I want to find a file, I just need to grep through my text
> files.

Backup scripts utilising rsync, tar, etc. can output a log file which contains
(some) details of all the backed up files. Nothing as sophisticated as
Frank's script, but it allows for a quick search against the name of the file
or directory, before extraction.

Am Fri, Dec 09, 2022 at 01:13:50PM +0000 schrieb Michael:

> > > I get the impression Dale isn't actually PLANNING his disk storage. It's
> > > just a case of "help I'm downloading all this stuff where do I put it!!!"
> >
> > Haha, thanks for the laugh.
>
> Actually this had me thinking what is the need to back up the ... Internet?
> […]
>
> I appreciate some of these video files may be rare finds, or there may be a
> risk some of these may be taken off the interwebs sooner or later. This
> should leave a rather small subset of all downloads, which may merit a local
> backup, just in case. I'd thought the availability of higher fiber download
> speeds negates the need for local backups, of readily downloadable media.

Good points. I am a big fan of having stuff locally as well, because I don’t
want to be dependent on a company’s servers and a working Internet connection.
But this mostly applies to my mobile device, because I don’t have a data plan
for mobile Internet.

> > Well, ts uses mpeg2 encoding, just like old video DVDs, which is very
> > inefficient when compared with modern h264/h265. Modern digital TV broadcast
> > uses h264 by now.
>
> Depending on the PVR make/model I've seen 1080p resolution recordings with
> .m2ts and .ts file extensions, while the codecs inside them are the same.

I wasn’t aware that ts could contain h264. But then again—I never really
bothered with live TV recordings in recent years. These days, if I find
something interesting, I download the show form the TV channel’s website
(called Mediathek in Germany, a word play on Bibliothek, meaning library).
Interestingly though, the picture quality is noticably worse than what I
receive via DVB-T.

> > ¹ I do have several external USB disks, plus the big NAS. All of which don’t
> > run very often. And I don’t want to turn them on just to look for a certain
> > file. That’s why I have another little script. ;-) It uses the `tree`
> > command to save the complete content listing of a directory into a text
> > file and names the file automatically by the name of the directory it
> > crawls. So if I want to find a file, I just need to grep through my text
> > files.
>
> Backup scripts utilising rsync, tar, etc. can output a log file which contains
> (some) details of all the backed up files. Nothing as sophisticated as
> Frank's script, but it allows for a quick search against the name of the file
> or directory, before extraction.

Naturally, I just discovered two bugs in the script while I was re-reading
my mail. One of them broke the creation of the symlink which points to the
most recent version of a script output. The other prevented normal operation
if only gzip was available amongst the used compressors.

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

We promise nothing, but that we keep.

Dale wrote:
> Howdy,
>
> <<<SNIP>>>
>
> Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as
> another option.  They come as parts, cases too, but the newer and faster
> models of Raspberry Pi 4 with more ram seem to work pretty well.  The
> old slower models with small amounts of ram don't fair as well.  While I
> want a descent speed, I'm not looking for or expecting it to be
> blazingly fast.  I just wonder, if from a upgrade and expansion point of
> view, if building a NAS would be better.  I've also noticed, it seems
> all Raspberry things come with a display port.  That means I could hook
> up a monitor and mouse/keyboard when needed.  That could be a bonus. 
> Heck, I may can even put some sort of Gentoo on that thing.  :-D
>
> <<<SNIP>>>
>
> Thoughts?  Info to share?  Ideas on a best path forward?  Buy already
> built or build?
>
> Thanks.
>
> Dale
>
> :-)  :-) 
>


For those interested, I'm pretty sure this video is about this thing. 
It was new and not released to the general public at the time so it does
mention some bugs and missing drivers.  I suspect those are fixed or
included by now.  Also, the case I saw appears to be newer and not
something that requires all the assembly shown in the video. 

https://www.youtube.com/watch?v=ahrdx3TYxZc

That same channel has another video that was a pretty insane build cost
wise.  It had huge SSD drives.  Bonus video for anyone bored to tears. lol

https://www.youtube.com/watch?v=G_px298IF2k

This is the case I'm looking at.  Pretty sure the card, CPU board and
drives goes into this.  Plus I think they added a display too. 

https://shop.allnetchina.cn/collections/sata-hat/products/quad-sata-kit-for-raspberry-pi-4-case-only

As I mentioned earlier, I plan to research that seller more.  I want to
make sure they stand behind their sales.  Most likely do but I don't
know that.  ;-)

It seems that people are working on making a really nice Raspberry Pi
device for use as a NAS.  It may not be perfect but even what is in the
1st video would likely work fine for me.  Storage wise.  I like the case
I linked to better tho. 

Dale

:-)  :-) 

On Fri, Dec 9, 2022 at 8:13 AM Michael <confabulate@kintzios.com> wrote:
>
> Actually this had me thinking what is the need to back up the ... Internet?

I'm sure the NSA knows the answer to this. Based on discussions I've
had with people who are into such things they basically have their own
Wayback machine, except it obviously doesn't respect robots.txt or
takedown requests.

I kind of wish the NSA sold IT services to the general public. I just
assume they probably have root on all my devices and their own backups
of everything on them. It would be nice if I had a disaster if I
could just pay them to buy back a copy of my data, instead of having
to have my own completely redundant backups.

I'm personally using duplicity for encrypted cloud backups of the
stuff that is most critical (documents, recent photos, etc), AWS
Glacier for stuff I want long-term backups of (older photos mostly),
and then bacula to store local copies of everything I have any
interest in because that is easier than trying to restore it all off
of Amazon if I lose an array or whatever. AWS Glacier is actually
pretty cheap for backup, but be prepared to pay a fair bit for
restoration. I'd only need to go to them in a serious disaster like a
house fire, so having to pay $100 or whatever to get them to mail me a
hard drive with my data isn't really that big of a deal. My backups
are generally one-way affairs.

--
Rich

Frank Steinmetzger wrote:
> Am Fri, Dec 09, 2022 at 01:13:50PM +0000 schrieb Michael:
>
>
> Good points. I am a big fan of having stuff locally as well, because I don?t
> want to be dependent on a company?s servers and a working Internet connection.
> But this mostly applies to my mobile device, because I don?t have a data plan
> for mobile Internet.
>
>

That is my reason.? I already have some videos that are no longer
available.? Even references to the video ever existing is hard to find.?
If I lose them, I may never get them again.? So, I try to keep copies
and hang onto them.? Plus, while I've been fortunate and have a good
stable internet connection, both with DSL and fibre, I don't know what
next year or even next month will bring.? You add in all the censorship
that is going on, it gets even worse.? I have many videos that I
downloaded from youtube that have been removed because of basically, a
political position.? The people running Youtube doesn't like the content
so it was removed.? Keep in mind, those videos were instructional and
nothing about illegal activity or anything.? They just don't like the
content.? It's also why other sites came along so people could post the
exact same type of content.?

I don't always save everything but when I find a video that might be
useful, whether it is about repairing my old washing machine, repairs to
a firearm, planting trees from seeds or just a funny cat video, I save
it if it has some future value.? The bad thing, some channels that have
been removed for a small amount of videos Youtube didn't like also had
other videos removed as well.? I've noticed entire channels disappear.?
I can't trust the people running video websites to allow content even
for short term.? Post something they don't like, true or not, they can
and often do remove content and often remove entire channels.?

I don't trust my data to be on just one set of hard drives either, it's
why I make backup copies.? If one of my main drives lets the smoke out,
I've got a backup to restore from.? I wouldn't mind having two sets of
backups and one day, I just may.? I may end up with two NAS boxes, each
a independent backup copy.? May on alternate weeks or something.? If
possible, I wouldn't mind having backups where I can even go back in
time a bit.? Just in case I need a file that I deleted and then was
deleted from the backups as well.?

If this Raspberry thing works and is fairly cheap, I may end up with two
NAS boxes as backups and a NAS box as the actual storage itself.?

Since I can't trust people who run websites, I have to trust myself.?
Keep local copies, back those up just in case.?

Dale

:-)? :-)?

P. S.? Seamonkey still doesn't automatically fetch emails.? It's really
annoying.? :/

On Friday, December 9, 2022 12:58:38 A.M. AEDT Dale wrote:


> I was thinking DAS was not a good option. It seems like a feature
> removed and cheaper version of NAS.

Maybe get the DAS, then connect it to the Rasberry Pi 4, to make a DIY NAS.

That way you get a propper encolsure for your drives, without needing to cobble
something together.


--
Reverend Paul Colquhoun, ULC. http://andor.dropbear.id.au/
Asking for technical help in newsgroups? Read this first:
http://catb.org/~esr/faqs/smart-questions.html#intro

On 09/12/2022 13:38, Frank Steinmetzger wrote:
>> Depending on the PVR make/model I've seen 1080p resolution recordings with
>> .m2ts and .ts file extensions, while the codecs inside them are the same.

> I wasn’t aware that ts could contain h264. But then again—I never really
> bothered with live TV recordings in recent years. These days, if I find
> something interesting, I download the show form the TV channel’s website
> (called Mediathek in Germany, a word play on Bibliothek, meaning library).
> Interestingly though, the picture quality is noticably worse than what I
> receive via DVB-T.
>
I think this is confusing CONTAINER and CODEC.

.ts is a container format, h264 is a codec. I don't understand it
myself, either, but think of ts as your directory structure and h264 as
your file structure.

Incidentally, sticking this stuff in a .tar is probably okay - that's
just another container, but sticking it in a .tar.gz is not, the gz is
your codec and will make the file BIGGER in all probability.

Cheers,
Wol

Am Sat, Dec 10, 2022 at 09:20:17AM +0000 schrieb Wols Lists:

> > > Depending on the PVR make/model I've seen 1080p resolution recordings with
> > > .m2ts and .ts file extensions, while the codecs inside them are the same.
-------------------------------------^^^^^^^^^^^^^^^^^^^^^^^
>
> > I wasn’t aware that ts could contain h264. But then again—I never really
---------------------------------^^^^^^^
> > bothered with live TV recordings in recent years.

> I think this is confusing CONTAINER and CODEC.

Where do we confuse those two? We specifically talked of codecs and
“contain”.

> .ts is a container format, h264 is a codec. I don't understand it myself,
> either but think of ts as your directory structure and h264 as your file
> structure.

Now you are confusing me. You say you don’t understand it, but then explain
it. TS is like AVI and MKV: a file structure for the payload data. And
payload data can be all kinds of stuff, from ASS plaintext subtitles, over
opus audio to mpeg2 or h264 video.

> Incidentally, sticking this stuff in a .tar is probably okay - that's just
> another container, but sticking it in a .tar.gz is not, the gz is your codec
> and will make the file BIGGER in all probability.

Tar does not compress, it simply puts all inputs in a 1:1 stream. It does
add some metadata (filename and so on). Packers reduce data volume by
increasing information-per-byte. So if the total information stays the same
(for lossless coding), the number of bytes decreases. Encoded video data
ideally has even entropy. It is indistinguishable from random noise. That’s
why compressing it again does not yield anything, or even adds some volume
again.

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“Mankind must put an end to war or war will put an end to mankind.”
– John F. Kennedy

On 10/12/2022 16:19, Frank Steinmetzger wrote:
> Where do we confuse those two? We specifically talked of codecs and
> “contain”.

"I didn't know .ts could contain h264".

If .ts is the container, then surely the assumption is it can contain
any codec? If not, why not?

(Yes I do get the impression I didn't read the OP properly. But then,
the OP didn't make sense properly so I'm not surprised I got it wrong :-)

Cheers,
Wol

On Saturday, 10 December 2022 16:30:03 GMT Wols Lists wrote:
> On 10/12/2022 16:19, Frank Steinmetzger wrote:
> > Where do we confuse those two? We specifically talked of codecs and
> > “contain”.
>
> "I didn't know .ts could contain h264".
>
> If .ts is the container, then surely the assumption is it can contain
> any codec? If not, why not?

Not any codec. Some container formats are only compatible with certain
codecs, or rather the other way around. Have a look here:

https://en.wikipedia.org/wiki/Comparison_of_video_container_formats

I suppose the answer to 'why not' boils down to the whatever structure and
data the container format is designed to be compatible with, but I don't know
more than this.

https://en.wikipedia.org/wiki/File:Informatik-Containerformate-Beispiele.svg

Someone has hacked my ph.im 44 years old an pay for every thing.house car
an wifi

Dave

On Sat, Dec 10, 2022, 12:28 PM Michael <confabulate@kintzios.com> wrote:

> On Saturday, 10 December 2022 16:30:03 GMT Wols Lists wrote:
> > On 10/12/2022 16:19, Frank Steinmetzger wrote:
> > > Where do we confuse those two? We specifically talked of codecs and
> > > “contain”.
> >
> > "I didn't know .ts could contain h264".
> >
> > If .ts is the container, then surely the assumption is it can contain
> > any codec? If not, why not?
>
> Not any codec. Some container formats are only compatible with certain
> codecs, or rather the other way around. Have a look here:
>
> https://en.wikipedia.org/wiki/Comparison_of_video_container_formats
>
> I suppose the answer to 'why not' boils down to the whatever structure and
> data the container format is designed to be compatible with, but I don't
> know
> more than this.
>
>
> https://en.wikipedia.org/wiki/File:Informatik-Containerformate-Beispiele.svg
>
>

Dale wrote:
> Howdy,
>
> I've pretty much reached a limit on my backups.  I'm up to a 16TB hard
> drive for one and even that won't last long.  Larger drives are much
> more costly.  A must have NAS is quickly approaching.  I've been
> searching around and find some things confusing.  I'm hoping someone can
> clear up that confusion.  I'm also debating what path to travel down. 
> I'd also like to keep costs down as well.  That said, I don't mind
> paying a little more for one that would offer a much better option. 
>
> Path one, buy a NAS, possibly used, that has no drives.  If possible, I
> may even replace the OS that comes on it or upgrade if I can.  I'm not
> looking for fancy, or even RAID.  Just looking for a two bay NAS that
> will work.  First, what is a DAS?  Is that totally different than a
> NAS?  From what I've found, a DAS is not what I'm looking for since I
> want a ethernet connection and the ability to control things over the
> network.  It seems DAS lacks that feature but not real sure.  I'm not
> sure I can upgrade the software/OS on a DAS either. 
>
> Next thing.  Let's say a NAS comes with two 4TB drives for a total of
> 8TB of capacity from the factory, using LVM or similar software I
> assume.  Is that limited to that capacity or can I for example replace
> one or both drives with for example 14TB drives for a total of 28TBs of
> capacity?  If one does that, let's say it uses LVM, can I somehow move
> data as well or is that beyond the abilities of a NAS?  Could it be done
> inside my computer for example?  Does this vary by brand or even model? 
>
> Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as
> another option.  They come as parts, cases too, but the newer and faster
> models of Raspberry Pi 4 with more ram seem to work pretty well.  The
> old slower models with small amounts of ram don't fair as well.  While I
> want a descent speed, I'm not looking for or expecting it to be
> blazingly fast.  I just wonder, if from a upgrade and expansion point of
> view, if building a NAS would be better.  I've also noticed, it seems
> all Raspberry things come with a display port.  That means I could hook
> up a monitor and mouse/keyboard when needed.  That could be a bonus. 
> Heck, I may can even put some sort of Gentoo on that thing.  :-D
>
> One reason I'm wanting to go this route, I'm trying to keep it small and
> able to fit inside my fire safe.  I plan to buy a media type safe that
> is larger but right now, it needs to fit inside my current safe.  Most
> of the 2 bay NAS or a Raspberry Pi based NAS are fairly small.  They not
> much bigger than the three external hard drives and a couple bare drives
> that currently occupy my safe. 
>
> One thing I'd like to have no matter what path I go down, the ability to
> encrypt the data.  My current backup drives are encrypted and I'd like
> to keep it that way.  If that is possible to do.  I suspect the
> Raspberry option would since I'd control the OS/software placed on it. 
> I could be wrong tho. 
>
> One last thing.  Are there any NAS type boxes that I should absolutely
> avoid if I go that route?  Maybe it is a model that has serious
> limitations or has other problems.  I think the DAS thing may be one for
> me to avoid but I'm not for sure what limits it has.  Google didn't help
> a lot. It also could be as simple as, avoid any model that says this in
> the description or uses some type of software that is bad or limits
> options. 
>
> Thoughts?  Info to share?  Ideas on a best path forward?  Buy already
> built or build?
>
> Thanks.
>
> Dale
>
> :-)  :-) 
>

It does appear that several people are making it so NAS boxes can be
easily built by us nerdy types and not be huge.  Things are a bit
interesting right now for hardware and such.  Some parts are expensive,
hard to find or just plain unavailable.  Still, I suspect that this will
get better later on.  The good thing, people are working on this. There
is interest in having a option. 

I may in the meantime have to split up my large directory and the
resulting backup.  Split it in half or something.  I just bought a 16TB
hard drive.  It's over 90% full already.  That said, my downloading is
slowing down quite a lot.  It will last a while.  I may setup a old
system as a NAS and just keep it in a outbuilding for the time being. 
Should offer me some protection at least.

If anyone runs up on some info that might relate to this, please share. 
Maybe something new will come out that we don't know about now. If I do
build something, I'll likely post and share what I used, how much effort
was involved and the end results. Maybe this will help others as well.

Thanks.

Dale

:-)  :-)

P. S.  I currently have a spare 14TB, 8TB and 6TB hard drive not in
use.  That's 28TBs available.  That's a good start on a NAS as far as
drives go.  Currently in use in external enclosures for backups, 16Tb,
8TB and a 6TB hard drive. 

On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com> wrote:
<SNIP>
> It does appear that several people are making it so NAS boxes can be
> easily built by us nerdy types and not be huge. Things are a bit
> interesting right now for hardware and such. Some parts are expensive,
> hard to find or just plain unavailable. Still, I suspect that this will
> get better later on. The good thing, people are working on this. There
> is interest in having a option.
>

I know I've said this before but yes, people are working on it and it
is buildable by people like us.

https://www.truenas.com/truenas-core/

Any old x64 PC and a few disks will get you up and running. It's
open source as much as BSD is open source. It's not Linux
so there was a little learning to do but it wasn't bad.

Mine has a small SSD as the boot drive and then RAID1 pairs
running OpenZFS for storage. It's inexpensive if you have an
old computer to build on.

Good luck,
Mark

Mark Knecht wrote:
>
>
> On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com
> <mailto:rdalek1967@gmail.com>> wrote:
> <SNIP>
> > It does appear that several people are making it so NAS boxes can be
> > easily built by us nerdy types and not be huge.  Things are a bit
> > interesting right now for hardware and such.  Some parts are expensive,
> > hard to find or just plain unavailable.  Still, I suspect that this will
> > get better later on.  The good thing, people are working on this. There
> > is interest in having a option.
> >
>
> I know I've said this before but yes, people are working on it and it
> is buildable by people like us.
>
> https://www.truenas.com/truenas-core/
>
> Any old x64 PC and a few disks will get you up and running. It's 
> open source as much as BSD is open source. It's not Linux
> so there was a little learning to do but it wasn't bad.
>
> Mine has a small SSD as the boot drive and then RAID1 pairs
> running OpenZFS for storage. It's inexpensive if you have an 
> old computer to build on.
>
> Good luck,
> Mark

I got a old rig I can use.  I actually burned OpenNAS, TrueNAS or
FreeNAS on a USB stick.  I can't recall which one I put on it tho.  I
downloaded all three.  lol  If you know that one is better than the
others, feel free to share.  Also, I'd like to keep using LVM if I can. 
If nothing else, I already got the data on the drives and won't have to
reformat and copy again.  It took almost 100 hours to copy to the new
16TB drive.  Using LVM would make that easier, and faster.

I'll have to work with what I got for now but I really like the
Raspberry option for its size and good options to upgrade later.  I'll
just make do with something else until that option is doable.  Maybe it
won't be to long. 

Dale

:-)  :-) 

Dave

On Sat, Dec 10, 2022, 4:28 PM Mark Knecht <markknecht@gmail.com> wrote:

>
>
> On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com> wrote:
> <SNIP>
> > It does appear that several people are making it so NAS boxes can be
> > easily built by us nerdy types and not be huge. Things are a bit
> > interesting right now for hardware and such. Some parts are expensive,
> > hard to find or just plain unavailable. Still, I suspect that this will
> > get better later on. The good thing, people are working on this. There
> > is interest in having a option.
> >
>
> I know I've said this before but yes, people are working on it and it
> is buildable by people like us.
>
> https://www.truenas.com/truenas-core/
>
> Any old x64 PC and a few disks will get you up and running. It's
> open source as much as BSD is open source. It's not Linux
> so there was a little learning to do but it wasn't bad.
>
> Mine has a small SSD as the boot drive and then RAID1 pairs
> running OpenZFS for storage. It's inexpensive if you have an
> old computer to build on.
>
> Good luck,
> Mark
>

Tanks

Dave

On Sat, Dec 10, 2022, 9:46 PM David Rosenbaum <rosenbaumd181@gmail.com>
wrote:

>
>
> Dave
>
> On Sat, Dec 10, 2022, 4:28 PM Mark Knecht <markknecht@gmail.com> wrote:
>
>>
>>
>> On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com> wrote:
>> <SNIP>
>> > It does appear that several people are making it so NAS boxes can be
>> > easily built by us nerdy types and not be huge. Things are a bit
>> > interesting right now for hardware and such. Some parts are expensive,
>> > hard to find or just plain unavailable. Still, I suspect that this will
>> > get better later on. The good thing, people are working on this. There
>> > is interest in having a option.
>> >
>>
>> I know I've said this before but yes, people are working on it and it
>> is buildable by people like us.
>>
>> https://www.truenas.com/truenas-core/
>>
>> Any old x64 PC and a few disks will get you up and running. It's
>> open source as much as BSD is open source. It's not Linux
>> so there was a little learning to do but it wasn't bad.
>>
>> Mine has a small SSD as the boot drive and then RAID1 pairs
>> running OpenZFS for storage. It's inexpensive if you have an
>> old computer to build on.
>>
>> Good luck,
>> Mark
>>
>

<SNIP>
>
> I got a old rig I can use. I actually burned OpenNAS, TrueNAS or FreeNAS
on a USB stick. I can't recall which one I put on it tho. I downloaded
all three. lol If you know that one is better than the others, feel free
to share. Also, I'd like to keep using LVM if I can. If nothing else, I
already got the data on the drives and won't have to reformat and copy
again. It took almost 100 hours to copy to the new 16TB drive. Using LVM
would make that easier, and faster.
>
> I'll have to work with what I got for now but I really like the Raspberry
option for its size and good options to upgrade later. I'll just make do
with something else until that option is doable. Maybe it won't be to
long.
>
> Dale
>
> :-) :-)

TrueNAS Core. It's the free one. Works great. Very stable, but it is BSD,
not Linux so you'll be frustrated sometimes. None the less it works very
well.

Mark Knecht wrote:
> <SNIP>
> >
> > I got a old rig I can use.  I actually burned OpenNAS, TrueNAS or
> FreeNAS on a USB stick.  I can't recall which one I put on it tho.  I
> downloaded all three.  lol  If you know that one is better than the
> others, feel free to share.  Also, I'd like to keep using LVM if I
> can.  If nothing else, I already got the data on the drives and won't
> have to reformat and copy again.  It took almost 100 hours to copy to
> the new 16TB drive.  Using LVM would make that easier, and faster.
> >
> > I'll have to work with what I got for now but I really like the
> Raspberry option for its size and good options to upgrade later.  I'll
> just make do with something else until that option is doable.  Maybe
> it won't be to long.
> >
> > Dale
> >
> > :-)  :-)
>
> TrueNAS Core. It's the free one. Works great. Very stable, but it is
> BSD, not Linux so you'll be frustrated sometimes. None the less it
> works very well.


Well, I booted it and it is FreeNAS.  I got it on a USB stick tho. 
Well, I put the installer on one stick and then installed on a second
stick.  Kinda odd but I get it.  I also noticed it is BSD based.  I
played with BSD once before.  One thing I can say, it's secure.  Big time.

I see it uses ZFS or something.  No mention of LVM.  I figured that.  Oh
well. 

If I can't hammer FreeNAS into shape, I'll try TrueNAS next.  If it
works, that's fine too.  ;-) 

Thanks.

Dale

:-)  :-) 

P. S.  When I first booted, I didn't have the ethernet plugged in.  It
wasn't happy about that.  Given it is NAS software, I should have known
better.  ROFL 

Can I get sum help with privacy an control of my ph.

David

On Sat, Dec 10, 2022, 9:49 PM David Rosenbaum <rosenbaumd181@gmail.com>
wrote:

> Tanks
>
> Dave
>
> On Sat, Dec 10, 2022, 9:46 PM David Rosenbaum <rosenbaumd181@gmail.com>
> wrote:
>
>>
>>
>> Dave
>>
>> On Sat, Dec 10, 2022, 4:28 PM Mark Knecht <markknecht@gmail.com> wrote:
>>
>>>
>>>
>>> On Sat, Dec 10, 2022 at 1:42 PM Dale <rdalek1967@gmail.com> wrote:
>>> <SNIP>
>>> > It does appear that several people are making it so NAS boxes can be
>>> > easily built by us nerdy types and not be huge. Things are a bit
>>> > interesting right now for hardware and such. Some parts are expensive,
>>> > hard to find or just plain unavailable. Still, I suspect that this
>>> will
>>> > get better later on. The good thing, people are working on this. There
>>> > is interest in having a option.
>>> >
>>>
>>> I know I've said this before but yes, people are working on it and it
>>> is buildable by people like us.
>>>
>>> https://www.truenas.com/truenas-core/
>>>
>>> Any old x64 PC and a few disks will get you up and running. It's
>>> open source as much as BSD is open source. It's not Linux
>>> so there was a little learning to do but it wasn't bad.
>>>
>>> Mine has a small SSD as the boot drive and then RAID1 pairs
>>> running OpenZFS for storage. It's inexpensive if you have an
>>> old computer to build on.
>>>
>>> Good luck,
>>> Mark
>>>
>>

Thanks

Dave

On Sat, Dec 10, 2022, 11:19 AM Frank Steinmetzger <Warp_7@gmx.de> wrote:

> Am Sat, Dec 10, 2022 at 09:20:17AM +0000 schrieb Wols Lists:
>
> > > > Depending on the PVR make/model I've seen 1080p resolution
> recordings with
> > > > .m2ts and .ts file extensions, while the codecs inside them are the
> same.
> -------------------------------------^^^^^^^^^^^^^^^^^^^^^^^
> >
> > > I wasn’t aware that ts could contain h264. But then again—I never
> really
> ---------------------------------^^^^^^^
> > > bothered with live TV recordings in recent years.
>
> > I think this is confusing CONTAINER and CODEC.
>
> Where do we confuse those two? We specifically talked of codecs and
> “contain”.
>
> > .ts is a container format, h264 is a codec. I don't understand it myself,
> > either but think of ts as your directory structure and h264 as your file
> > structure.
>
> Now you are confusing me. You say you don’t understand it, but then explain
> it. TS is like AVI and MKV: a file structure for the payload data. And
> payload data can be all kinds of stuff, from ASS plaintext subtitles, over
> opus audio to mpeg2 or h264 video.
>
> > Incidentally, sticking this stuff in a .tar is probably okay - that's
> just
> > another container, but sticking it in a .tar.gz is not, the gz is your
> codec
> > and will make the file BIGGER in all probability.
>
> Tar does not compress, it simply puts all inputs in a 1:1 stream. It does
> add some metadata (filename and so on). Packers reduce data volume by
> increasing information-per-byte. So if the total information stays the same
> (for lossless coding), the number of bytes decreases. Encoded video data
> ideally has even entropy. It is indistinguishable from random noise. That’s
> why compressing it again does not yield anything, or even adds some volume
> again.
>
> --
> Grüße | Greetings | Salut | Qapla’
> Please do not share anything from, with or about me on any social network.
>
> “Mankind must put an end to war or war will put an end to mankind.”
> – John F. Kennedy
>

On Sat, Dec 10, 2022 at 9:35 PM Dale <rdalek1967@gmail.com> wrote:
>
> Mark Knecht wrote:
> > <SNIP>
> > >
> > > I got a old rig I can use. I actually burned OpenNAS, TrueNAS or
> > FreeNAS on a USB stick. I can't recall which one I put on it tho. I
> > downloaded all three. lol If you know that one is better than the
> > others, feel free to share. Also, I'd like to keep using LVM if I
> > can. If nothing else, I already got the data on the drives and won't
> > have to reformat and copy again. It took almost 100 hours to copy to
> > the new 16TB drive. Using LVM would make that easier, and faster.
> > >
> > > I'll have to work with what I got for now but I really like the
> > Raspberry option for its size and good options to upgrade later. I'll
> > just make do with something else until that option is doable. Maybe
> > it won't be to long.
> > >
> > > Dale
> > >
> > > :-) :-)
> >
> > TrueNAS Core. It's the free one. Works great. Very stable, but it is
> > BSD, not Linux so you'll be frustrated sometimes. None the less it
> > works very well.
>
>
> Well, I booted it and it is FreeNAS. I got it on a USB stick tho.
> Well, I put the installer on one stick and then installed on a second
> stick. Kinda odd but I get it. I also noticed it is BSD based. I
> played with BSD once before. One thing I can say, it's secure. Big time.
>

I'm not clear exactly but FreeNAS _BECAME_ TrueNAS Core and TrueNAS
(all 3 versions) are the ones being worked on.

Installing from USB is pretty standard. Installing to a USB flash drive
is not unheard of in home NAS servers but be careful of machine
placement because people talk about USB sockets being unreliable
long term. I'm sure you'll figure it out, but make sure you're using
TrueNAS Core.

> I see it uses ZFS or something. No mention of LVM. I figured that. Oh
> well.

I see LVM as something that belongs on your machine, not your NAS
device. Your LVM volumes will just be directories on the NAS. You will
make your pools as large as you can afford and the NAS will just store
your data. You don't really need to worry about that much. My NAS
stores backups from 3 different machine, but all the backup data
is in a single ZFS RAID1 pool located in directories which macth the
name of the machine that wrote them.

>
> If I can't hammer FreeNAS into shape, I'll try TrueNAS next. If it
> works, that's fine too. ;-)

My input for the third time. Move to TrueNAS Core. That's the one
that is being developed and getting support.

Mark

Mark Knecht wrote:
>
>
> On Sat, Dec 10, 2022 at 9:35 PM Dale <rdalek1967@gmail.com
> <mailto:rdalek1967@gmail.com>> wrote:
> >
> > Mark Knecht wrote:
> > > <SNIP>
> > > >
> > > > I got a old rig I can use.  I actually burned OpenNAS, TrueNAS or
> > > FreeNAS on a USB stick.  I can't recall which one I put on it tho.  I
> > > downloaded all three.  lol  If you know that one is better than the
> > > others, feel free to share.  Also, I'd like to keep using LVM if I
> > > can.  If nothing else, I already got the data on the drives and won't
> > > have to reformat and copy again.  It took almost 100 hours to copy to
> > > the new 16TB drive.  Using LVM would make that easier, and faster.
> > > >
> > > > I'll have to work with what I got for now but I really like the
> > > Raspberry option for its size and good options to upgrade later.  I'll
> > > just make do with something else until that option is doable.  Maybe
> > > it won't be to long.
> > > >
> > > > Dale
> > > >
> > > > :-)  :-)
> > >
> > > TrueNAS Core. It's the free one. Works great. Very stable, but it is
> > > BSD, not Linux so you'll be frustrated sometimes. None the less it
> > > works very well.
> >
> >
> > Well, I booted it and it is FreeNAS.  I got it on a USB stick tho.
> > Well, I put the installer on one stick and then installed on a second
> > stick.  Kinda odd but I get it.  I also noticed it is BSD based.  I
> > played with BSD once before.  One thing I can say, it's secure.  Big
> time.
> >
>
> I'm not clear exactly but FreeNAS _BECAME_ TrueNAS Core and TrueNAS
> (all 3 versions) are the ones being worked on.
>
> Installing from USB is pretty standard. Installing to a USB flash drive
> is not unheard of in home NAS servers but be careful of machine
> placement because people talk about USB sockets being unreliable
> long term. I'm sure you'll figure it out, but make sure you're using
> TrueNAS Core. 
>
> > I see it uses ZFS or something.  No mention of LVM.  I figured that.  Oh
> > well.
>
> I see LVM as something that belongs on your machine, not your NAS
> device. Your LVM volumes will just be directories on the NAS. You will
> make your pools as large as you can afford and the NAS will just store
> your data. You don't really need to worry about that much. My NAS
> stores backups from 3 different machine, but all the backup data
> is in a single ZFS RAID1 pool located in directories which macth the 
> name of the machine that wrote them.
>  
> >
> > If I can't hammer FreeNAS into shape, I'll try TrueNAS next.  If it
> > works, that's fine too.  ;-)
>
> My input for the third time. Move to TrueNAS Core. That's the one
> that is being developed and getting support.
>  
> Mark


I think I'm going to switch.  I need to start over anyway.  I set up a
user account and a large pool but while I can mount it, I can't put
anything in it yet.  I get a permission error.  I likely missed a step
or something.  Starting over will help correct that.  lol 

By the way, when I got it installed, it did update to a newer version. 
I didn't look to see if it was dated in any way but updates seem to be
available for FreeNAS.  I dunno. 

Thanks for the info.

Dale

:-)  :-)