Mailing List Archive

On Sat, Apr 15, 2023 at 3:47?PM Dale <rdalek1967@gmail.com> wrote:
>
> Howdy,
>
> I finally broke down and bought a SSD. It's a Samsung V-Nand 870 EVO
> 500GB. My current OS sits on a 160GB drive so should be plenty. I plan
> to even add a boot image for the Gentoo LiveGUI thingy, maybe Knoppix or
> something plus my usual OS. By the way, caught one for sale for
> $40.00. It has a production date of 5/2021.
>
> My question is this. Do I need anything special in the kernel or
> special fstab options for this thing? I know at one point there was
> folks having problems with certain settings. I did some googling and it
> seems to be worked out but I want to be sure I don't blow this thing up
> or something.
>
> Anything else that makes these special? Any tips or tricks?
>
> Dale
>
> :-) :-)
>
> P. S. I'm hoping this will make my system a little more responsive.
> Maybe. Either way, that 160GB drive is getting a little full.
>

Dale,
I have 500GB SSDs and 1TB M.2 drives in all of my machines. No
machine boots from a spinning drive anyhmore. Never had any
problems.

The only thing I've done differently is the errors=remount=ro item
below. Other than that if whatever OS you install sets up boot, and
the machine boots, then it's just a drive in my experience

Best wishes,
Mark

# / was on /dev/nvme1n1p3 during installation
UUID=3fe6798f-653f-42e8-8e96-7ba0d490bfdf / ext4
errors=remount-ro 0 1
# /boot/efi was on /dev/nvme0n1p1 during installation
UUID=60DF-9F56 /boot/efi vfat umask=0077 0 1

On 4/15/23 17:24, Mark Knecht wrote:
>
>
> On Sat, Apr 15, 2023 at 3:47?PM Dale <rdalek1967@gmail.com <mailto:rdalek1967@gmail.com>> wrote:
> >
> > Howdy,
> >
> > I finally broke down and bought a SSD.  It's a Samsung V-Nand 870 EVO
> > 500GB.  My current OS sits on a 160GB drive so should be plenty.  I plan
> > to even add a boot image for the Gentoo LiveGUI thingy, maybe Knoppix or
> > something plus my usual OS.  By the way, caught one for sale for
> > $40.00.  It has a production date of 5/2021.
> >
> > My question is this.  Do I need anything special in the kernel or
> > special fstab options for this thing?  I know at one point there was
> > folks having problems with certain settings.  I did some googling and it
> > seems to be worked out but I want to be sure I don't blow this thing up
> > or something.
> >
> > Anything else that makes these special?  Any tips or tricks?
> >
> > Dale
> >
> > :-)  :-)
> >
> > P. S.  I'm hoping this will make my system a little more responsive.
> > Maybe.  Either way, that 160GB drive is getting a little full.
> >
>
> Dale,
>    I have 500GB SSDs and 1TB M.2 drives in all of my machines. No
> machine boots from a spinning drive anyhmore. Never had any
> problems.
>
>    The only thing I've done differently is the errors=remount=ro item
> below. Other than that if whatever OS you install sets up boot, and
> the machine boots, then it's just a drive in my experience
>
> Best wishes,
> Mark
>
> # / was on /dev/nvme1n1p3 during installation
> UUID=3fe6798f-653f-42e8-8e96-7ba0d490bfdf /               ext4    errors=remount-ro 0       1
> # /boot/efi was on /dev/nvme0n1p1 during installation
> UUID=60DF-9F56  /boot/efi       vfat    umask=0077      0       1

My 5-year old small box running 500GB SSD INTEL SSDSC2BF48 Atom processor, it is ON 24/7 running Asterisk and hylafax.
Never had a problem with it.
But it is recommended to to run via cron fstrim:
30 18 * * 2 /sbin/fstrim -v /

On 16/4/23 06:47, Dale wrote:
> Howdy,
>
> I finally broke down and bought a SSD.  It's a Samsung V-Nand 870 EVO
> 500GB.  My current OS sits on a 160GB drive so should be plenty.  I plan
> to even add a boot image for the Gentoo LiveGUI thingy, maybe Knoppix or
> something plus my usual OS.  By the way, caught one for sale for
> $40.00.  It has a production date of 5/2021.
>
> My question is this.  Do I need anything special in the kernel or
> special fstab options for this thing?  I know at one point there was
> folks having problems with certain settings.  I did some googling and it
> seems to be worked out but I want to be sure I don't blow this thing up
> or something.
>
> Anything else that makes these special?  Any tips or tricks?
>
> Dale
>
> :-)  :-)
>
> P. S.  I'm hoping this will make my system a little more responsive.
> Maybe.  Either way, that 160GB drive is getting a little full.
>
>
>
look into mount options for SSD's (discard option) and "fstrim" for
maintenance. (read up on trimmimg - doing a manual trim before the drive
reaches full allocation (they delete files, but do not erase them
because erasing is time consuming so its an OS controlled operation) or
auto trimming (which can cause serious pauses at awkward times) can
prevent serious performance degradation as it has to erase before
writing.  I am not sure of the current status but in the early days of
SSD's, this was serious concern.

BillK


BillK

On Sunday, 16 April 2023 02:47:00 BST William Kenworthy wrote:

> look into mount options for SSD's (discard option) and "fstrim" for
> maintenance. (read up on trimmimg - doing a manual trim before the drive
> reaches full allocation (they delete files, but do not erase them
> because erasing is time consuming so its an OS controlled operation) or
> auto trimming (which can cause serious pauses at awkward times) can
> prevent serious performance degradation as it has to erase before
> writing. I am not sure of the current status but in the early days of
> SSD's, this was serious concern.

In short, see https://wiki.gentoo.org/wiki/SSD . :)

--
Regards,
Peter.

On 16/4/23 15:18, Peter Humphrey wrote:
> On Sunday, 16 April 2023 02:47:00 BST William Kenworthy wrote:
>
>> look into mount options for SSD's (discard option) and "fstrim" for
>> maintenance. (read up on trimmimg - doing a manual trim before the drive
>> reaches full allocation (they delete files, but do not erase them
>> because erasing is time consuming so its an OS controlled operation) or
>> auto trimming (which can cause serious pauses at awkward times) can
>> prevent serious performance degradation as it has to erase before
>> writing. I am not sure of the current status but in the early days of
>> SSD's, this was serious concern.
> In short, see https://wiki.gentoo.org/wiki/SSD . :)
>
Excellent, condenses it nicely.

BillK

On Sun, Apr 16, 2023 at 1:44?AM William Kenworthy <billk@iinet.net.au>
wrote:
>
>
> On 16/4/23 15:18, Peter Humphrey wrote:
> > On Sunday, 16 April 2023 02:47:00 BST William Kenworthy wrote:
> >
> >> look into mount options for SSD's (discard option) and "fstrim" for
> >> maintenance. (read up on trimmimg - doing a manual trim before the
drive
> >> reaches full allocation (they delete files, but do not erase them
> >> because erasing is time consuming so its an OS controlled operation) or
> >> auto trimming (which can cause serious pauses at awkward times) can
> >> prevent serious performance degradation as it has to erase before
> >> writing. I am not sure of the current status but in the early days of
> >> SSD's, this was serious concern.
> > In short, see https://wiki.gentoo.org/wiki/SSD . :)
> >
> Excellent, condenses it nicely.
>
> BillK
>
>

OK Dale, I'm completely wrong, but also 'slightly' right.

If you have an SSD or nvme drive installed then fstrim should be
installed and run on a regular basis. However it's not 'required'.

Your system will still work, but after all blocks on the drive have
been used for file storage and later deleted, if they are not
written back to zeros then the next time you go to use that
block the write will be slower as the write must first write
zeros and then your data.

fstrim does the write to zeros so that during normal operation
you don't wait.

I've become so completely used to Kubuntu that I had to read
that this is all set up automatically when the system finds an
SSD or nvme. In Gentoo land you have to do this yourself.

Sorry for any confusion. Time to unsubscribe from this list
I guess and leave you all to your beloved distro.

Bye,
Mark

Mark Knecht wrote:
>
>
> On Sun, Apr 16, 2023 at 1:44?AM William Kenworthy <billk@iinet.net.au
> <mailto:billk@iinet.net.au>> wrote:
> >
> >
> > On 16/4/23 15:18, Peter Humphrey wrote:
> > > On Sunday, 16 April 2023 02:47:00 BST William Kenworthy wrote:
> > >
> > >> look into mount options for SSD's (discard option) and "fstrim" for
> > >> maintenance. (read up on trimmimg - doing a manual trim before
> the drive
> > >> reaches full allocation (they delete files, but do not erase them
> > >> because erasing is time consuming so its an OS controlled
> operation) or
> > >> auto trimming (which can cause serious pauses at awkward times) can
> > >> prevent serious performance degradation as it has to erase before
> > >> writing.  I am not sure of the current status but in the early
> days of
> > >> SSD's, this was serious concern.
> > > In short, see https://wiki.gentoo.org/wiki/SSD .  :)
> > >
> > Excellent, condenses it nicely.
> >
> > BillK
> >
> >
>
> OK Dale, I'm completely wrong, but also 'slightly' right.
>
> If you have an SSD or nvme drive installed then fstrim should be 
> installed and run on a regular basis. However it's not 'required'.
>
> Your system will still work, but after all blocks on the drive have
> been used for file storage and later deleted, if they are not
> written back to zeros then the next time you go to use that
> block the write will be slower as the write must first write
> zeros and then your data.
>
> fstrim does the write to zeros so that during normal operation
> you don't wait.
>
> I've become so completely used to Kubuntu that I had to read
> that this is all set up automatically when the system finds an
> SSD or nvme. In Gentoo land you have to do this yourself.
>
> Sorry for any confusion. Time to unsubscribe from this list
> I guess and leave you all to your beloved distro.
>
> Bye,
> Mark


Oh, please, don't go anywhere.  <begging>  We already lost the long term
Alan.  BTW, I checked on him a while back.  He's still OK.  It's been a
while tho. 

I read during a google search that some distros handle this sort of
thing automatically, some sort of firmware thing or something.  I
figured Gentoo didn't, it rarely does since that is the point of
Gentoo.  So, no harm.  Heck, I just now applied power to the thing.  I
don't even have it partitioned or anything yet.  Just rebooted after
rearranging all the cables, adding power splitter etc etc.

I do have one gripe.  Why can't drive makers pick a screw size and stick
to it on ALL drives?  It took some digging to find a screw that would
fit.  Some I bought that are supposed to work on SSDs were to short.  It
would likely work on a metal adapter but not a thicker plastic one. 
Luckily, I found 4 screws.  No clue where they came from.  Just in my
junk box.  Before this week, never laid eyes on a SSD before.  Anyone
know the thread size and count on those things?  I want to order a few,
just in case. 

Is running fstrim once a week to often?  I update my OS once a week but
given the amount of extra space, I'd think once a month would be often
enough.  After all, it is 500GB and I'll likely only use less than half
of that.  Most of the extra space will be extra boot options like
Knoppix or something.  I'm just thinking it would give it a longer
life.  Maybe my thinking is wrong???

Now to play with this thing.  I got to remember what all has to be
copied over so I can boot the new thing.  :/  Been ages since I moved a
OS to another hard drive.  Maybe a reinstall would work better.  :-\

Thanks to all. 

Dale

:-)  :-) 

P. S.  CCing Mark just in case. 

<SNIP>
>
>
> Oh, please, don't go anywhere. <begging> We already lost the long term
Alan. BTW, I checked on him a while back. He's still OK. It's been a
while tho.
>

Dale - I touched - truly. I'm happy to stick around but I no longer
run Gentoo and don't want to cause problems or piss off the
folks who need to be here.

> I read during a google search that some distros handle this sort of thing
automatically, some sort of firmware thing or something. I figured Gentoo
didn't, it rarely does since that is the point of Gentoo. So, no harm.
Heck, I just now applied power to the thing. I don't even have it
partitioned or anything yet. Just rebooted after rearranging all the
cables, adding power splitter etc etc.

I don't know of any distros that do any of this in firmware. Maybe someone
else can address that. Many distros now install fstrim by default and
update crontab. The Ubuntu family does, which I didn't know before this
thread. However it is on both my Kubuntu machine and my Ubuntu
Server machine.

BTW - Windows does this in Disk Defragmenter but you have to
schedule it yourself according to Bard and ChatGPT. I'll be in
Windows later today to record some new music and plan to look into
that then.

Answering the question from below - weekly is what was set up by
default here. If your drive isn't near full and you're not writing a lot of
new
data on it each week then weekly would seem reasonable to me.
However being that you are running Gentoo and hence compiling
lots and lots and lots of stuff every week it's possible that you
might _possibly_ want to run fstrim more often if your intermediate
files are going to this drive.


>
> I do have one gripe. Why can't drive makers pick a screw size and stick
to it on ALL drives? It took some digging to find a screw that would fit.
Some I bought that are supposed to work on SSDs were to short. It would
likely work on a metal adapter but not a thicker plastic one. Luckily, I
found 4 screws. No clue where they came from. Just in my junk box.
Before this week, never laid eyes on a SSD before. Anyone know the thread
size and count on those things? I want to order a few, just in case.

I second your gripe. I've purchased a couple of PC builder screw
sets from Amazon.

>
> Is running fstrim once a week to often? I update my OS once a week but
given the amount of extra space, I'd think once a month would be often
enough. After all, it is 500GB and I'll likely only use less than half of
that. Most of the extra space will be extra boot options like Knoppix or
something. I'm just thinking it would give it a longer life. Maybe my
thinking is wrong???
>
> Now to play with this thing. I got to remember what all has to be copied
over so I can boot the new thing. :/ Been ages since I moved a OS to
another hard drive. Maybe a reinstall would work better. :-\
>

I think you have at least 3 options to play with the drive:

1) It's Gentoo so install from scratch. You'll feel great
if it works. It will only take you a day or two.

2) Possibly dd the old drive to the SSD. If the new
SSD boots as the same /dev/sdX device it should
work, maybe, maybe not.

3) If you have another SATA port then dual boot,
either with Gentoo on both or something simple
like Kubuntu. A base Kubuntu install takes about
15 minutes and will probably give you its own
dual boot grub config. When you're sick of Kubuntu
you can once again install Gentoo.

Good luck no matter what path you take.

Mark

Mark Knecht wrote:
> <SNIP>
> >
> >
> > Oh, please, don't go anywhere.  <begging>  We already lost the long
> term Alan.  BTW, I checked on him a while back.  He's still OK.  It's
> been a while tho.
> >
>
> Dale - I touched - truly. I'm happy to stick around but I no longer 
> run Gentoo and don't want to cause problems or piss off the 
> folks who need to be here. 
>
> > I read during a google search that some distros handle this sort of
> thing automatically, some sort of firmware thing or something.  I
> figured Gentoo didn't, it rarely does since that is the point of
> Gentoo.  So, no harm.  Heck, I just now applied power to the thing.  I
> don't even have it partitioned or anything yet.  Just rebooted after
> rearranging all the cables, adding power splitter etc etc.
>
> I don't know of any distros that do any of this in firmware. Maybe someone
> else can address that. Many distros now install fstrim by default and 
> update crontab. The Ubuntu family does, which I didn't know before this
> thread. However it is on both my Kubuntu machine and my Ubuntu
> Server machine.
>

I think what I read is that it is done automatically.  They could have
meant a cron job.  I don't think they said how, just that it is already
set up to do it.  Firmware was mentioned in the thread somewhere so I
thought maybe that was it.  Either way, fstrim is installed here.  It's
part of util-linux and that is pulled in by several packages.  I doubt
it will be going away here anytime soon given the long list of packages
that need it. Just to set up a cron job for it.  Remembering the steps
for that will take time tho.  o_O


> BTW - Windows does this in Disk Defragmenter but you have to 
> schedule it yourself according to Bard and ChatGPT. I'll be in 
> Windows later today to record some new music and plan to look into
> that then.
>
> Answering the question from below - weekly is what was set up by 
> default here. If your drive isn't near full and you're not writing a
> lot of new
> data on it each week then weekly would seem reasonable to me. 
> However being that you are running Gentoo and hence compiling
> lots and lots and lots of stuff every week it's possible that you 
> might _possibly_ want to run fstrim more often if your intermediate 
> files are going to this drive. 
>
>
> >
> > I do have one gripe.  Why can't drive makers pick a screw size and
> stick to it on ALL drives?  It took some digging to find a screw that
> would fit.  Some I bought that are supposed to work on SSDs were to
> short.  It would likely work on a metal adapter but not a thicker
> plastic one.  Luckily, I found 4 screws.  No clue where they came
> from.  Just in my junk box.  Before this week, never laid eyes on a
> SSD before.  Anyone know the thread size and count on those things?  I
> want to order a few, just in case.
>
> I second your gripe. I've purchased a couple of PC builder screw 
> sets from Amazon.
>  
> >
> > Is running fstrim once a week to often?  I update my OS once a week
> but given the amount of extra space, I'd think once a month would be
> often enough.  After all, it is 500GB and I'll likely only use less
> than half of that.  Most of the extra space will be extra boot options
> like Knoppix or something.  I'm just thinking it would give it a
> longer life.  Maybe my thinking is wrong???
> >
> > Now to play with this thing.  I got to remember what all has to be
> copied over so I can boot the new thing.  :/  Been ages since I moved
> a OS to another hard drive.  Maybe a reinstall would work better.  :-\
> >
>
> I think you have at least 3 options to play with the drive:
>
> 1) It's Gentoo so install from scratch. You'll feel great 
> if it works. It will only take you a day or two.
>
> 2) Possibly dd the old drive to the SSD. If the new 
> SSD boots as the same /dev/sdX device it should 
> work, maybe, maybe not.
>
> 3) If you have another SATA port then dual boot, 
> either with Gentoo on both or something simple 
> like Kubuntu. A base Kubuntu install takes about
> 15 minutes and will probably give you its own
> dual boot grub config. When you're sick of Kubuntu
> you can once again install Gentoo.
>
> Good luck no matter what path you take.
>
> Mark


I've thought of a few options myself.  I sort of have a OS copy/backup
already.  I currently do the compiling in a chroot on a separate drive. 
I then copy the compiled packages and use the -k option to update the
live OS.  I'll continue to do that when I start booting from the SSD. 
That should limit writes and such to the SSD.  I also got to rearrange
things so I can put swap on that spare drive I compile on.  I don't want
swap on a SSD.  I wish this thing would stop using swap completely.  I
have swappiness set to 1 already and it still uses swap.

Right now, I'm debating the size of /boot.  Knoppix is pretty large. 
The Gentoo LiveGUI thingy is too.  So, it will have to be larger than
the few hundred megabytes my current one is.  I'm thinking 10GBs or so. 
Maybe 12GBs to make sure I'm good to go for the foreseeable future. 
They may limit them to DVD size right now but one day they could pass
that limit by.  Software isn't getting smaller.  Besides, USB is the
thing now.

Lots of options.

Dale

:-)  :-) 

On Sun, Apr 16, 2023 at 4:09?PM Mark Knecht <markknecht@gmail.com> wrote:

> Sorry for any confusion. Time to unsubscribe from this list
> I guess and leave you all to your beloved distro.
>
Please don't.
I doubt someone is pissed off with you.

Jorge Almeida

> Bye,
See above.
> Mark

Am Sun, Apr 16, 2023 at 08:08:59AM -0700 schrieb Mark Knecht:

> If you have an SSD or nvme drive installed then fstrim should be
> installed and run on a regular basis. However it's not 'required'.
>
> Your system will still work, but after all blocks on the drive have
> been used for file storage and later deleted, if they are not
> written back to zeros then the next time you go to use that
> block the write will be slower as the write must first write
> zeros and then your data.
>
> fstrim does the write to zeros so that during normal operation
> you don't wait.

That is not quite correct. Trimming is about the oppisite of what you say,
namely to *not* rewrite areas. Flash memory can only be written to in
relatively large blocks. So if your file system wants to write 4 KiB, the
drive needs to read all the many kB around it (several hundreds at least,
perhaps eben MiBs, I’m not certain), change the small part in question and
write the whole block back. This is called write amplification. This also
occurs on hard drives, for example when you run a database which uses 4 kiB
datafile chunks, but on a file system with larger sectors. Then the file
system is the cause for write amplification.

If the SSD knew beforehand that the area is unused, it does not need to read
it all in and then write it back. The SSD controller has no knowledge of
file systems. And this is where trim comes in: it does know file systems,
detects the unused areas and translates that info for the drive controller.
Also, only trimmed areas (i.e. areas the controller knows are unused) can be
used for wear leveling.

I even think that If you read from a trimmed area, the controller does not
actually read the flash device, but simply returns zeroes. This is basically
what a quick erase does; it trims the entire drive, which takes only a few
seconds, and then all the data has become inaccessible (unless you address
the memory chips directly). It is similar to deleting a file: you erase its
entry in the directory, but not the actual payload bytes.

AFAIK, SMR HDDs also support trim these days, so they don’t need to do their
SMR reshuffling. I have a WD Passport Ultra external 2.5? HDD with 5 TB, and
it supports trim. However, a WD Elements 2.5? 4 TB does not. Perhaps because
it is a cheaper series. Every laptop HDD of 2 (or even 1) TB is SMR.

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

“It is hard to be a conquering hero when it is not in your nature.”
– Captain Hans Geering, ’Allo ’Allo

On Sun, Apr 16, 2023 at 9:55?AM Dale <rdalek1967@gmail.com> wrote:
<SNIP>
>
> I think what I read is that it is done automatically. They could have
> meant a cron job. I don't think they said how, just that it is already
> set up to do it. Firmware was mentioned in the thread somewhere so I
> thought maybe that was it. Either way, fstrim is installed here. It's
> part of util-linux and that is pulled in by several packages. I doubt
> it will be going away here anytime soon given the long list of packages
> that need it. Just to set up a cron job for it. Remembering the steps
> for that will take time tho. o_O
>

Hey - it's the Internet. I thought we could trust everything we read...

<SNIP>
>
>
> I've thought of a few options myself. I sort of have a OS copy/backup
> already. I currently do the compiling in a chroot on a separate drive.
> I then copy the compiled packages and use the -k option to update the
> live OS. I'll continue to do that when I start booting from the SSD.
> That should limit writes and such to the SSD. I also got to rearrange
> things so I can put swap on that spare drive I compile on. I don't want
> swap on a SSD. I wish this thing would stop using swap completely. I
> have swappiness set to 1 already and it still uses swap.
>

Well, that sounds like a solution to do your emerge work although
you're limited to the speed of that hard drive. If it's done in the
background then maybe you don't care.

> Right now, I'm debating the size of /boot. Knoppix is pretty large.
> The Gentoo LiveGUI thingy is too. So, it will have to be larger than
> the few hundred megabytes my current one is. I'm thinking 10GBs or so.
> Maybe 12GBs to make sure I'm good to go for the foreseeable future.
> They may limit them to DVD size right now but one day they could pass
> that limit by. Software isn't getting smaller. Besides, USB is the
> thing now.
>

I guess I don't understand why you would put Knoppix in the boot
partition vs somewhere else. Is this for some sort of recovery
process you're comfortable with vs recovering from a bootable DVD?

- Mark

Mark Knecht wrote:
> <<< SNIP >>>
> I guess I don't understand why you would put Knoppix in the boot 
> partition vs somewhere else. Is this for some sort of recovery 
> process you're comfortable with vs recovering from a bootable DVD?
>
> - Mark

I'm wanting to be able to boot something from the hard drive in the
event the OS itself won't boot.  The other day I had to dig around and
find a bootable USB stick and also found a DVD.  Ended up with the DVD
working best.  I already have memtest on /boot.  Thing is, I very rarely
use it.  ;-)

Dale

:-)  :-) 

On Sun, Apr 16, 2023 at 11:54?AM Dale <rdalek1967@gmail.com> wrote:
>
> Mark Knecht wrote:
> > <<< SNIP >>>
> > I guess I don't understand why you would put Knoppix in the boot
> > partition vs somewhere else. Is this for some sort of recovery
> > process you're comfortable with vs recovering from a bootable DVD?
> >
> > - Mark
>
> I'm wanting to be able to boot something from the hard drive in the
> event the OS itself won't boot. The other day I had to dig around and
> find a bootable USB stick and also found a DVD. Ended up with the DVD
> working best. I already have memtest on /boot. Thing is, I very rarely
> use it. ;-)

So in the scenario you are suggesting, is grub working, giving you a
boot choice screen, and your new Gentoo install is not working so
you want to choose Knoppix to repair whatever is wrong with
Gentoo?

If that's the case why shoehorn Knoppix into the boot partition
vs just give it its own partition?

Frank,
Thank you for the in-depth explanation.

I need to do some study before commenting further other than to say
so far I'm finding different comments depending on whether it's
an SSD or an M.2 drive.

Much appreciated,
Mark

On Sun, Apr 16, 2023 at 11:08?AM Frank Steinmetzger <Warp_7@gmx.de> wrote:

> Am Sun, Apr 16, 2023 at 08:08:59AM -0700 schrieb Mark Knecht:
>
> > If you have an SSD or nvme drive installed then fstrim should be
> > installed and run on a regular basis. However it's not 'required'.
> >
> > Your system will still work, but after all blocks on the drive have
> > been used for file storage and later deleted, if they are not
> > written back to zeros then the next time you go to use that
> > block the write will be slower as the write must first write
> > zeros and then your data.
> >
> > fstrim does the write to zeros so that during normal operation
> > you don't wait.
>
> That is not quite correct. Trimming is about the oppisite of what you say,
> namely to *not* rewrite areas. Flash memory can only be written to in
> relatively large blocks. So if your file system wants to write 4 KiB, the
> drive needs to read all the many kB around it (several hundreds at least,
> perhaps eben MiBs, I’m not certain), change the small part in question and
> write the whole block back. This is called write amplification. This also
> occurs on hard drives, for example when you run a database which uses 4
> kiB
> datafile chunks, but on a file system with larger sectors. Then the file
> system is the cause for write amplification.
>
> If the SSD knew beforehand that the area is unused, it does not need to
> read
> it all in and then write it back. The SSD controller has no knowledge of
> file systems. And this is where trim comes in: it does know file systems,
> detects the unused areas and translates that info for the drive
> controller.
> Also, only trimmed areas (i.e. areas the controller knows are unused) can
> be
> used for wear leveling.
>
> I even think that If you read from a trimmed area, the controller does not
> actually read the flash device, but simply returns zeroes. This is
> basically
> what a quick erase does; it trims the entire drive, which takes only a few
> seconds, and then all the data has become inaccessible (unless you address
> the memory chips directly). It is similar to deleting a file: you erase
> its
> entry in the directory, but not the actual payload bytes.
>
> AFAIK, SMR HDDs also support trim these days, so they don’t need to do
> their
> SMR reshuffling. I have a WD Passport Ultra external 2.5? HDD with 5 TB,
> and
> it supports trim. However, a WD Elements 2.5? 4 TB does not. Perhaps
> because
> it is a cheaper series. Every laptop HDD of 2 (or even 1) TB is SMR.
>
> --
> Grüße | Greetings | Salut | Qapla’
> Please do not share anything from, with or about me on any social network.
>
> “It is hard to be a conquering hero when it is not in your nature.”
> – Captain Hans Geering, ’Allo ’Allo
>

Am Sun, Apr 16, 2023 at 01:22:32PM -0700 schrieb Mark Knecht:
> Frank,
> Thank you for the in-depth explanation.
>
> I need to do some study before commenting further other than to say
> so far I'm finding different comments depending on whether it's
> an SSD or an M.2 drive.

Uhm, I think you mix up some terms here. An M.2 drive *is* an SSD
(literally, as the name says, a solid state drive). By “SSD”, did you mean
the classic laptop form factor for SATA HDDs and SSDs?

Because M.2 is also only a physical form factor. It supports both NVMe and
SATA. While NVMe is more modern and better suited for solid state media and
their properties, in the end it is still only a data protocol to transfer
data to and fro.

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

„He who prefers security to freedom deserves to be a slave.“ – Aristotle

Mark Knecht wrote:
>
>
> On Sun, Apr 16, 2023 at 11:54?AM Dale <rdalek1967@gmail.com
> <mailto:rdalek1967@gmail.com>> wrote:
> >
> > Mark Knecht wrote:
> > > <<< SNIP >>>
> > > I guess I don't understand why you would put Knoppix in the boot
> > > partition vs somewhere else. Is this for some sort of recovery
> > > process you're comfortable with vs recovering from a bootable DVD?
> > >
> > > - Mark
> >
> > I'm wanting to be able to boot something from the hard drive in the
> > event the OS itself won't boot.  The other day I had to dig around and
> > find a bootable USB stick and also found a DVD.  Ended up with the DVD
> > working best.  I already have memtest on /boot.  Thing is, I very rarely
> > use it.  ;-)
>
> So in the scenario you are suggesting, is grub working, giving you a
> boot choice screen, and your new Gentoo install is not working so
> you want to choose Knoppix to repair whatever is wrong with 
> Gentoo? 
>
> If that's the case why shoehorn Knoppix into the boot partition
> vs just give it its own partition? 
>
>


Dang Mark, I hadn't thought of that.  <slaps forehead>  See, that's why
you need to stick around here.  ROFL  I wonder, can I do that with the
Gentoo LiveGUI too??  Anyone know?  I may need help getting Grub to see
those tho.  I let it do it's thing with my kernels but I have no idea on
pointing it to something else. 

Given I have a 500GB drive, I got plenty of space.  Heck, a 10GB
partition each is more than enough for either Knoppix or LiveGUI.  I
could even store info on there about drive partitions and scripts that I
use a lot.  Jeez, that's a idea. 

Thanks.

Dale

:-)  :-)  :-)  :-) 

P. S. Extra happy there.  ;-)

Am Sun, Apr 16, 2023 at 05:26:15PM -0500 schrieb Dale:

> > > I'm wanting to be able to boot something from the hard drive in the
> > > event the OS itself won't boot.  The other day I had to dig around and
> > > find a bootable USB stick and also found a DVD.  Ended up with the DVD
> > > working best.  I already have memtest on /boot.  Thing is, I very rarely
> > > use it.  ;-)
> >
> > So in the scenario you are suggesting, is grub working, giving you a
> > boot choice screen, and your new Gentoo install is not working so
> > you want to choose Knoppix to repair whatever is wrong with 
> > Gentoo? 
>
> Given I have a 500GB drive, I got plenty of space.  Heck, a 10GB
> partition each is more than enough for either Knoppix or LiveGUI.  I
> could even store info on there about drive partitions and scripts that I
> use a lot.  Jeez, that's a idea. 

Back in the day, I was annoyed that whenever I needed $LIVE_SYSTEM, I had to
reformat an entire USB stick for that. In times when you don’t even get
sticks below 8 GB anymore, I found it a waste of material and useful storage
space.

And then I found ventoy: https://www.ventoy.net/

It is a mini-Bootloader which you install once to a USB device, kind-of a
live system of its own. But when booting it, it dynamically scans the
content of its device and creates a new boot menu from it. So you can put
many ISOs on one device as simple files, delete them, upgrade them,
whatever, and then you can select one to boot from. Plus, the rest of the
stick remains usable as storage, unlike sticks that were dd’ed with an ISO.

--
Grüße | Greetings | Salut | Qapla’
Please do not share anything from, with or about me on any social network.

The four elements: earth, air and firewater.

On Sun, Apr 16, 2023 at 3:18?PM Frank Steinmetzger <Warp_7@gmx.de> wrote:
>
> Am Sun, Apr 16, 2023 at 01:22:32PM -0700 schrieb Mark Knecht:
> > Frank,
> > Thank you for the in-depth explanation.
> >
> > I need to do some study before commenting further other than to say
> > so far I'm finding different comments depending on whether it's
> > an SSD or an M.2 drive.
>
> Uhm, I think you mix up some terms here. An M.2 drive *is* an SSD
> (literally, as the name says, a solid state drive). By “SSD”, did you mean
> the classic laptop form factor for SATA HDDs and SSDs?
>
> Because M.2 is also only a physical form factor. It supports both NVMe and
> SATA. While NVMe is more modern and better suited for solid state media
and
> their properties, in the end it is still only a data protocol to transfer
> data to and fro.
>

No, I don't believe I've mixed them up but if you see something I'm wrong
about
let me know.

When I speak of SSDs I do mean devices that are marketed as SSDs &
probably use SATA.

When I speak of M.2 I mean what you and I both call M.2.

While SSD & M.2 are both Flash devices they don't provide the same info
when queried by smartctl which makes a direct comparison more
difficult.

Depending on the manufacturer & the foundry they build the chips in the
technologies in these devices can be quite different independent of whether
they are M.2 or SSD.

1) Dale's Samsung 870 EVO - V-NAND TLC (8-bits/cell) and 600TB written
2) My Crucial 1TB M.2 is QLC (16 bits/cell) and 450TB written
3) My Sabrent 1TB M.2 is TLC (8 bits/cell) and 700TB written
4) My Crucial 250GB is unknown because Crucial sells 5 versions
that come from different fabs and have different specs.

All 4 drives are warranted for 5 years or hitting the TB written value.

All 4 drives have 16K page sizes.

That said, I've been using the Crucial on my Kubuntu dual boot for over a
year and only have 28TB written so I'm a long way from the 450TB spec and
likely won't come close in 5 years. (If I'm even still using this machine.)

On the Windows side which I use far less I've only written about 2TB total.

In my case the workloads are generally fairly large files. They are
generally
either 24MB photo files for astrophotography or audio recording files which
are typically 50-100K. Neither of them are 'modified' and need to be
rewritten. They are either saved or deleted.

Whether the write amplification makes a difference or not in real life
I don't know. I'm sure for some work loads it does but the 'percent
used' value that smartctl returns is 2% for the Crucial and 0% for
the Sabrent so both appear to have a lot of life left in them.

Mark

Frank Steinmetzger wrote:
> Am Sun, Apr 16, 2023 at 05:26:15PM -0500 schrieb Dale:
>
>>>> I'm wanting to be able to boot something from the hard drive in the
>>>> event the OS itself won't boot.  The other day I had to dig around and
>>>> find a bootable USB stick and also found a DVD.  Ended up with the DVD
>>>> working best.  I already have memtest on /boot.  Thing is, I very rarely
>>>> use it.  ;-)
>>> So in the scenario you are suggesting, is grub working, giving you a
>>> boot choice screen, and your new Gentoo install is not working so
>>> you want to choose Knoppix to repair whatever is wrong with 
>>> Gentoo? 
>> Given I have a 500GB drive, I got plenty of space.  Heck, a 10GB
>> partition each is more than enough for either Knoppix or LiveGUI.  I
>> could even store info on there about drive partitions and scripts that I
>> use a lot.  Jeez, that's a idea. 
> Back in the day, I was annoyed that whenever I needed $LIVE_SYSTEM, I had to
> reformat an entire USB stick for that. In times when you don’t even get
> sticks below 8 GB anymore, I found it a waste of material and useful storage
> space.
>
> And then I found ventoy: https://www.ventoy.net/
>
> It is a mini-Bootloader which you install once to a USB device, kind-of a
> live system of its own. But when booting it, it dynamically scans the
> content of its device and creates a new boot menu from it. So you can put
> many ISOs on one device as simple files, delete them, upgrade them,
> whatever, and then you can select one to boot from. Plus, the rest of the
> stick remains usable as storage, unlike sticks that were dd’ed with an ISO.
>

My current install is over a decade old.  My /boot partition is about
375MBs.  I should have made it larger but at the time, I booted CD/DVD
media when needed.  I didn't have USB sticks at the time.  This time, I
plan to make some changes.  If I put Knoppix and/or Gentoo LiveGUI in
/boot, it will be larger.  Much larger.  Mark's idea is best tho.  If I
can get Grub to work and boot it. 

I'll look into your link more.  It sounds interesting but can't figure
out exactly how it works.  May check youtube for a video.  Should clear
up the muddy water. 

Dale

:-)  :-) 

On 17/04/2023 02:14, Dale wrote:
> My current install is over a decade old.  My /boot partition is about
> 375MBs.  I should have made it larger but at the time, I booted CD/DVD
> media when needed.  I didn't have USB sticks at the time.  This time, I
> plan to make some changes.  If I put Knoppix and/or Gentoo LiveGUI in
> /boot, it will be larger.  Much larger.  Mark's idea is best tho.  If I
> can get Grub to work and boot it.

If you dd your boot partition across, you can copy it into a larger
partition on the new drive, and then just expand the filesystem.

So changing partition sizes isn't a problem if you want to just copy
your system drive onto a new disk.

Cheers,
Wol

On Mon, Apr 17, 2023 at 2:41?AM Wols Lists <antlists@youngman.org.uk> wrote:
>
> On 17/04/2023 02:14, Dale wrote:
> > My current install is over a decade old. My /boot partition is about
> > 375MBs. I should have made it larger but at the time, I booted CD/DVD
> > media when needed. I didn't have USB sticks at the time. This time, I
> > plan to make some changes. If I put Knoppix and/or Gentoo LiveGUI in
> > /boot, it will be larger. Much larger. Mark's idea is best tho. If I
> > can get Grub to work and boot it.
>
> If you dd your boot partition across, you can copy it into a larger
> partition on the new drive, and then just expand the filesystem.
>
> So changing partition sizes isn't a problem if you want to just copy
> your system drive onto a new disk.
>
> Cheers,
> Wol

I'm not sure I'd use dd in this case. If he's moving from an HDD with
a 4K block size and a 4K file system block size to an SDD with a 16K
physical block size he might want to consider changing the filesystem
block size to 16K which should help on the write amplification side.

Maybe dd can do that but I wouldn't think so.

And I don't know that formatting ext4 or some other FS to 16K
really helps the write amplification issue but it makes sense to
me to match the file system blocks to the underlying flash
block size. Real speed testing would be required to ensure reading
16K blocks doesn't slow him down though.

Just a thought,
Mark

Mark Knecht wrote:
>
>
> On Mon, Apr 17, 2023 at 2:41?AM Wols Lists <antlists@youngman.org.uk
> <mailto:antlists@youngman.org.uk>> wrote:
> >
> > On 17/04/2023 02:14, Dale wrote:
> > > My current install is over a decade old.  My /boot partition is about
> > > 375MBs.  I should have made it larger but at the time, I booted CD/DVD
> > > media when needed.  I didn't have USB sticks at the time.  This
> time, I
> > > plan to make some changes.  If I put Knoppix and/or Gentoo LiveGUI in
> > > /boot, it will be larger.  Much larger.  Mark's idea is best tho. 
> If I
> > > can get Grub to work and boot it.
> >
> > If you dd your boot partition across, you can copy it into a larger
> > partition on the new drive, and then just expand the filesystem.
> >
> > So changing partition sizes isn't a problem if you want to just copy
> > your system drive onto a new disk.
> >
> > Cheers,
> > Wol
>
> I'm not sure I'd use dd in this case. If he's moving from an HDD with
> a 4K block size and a 4K file system block size to an SDD with a 16K 
> physical block size he might want to consider changing the filesystem 
> block size to 16K which should help on the write amplification side.
>
> Maybe dd can do that but I wouldn't think so.
>
> And I don't know that formatting ext4 or some other FS to 16K 
> really helps the write amplification issue but it makes sense to
> me to match the file system blocks to the underlying flash
> block size. Real speed testing would be required to ensure reading
> 16K blocks doesn't slow him down though.
>
> Just a thought,
> Mark


I still haven't got around to partitioning the drive or anything so I'm
glad you mentioned the block size.  I need to try and remember that.  It
may detect it itself but may not.  I'd rather fix it now than wish I did
later on.  I assume that setting is in the man page. 

Thanks for that tidbit.  Now to remember it.  :/

Dale

:-)  :-) 

P. s.  It's garden time for folks around here.  I been busy the past few
days.  Tractor and tiller too.

Am Mon, Apr 17, 2023 at 10:45:46AM -0700 schrieb Mark Knecht:

> And I don't know that formatting ext4 or some other FS to 16K
> really helps the write amplification issue but it makes sense to
> me to match the file system blocks to the underlying flash
> block size.

The problem is finding out the write block size. This 7-year-old post says
it’s reached 16 K: https://superuser.com/questions/976257/page-sizes-ssd

So I would say don’t bother. If everything is trimmed, there is no
amplification. And if the disk becomes full and you get WA when writing
itsy-bitsy 4 K files, you probably still won’t notice much difference, as
random 4 K writes are slow anyways and how often do you write thousands of
4 K files outside of portage?

Erase block sizes probably go into the megabytes these days:
https://news.ycombinator.com/item?id=29165202

Some more detailed explanation:
https://spdk.io/doc/ssd_internals.html
“For each erase block, each bit may be written to (i.e. have its bit
flipped from 0 to 1) with bit-granularity once. In order to write to the
erase block a second time, the entire block must be erased (i.e. all bits
in the block are flipped back to 0).”

This sounds like my initial statement was partially wrong – trimming does
cause writing zeroes, because that’s what an erase does. But it still
prevents write amplification (and one extra erase cycle) because
neighbouring blocks don’t need to be read and written back.

> Real speed testing would be required to ensure reading
> 16K blocks doesn't slow him down though.

Here are some numbers and a conclusion gathered from a read test:
https://superuser.com/questions/728858/how-to-determine-ssds-nand-erase-block-size

Unless I positively need the speed for high-performance computing, I’d
rather keep the smaller granularity for more capacity at low file sizes.

A problem is what some call “parts lottery” these days: manufacturers
promise some performance on the data sheet (“up to xxx”), but not with which
parts they want to achieve this (types of flash chips, TLC/QLC, controller,
DRAM and so on). Meaning during the lifetime of a product, its internals may
change and as a consequence those specs are not in the data sheet:

https://unix.stackexchange.com/questions/334804/is-there-a-way-to-find-out-ssd-page-size-on-linux-unix-what-is-physical-block
“There is no standard way for a SSD to report its page size or erase block
size. Few if any manufacturers report them in the datasheets. (Because
they may change during the lifetime of a SKU, for example because of
changing suppliers.)
For practical use just align all your data structures (partitions, payload
of LUKS containers, LVM logical volumes) to 1 or 2 MiB boundaries. It's an
SSD after all--it is designed to cope with usual filesystems, such as NTFS
(which uses 4 KiB allocation units).”

--
Grüße | Greetings | Qapla’
Please do not share anything from, with or about me on any social network.

The worst disease is indifference. So what?