Mailing List Archive

Is there a way to misconfigure USB ports in the kernel?
Greetings,

since my old 64 GB Verbatim USB sticks became too small, I bought two
new 128 GB Philips sticks. Because I need to read and write them on
both, a stand-alone Windows laptop (not connected to the internet) runn-
ing Windows Vista and Cygwin and my Gentoo laptop, I encrypted them with
old TrueCrypt on the Windows box, using them under Gentoo in TrueCrypt
compatibility mode.

This worked well with the Verbatim USB sticks (which probably are USB
2.0), but while reading the new USB 3.0 Philips USB sticks is signific-
antly faster than reading the old Verbatim USB sticks, writing to them
is slow as hell under Gentoo. And writing to the Philips USB sticks on
the old Vista laptop with USB 2.0 ports clearly outperforms writing to
them using the Gentoo laptop's USB 3.0 ports.

This could be a problem with TrueCrypt/VeraCrypt or with somehow miscon-
figured USB ports. To check for the latter I provide below all kernel
configuration variables I regard USB related in the hope that some know-
ledgable people might find a glitch in there:

$ grep -E 'Linux/|DEV_SD|SCSI|USB' /usr/src/linux/.config
# Linux/x86 5.4.72-gentoo Kernel Configuration
CONFIG_BLK_SCSI_REQUEST=y
CONFIG_BT_HCIBTUSB=y
# CONFIG_BT_HCIBTUSB_AUTOSUSPEND is not set
CONFIG_BT_HCIBTUSB_BCM=y
# CONFIG_BT_HCIBTUSB_MTK is not set
CONFIG_BT_HCIBTUSB_RTL=y
# CONFIG_BT_HCIBFUSB is not set
# CONFIG_MISC_RTSX_USB is not set
# SCSI device support
CONFIG_SCSI_MOD=y
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_SCSI_PROC_FS=y
# SCSI support type (disk, tape, CD-ROM)
CONFIG_BLK_DEV_SD=y
CONFIG_SCSI_CONSTANTS=y
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set
# SCSI Transports
CONFIG_SCSI_SPI_ATTRS=y
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# end of SCSI Transports
# CONFIG_SCSI_LOWLEVEL is not set
# CONFIG_SCSI_DH is not set
# end of SCSI device support
CONFIG_USB_NET_DRIVERS=y
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_RTL8152 is not set
# CONFIG_USB_LAN78XX is not set
CONFIG_USB_USBNET=y
CONFIG_USB_NET_AX8817X=y
CONFIG_USB_NET_AX88179_178A=y
CONFIG_USB_NET_CDCETHER=y
CONFIG_USB_NET_CDC_EEM=y
CONFIG_USB_NET_CDC_NCM=y
# CONFIG_USB_NET_HUAWEI_CDC_NCM is not set
# CONFIG_USB_NET_CDC_MBIM is not set
# CONFIG_USB_NET_DM9601 is not set
# CONFIG_USB_NET_SR9700 is not set
# CONFIG_USB_NET_SR9800 is not set
# CONFIG_USB_NET_SMSC75XX is not set
# CONFIG_USB_NET_SMSC95XX is not set
# CONFIG_USB_NET_GL620A is not set
CONFIG_USB_NET_NET1080=y
# CONFIG_USB_NET_PLUSB is not set
# CONFIG_USB_NET_MCS7830 is not set
CONFIG_USB_NET_RNDIS_HOST=y
CONFIG_USB_NET_CDC_SUBSET_ENABLE=y
CONFIG_USB_NET_CDC_SUBSET=y
# CONFIG_USB_ALI_M5632 is not set
# CONFIG_USB_AN2720 is not set
CONFIG_USB_BELKIN=y
CONFIG_USB_ARMLINUX=y
# CONFIG_USB_EPSON2888 is not set
# CONFIG_USB_KC2190 is not set
CONFIG_USB_NET_ZAURUS=y
# CONFIG_USB_NET_CX82310_ETH is not set
# CONFIG_USB_NET_KALMIA is not set
# CONFIG_USB_NET_QMI_WWAN is not set
# CONFIG_USB_HSO is not set
# CONFIG_USB_NET_INT51X1 is not set
# CONFIG_USB_IPHETH is not set
# CONFIG_USB_SIERRA_NET is not set
# CONFIG_USB_VL600 is not set
# CONFIG_USB_NET_CH9200 is not set
# CONFIG_USB_NET_AQC111 is not set
# CONFIG_USB_NET_RNDIS_WLAN is not set
# CONFIG_MOUSE_SYNAPTICS_USB is not set
# CONFIG_TABLET_USB_ACECAD is not set
# CONFIG_TABLET_USB_AIPTEK is not set
# CONFIG_TABLET_USB_GTCO is not set
# CONFIG_TABLET_USB_HANWANG is not set
# CONFIG_TABLET_USB_KBTAB is not set
# CONFIG_TABLET_USB_PEGASUS is not set
# CONFIG_TOUCHSCREEN_USB_COMPOSITE is not set
# CONFIG_I2C_TINY_USB is not set
# USB-based Watchdog Cards
# CONFIG_USBPCWATCHDOG is not set
CONFIG_SND_USB=y
CONFIG_SND_USB_AUDIO=y
# CONFIG_SND_USB_UA101 is not set
# CONFIG_SND_USB_USX2Y is not set
# CONFIG_SND_USB_CAIAQ is not set
# CONFIG_SND_USB_US122L is not set
# CONFIG_SND_USB_6FIRE is not set
# CONFIG_SND_USB_HIFACE is not set
# CONFIG_SND_USB_POD is not set
# CONFIG_SND_USB_PODHD is not set
# CONFIG_SND_USB_TONEPORT is not set
# CONFIG_SND_USB_VARIAX is not set
# USB HID support
CONFIG_USB_HID=y
CONFIG_USB_HIDDEV=y
# end of USB HID support
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
CONFIG_USB_SUPPORT=y
CONFIG_USB_COMMON=y
# CONFIG_USB_LED_TRIG is not set
# CONFIG_USB_ULPI_BUS is not set
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB=y
CONFIG_USB_PCI=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
# Miscellaneous USB options
CONFIG_USB_DEFAULT_PERSIST=y
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_OTG is not set
# CONFIG_USB_OTG_WHITELIST is not set
# CONFIG_USB_OTG_BLACKLIST_HUB is not set
# CONFIG_USB_LEDS_TRIGGER_USBPORT is not set
CONFIG_USB_AUTOSUSPEND_DELAY=2
CONFIG_USB_MON=y
# USB Host Controller Drivers
# CONFIG_USB_C67X00_HCD is not set
CONFIG_USB_XHCI_HCD=y
# CONFIG_USB_XHCI_DBGCAP is not set
CONFIG_USB_XHCI_PCI=y
# CONFIG_USB_XHCI_PLATFORM is not set
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_EHCI_PCI=y
# CONFIG_USB_EHCI_FSL is not set
# CONFIG_USB_EHCI_HCD_PLATFORM is not set
# CONFIG_USB_OXU210HP_HCD is not set
# CONFIG_USB_ISP116X_HCD is not set
# CONFIG_USB_FOTG210_HCD is not set
CONFIG_USB_OHCI_HCD=y
CONFIG_USB_OHCI_HCD_PCI=y
# CONFIG_USB_OHCI_HCD_PLATFORM is not set
CONFIG_USB_UHCI_HCD=y
# CONFIG_USB_SL811_HCD is not set
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_HCD_TEST_MODE is not set
# USB Device Class drivers
CONFIG_USB_ACM=y
# CONFIG_USB_PRINTER is not set
CONFIG_USB_WDM=y
# CONFIG_USB_TMC is not set
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
# also be needed; see USB_STORAGE Help for more info
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_REALTEK is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_USBAT is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_STORAGE_ALAUDA is not set
# CONFIG_USB_STORAGE_ONETOUCH is not set
# CONFIG_USB_STORAGE_KARMA is not set
# CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
# CONFIG_USB_STORAGE_ENE_UB6250 is not set
# CONFIG_USB_UAS is not set
# USB Imaging devices
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set
# CONFIG_USBIP_CORE is not set
# CONFIG_USB_CDNS3 is not set
# CONFIG_USB_MUSB_HDRC is not set
# CONFIG_USB_DWC3 is not set
# CONFIG_USB_DWC2 is not set
# CONFIG_USB_CHIPIDEA is not set
# CONFIG_USB_ISP1760 is not set
# USB port drivers
CONFIG_USB_SERIAL=y
# CONFIG_USB_SERIAL_CONSOLE is not set
# CONFIG_USB_SERIAL_GENERIC is not set
# CONFIG_USB_SERIAL_SIMPLE is not set
# CONFIG_USB_SERIAL_AIRCABLE is not set
# CONFIG_USB_SERIAL_ARK3116 is not set
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_CH341 is not set
# CONFIG_USB_SERIAL_WHITEHEAT is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_CP210X is not set
# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
# CONFIG_USB_SERIAL_EMPEG is not set
# CONFIG_USB_SERIAL_FTDI_SIO is not set
# CONFIG_USB_SERIAL_VISOR is not set
# CONFIG_USB_SERIAL_IPAQ is not set
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_F81232 is not set
# CONFIG_USB_SERIAL_F8153X is not set
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
# CONFIG_USB_SERIAL_IUU is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
# CONFIG_USB_SERIAL_KEYSPAN is not set
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not set
# CONFIG_USB_SERIAL_METRO is not set
# CONFIG_USB_SERIAL_MOS7720 is not set
# CONFIG_USB_SERIAL_MOS7840 is not set
# CONFIG_USB_SERIAL_MXUPORT is not set
# CONFIG_USB_SERIAL_NAVMAN is not set
CONFIG_USB_SERIAL_PL2303=y
# CONFIG_USB_SERIAL_OTI6858 is not set
# CONFIG_USB_SERIAL_QCAUX is not set
# CONFIG_USB_SERIAL_QUALCOMM is not set
# CONFIG_USB_SERIAL_SPCP8X5 is not set
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_SIERRAWIRELESS is not set
# CONFIG_USB_SERIAL_SYMBOL is not set
# CONFIG_USB_SERIAL_TI is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
# CONFIG_USB_SERIAL_OPTION is not set
# CONFIG_USB_SERIAL_OMNINET is not set
# CONFIG_USB_SERIAL_OPTICON is not set
# CONFIG_USB_SERIAL_XSENS_MT is not set
# CONFIG_USB_SERIAL_WISHBONE is not set
# CONFIG_USB_SERIAL_SSU100 is not set
# CONFIG_USB_SERIAL_QT2 is not set
# CONFIG_USB_SERIAL_UPD78F0730 is not set
# CONFIG_USB_SERIAL_DEBUG is not set
# USB Miscellaneous drivers
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_ADUTUX is not set
# CONFIG_USB_SEVSEG is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_FTDI_ELAN is not set
# CONFIG_USB_APPLEDISPLAY is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
# CONFIG_USB_EHSET_TEST_FIXTURE is not set
# CONFIG_USB_ISIGHTFW is not set
# CONFIG_USB_YUREX is not set
# CONFIG_USB_EZUSB_FX2 is not set
# CONFIG_USB_HUB_USB251XB is not set
# CONFIG_USB_HSIC_USB3503 is not set
# CONFIG_USB_HSIC_USB4604 is not set
# CONFIG_USB_LINK_LAYER_TEST is not set
# CONFIG_USB_CHAOSKEY is not set
# USB Physical Layer drivers
# CONFIG_NOP_USB_XCEIV is not set
# CONFIG_USB_ISP1301 is not set
# end of USB Physical Layer drivers
CONFIG_USB_GADGET=y
# CONFIG_USB_GADGET_DEBUG is not set
# CONFIG_USB_GADGET_DEBUG_FILES is not set
# CONFIG_USB_GADGET_DEBUG_FS is not set
CONFIG_USB_GADGET_VBUS_DRAW=2
CONFIG_USB_GADGET_STORAGE_NUM_BUFFERS=2
# USB Peripheral Controller
# CONFIG_USB_FOTG210_UDC is not set
# CONFIG_USB_GR_UDC is not set
# CONFIG_USB_R8A66597 is not set
# CONFIG_USB_PXA27X is not set
# CONFIG_USB_MV_UDC is not set
# CONFIG_USB_MV_U3D is not set
# CONFIG_USB_M66592 is not set
# CONFIG_USB_BDC_UDC is not set
# CONFIG_USB_AMD5536UDC is not set
# CONFIG_USB_NET2272 is not set
# CONFIG_USB_NET2280 is not set
# CONFIG_USB_GOKU is not set
# CONFIG_USB_EG20T is not set
# CONFIG_USB_DUMMY_HCD is not set
# end of USB Peripheral Controller
CONFIG_USB_LIBCOMPOSITE=y
CONFIG_USB_U_ETHER=y
CONFIG_USB_F_ECM=y
CONFIG_USB_F_SUBSET=y
CONFIG_USB_F_RNDIS=y
# CONFIG_USB_CONFIGFS is not set
# CONFIG_USB_ZERO is not set
# CONFIG_USB_AUDIO is not set
CONFIG_USB_ETH=y
CONFIG_USB_ETH_RNDIS=y
# CONFIG_USB_ETH_EEM is not set
# CONFIG_USB_G_NCM is not set
# CONFIG_USB_GADGETFS is not set
# CONFIG_USB_FUNCTIONFS is not set
# CONFIG_USB_MASS_STORAGE is not set
# CONFIG_USB_G_SERIAL is not set
# CONFIG_USB_MIDI_GADGET is not set
# CONFIG_USB_G_PRINTER is not set
# CONFIG_USB_CDC_COMPOSITE is not set
# CONFIG_USB_G_ACM_MS is not set
# CONFIG_USB_G_MULTI is not set
# CONFIG_USB_G_HID is not set
# CONFIG_USB_G_DBGP is not set
# CONFIG_USB_ROLE_SWITCH is not set
# LED driver for blink(1) USB RGB LED is under Special HID drivers (HID_THINGM)
# CONFIG_BCM_KONA_USB2_PHY is not set
# CONFIG_PHY_PXA_28NM_USB2 is not set
CONFIG_EARLY_PRINTK_USB=y
# CONFIG_EARLY_PRINTK_USB_XDBC is not set
$

Any help appreciated :-)

Sincerely,
Rainer
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Wednesday, 25 November 2020 17:37:15 GMT Dr Rainer Woitok wrote:
> Greetings,
>
> since my old 64 GB Verbatim USB sticks became too small, I bought two
> new 128 GB Philips sticks. Because I need to read and write them on
> both, a stand-alone Windows laptop (not connected to the internet) runn-
> ing Windows Vista and Cygwin and my Gentoo laptop, I encrypted them with
> old TrueCrypt on the Windows box, using them under Gentoo in TrueCrypt
> compatibility mode.
>
> This worked well with the Verbatim USB sticks (which probably are USB
> 2.0), but while reading the new USB 3.0 Philips USB sticks is signific-
> antly faster than reading the old Verbatim USB sticks, writing to them
> is slow as hell under Gentoo. And writing to the Philips USB sticks on
> the old Vista laptop with USB 2.0 ports clearly outperforms writing to
> them using the Gentoo laptop's USB 3.0 ports.
>
> This could be a problem with TrueCrypt/VeraCrypt or with somehow miscon-
> figured USB ports. To check for the latter I provide below all kernel
> configuration variables I regard USB related in the hope that some know-
> ledgable people might find a glitch in there:

Check dmesg to see if initialisation of the USB 3.0 drive throws up any
errors. Then check 'lsusb -t' to make sure it has been recognised as a USB
3.0.

If write operations without TrueCrypt/VeraCrypt are equally slow, then
obviously the problem is not with encryption.

I've read in a number of articles the erase block size on most USB flash
(NAND) is 128KB, which incurs a lot of operations on a write, when using Linux
with its 4K size sectors. Partitioning the USB drive to use 128KB sectors and
then aligning the fs on it should improve matters.

I found this article which mentions an experiment with ext4 fs. A more
effective search should hopefully bring up examples on FAT fs.

HTH.
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Thursday, 26 November 2020 00:10:00 GMT Michael wrote:
> On Wednesday, 25 November 2020 17:37:15 GMT Dr Rainer Woitok wrote:
> > Greetings,
> >
> > since my old 64 GB Verbatim USB sticks became too small, I bought two
> > new 128 GB Philips sticks. Because I need to read and write them on
> > both, a stand-alone Windows laptop (not connected to the internet) runn-
> > ing Windows Vista and Cygwin and my Gentoo laptop, I encrypted them with
> > old TrueCrypt on the Windows box, using them under Gentoo in TrueCrypt
> > compatibility mode.
> >
> > This worked well with the Verbatim USB sticks (which probably are USB
> > 2.0), but while reading the new USB 3.0 Philips USB sticks is signific-
> > antly faster than reading the old Verbatim USB sticks, writing to them
> > is slow as hell under Gentoo. And writing to the Philips USB sticks on
> > the old Vista laptop with USB 2.0 ports clearly outperforms writing to
> > them using the Gentoo laptop's USB 3.0 ports.
> >
> > This could be a problem with TrueCrypt/VeraCrypt or with somehow miscon-
> > figured USB ports. To check for the latter I provide below all kernel
> > configuration variables I regard USB related in the hope that some know-
>
> > ledgable people might find a glitch in there:
> Check dmesg to see if initialisation of the USB 3.0 drive throws up any
> errors. Then check 'lsusb -t' to make sure it has been recognised as a USB
> 3.0.
>
> If write operations without TrueCrypt/VeraCrypt are equally slow, then
> obviously the problem is not with encryption.
>
> I've read in a number of articles the erase block size on most USB flash
> (NAND) is 128KB, which incurs a lot of operations on a write, when using
> Linux with its 4K size sectors. Partitioning the USB drive to use 128KB
> sectors and then aligning the fs on it should improve matters.
>
> I found this article which mentions an experiment with ext4 fs. A more
> effective search should hopefully bring up examples on FAT fs.
>
> HTH.

Apologies, I seem to have forgotten to include the link. Here's another link
I came across today and which offers more detail on this topic:

http://wiki.laptop.org/go/How_to_Damage_a_FLASH_Storage_Device
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
Michael,

On Thursday, 2020-11-26 00:10:00 +0000, you wrote:

> ...
> Check dmesg to see if initialisation of the USB 3.0 drive throws up any
> errors.

No errors.

> Then check 'lsusb -t' to make sure it has been recognised as a USB
> 3.0.

"lsusb -tv" showed the stick to be USB 3.0.

> ...
> Partitioning the USB drive to use 128KB sectors and
> then aligning the fs on it should improve matters.

Since the USB sticks contain symbolic links and have to be accessible
from both, Linux and Windows they are NTFS formatted, and according to
"mkntfs(8)" the sector size can be at most 4096, while the cluster size
is limited to 2097152, that is 2G. However, when NTFS formatting an USB
stick from within TrueCrypt/VeraCrypt or directly in Windows the maximum
cluster size is 64K, with the only difference that Windows calls it
"allocation unit size".

So I think above you were talking about 128K clusters rather than sect-
ors. I'll give that a try and will reformat the USB sticks using the
maximum cluster size of 64K. But I don't see a way to "align" the file
system on these USB sticks.

> I found this article which mentions an experiment with ext4 fs.

Thanks for the link you sent in your other mail and thanks for pointing
all this out :-)

Sincerely,
Rainer
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
Hi Rainer,

On Friday, 27 November 2020 16:01:29 GMT Dr Rainer Woitok wrote:

> Since the USB sticks contain symbolic links and have to be accessible
> from both, Linux and Windows they are NTFS formatted, and according to
> "mkntfs(8)" the sector size can be at most 4096, while the cluster size
> is limited to 2097152, that is 2G. However, when NTFS formatting an USB
> stick from within TrueCrypt/VeraCrypt or directly in Windows the maximum
> cluster size is 64K, with the only difference that Windows calls it
> "allocation unit size".

Ohh! STOP RIGHT THERE! :-)

I mistakenly thought you were using FAT. NTFS on linux uses the ntfs-3g
driver, which relies on FUSE. This 'Filesystem in Userspace' is inevitably
slower than kernel filesystem drivers, because it has to jump through hoops
and libs, acting as a virtual filesystem. CPU usage will also be higher as a
result, than when using a native kernel filesystem driver.

A 4k block size is recommended for ntfs-3g which is the default sector created
by fdisk and friends on Linux these days. This will align your partition
optimally. In addition, mkfs.ntfs will use 4096 bytes as the default cluster
size, so you should be good in that respect.

Another setting you may want to try is mounting the USB with 'big_writes' -
check the man page. This should help particularly with large files, which
will use larger blocks up to 128KB when copying data to the NTFS.

Also, read the FAQs under the heading "Performance" for more useful
information:

https://www.tuxera.com/community/ntfs-3g-faq/

Hope this helps.
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
Michael,

On Friday, 2020-11-27 19:07:17 +0000, you wrote:

> ...
> A 4k block size is recommended for ntfs-3g which is the default sector created
> by fdisk and friends on Linux these days. This will align your partition
> optimally. In addition, mkfs.ntfs will use 4096 bytes as the default cluster
> size, so you should be good in that respect.
>
> Another setting you may want to try is mounting the USB with 'big_writes' -
> check the man page. This should help particularly with large files, which
> will use larger blocks up to 128KB when copying data to the NTFS.

Both, the VeraCrypt command line (--fs-options=big_writes) and the Vera-
Crypt GUI (under "Settings --> Preferences") allow setting this mount
option. But

$ mount | grep veracrypt

never shows it, initially causing me to erroneously believe it wasn't
set and to try finding on the web another way of setting it. By pure
chance I finally found out that

$ ps -ef | grep veracrypt

lists a "/usr/sbin/mount.ntfs" task which shows the options really in
effect. However, I haven't yet had the time to test the effect of this
option when writing plenty of really big files. I will report on that
later.

Sincerely,
Rainer
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote:

("> >" refers to Michael <confabulate@kintzios.com>)

> Michael,
>
> On Friday, 2020-11-27 19:07:17 +0000, you wrote:
>
> > ...
> > A 4k block size is recommended for ntfs-3g which is the default sector created
> > by fdisk and friends on Linux these days. This will align your partition
> > optimally. In addition, mkfs.ntfs will use 4096 bytes as the default cluster
> > size, so you should be good in that respect.
> >
> > Another setting you may want to try is mounting the USB with 'big_writes' -
> > check the man page. This should help particularly with large files, which
> > will use larger blocks up to 128KB when copying data to the NTFS.
>
> Both, the VeraCrypt command line (--fs-options=big_writes) and the Vera-
> Crypt GUI (under "Settings --> Preferences") allow setting this mount
> option. But
>
> $ mount | grep veracrypt
>
> never shows it, initially causing me to erroneously believe it wasn't
> set and to try finding on the web another way of setting it. By pure
> chance I finally found out that
>
> $ ps -ef | grep veracrypt
>
> lists a "/usr/sbin/mount.ntfs" task which shows the options really in
> effect. However, I haven't yet had the time to test the effect of this
> option when writing plenty of really big files. I will report on that
> later.

Well, it's been quite a while, due to my being almost permanently con-
fronted with more pressing tasks ... :-(

To sum up my experience with my new 128 GB Philips USB 3.0 sticks: while
the Philips sticks are significantly faster for reading operations than
my old 64 GB Verbatim ones (probably USB 2.0), writing operations to the
Philips sticks are unbearably slow, regardless of whether I created a
normal unencrypted NTFS filesystem on them or an encrypted NTFS filesys-
tem using VeraCrypt. Writing to the USB stick while at the same time
reading from it in a different terminal window caused commands like "cd"
or "ls" to simply stall. Thus while running

$ cp --preserve=timestamps -ru $source_dir .

in one terminal window, I ran

$ while true
> do n=$(ps -ef|g 'cp --preserve'|g -v grep)
> if [[ "$n" = "${o-}" ]]
> then sleep 10
> else o="$n"
> echo "$n"
> fi
> done

in another, to get the wall clock times when copying a new file began.
That way I found that copying a 30 MB file took about 40 minutes.

So what are my options?

- Stay away from Philips USB 3.0 sticks?

- Stay away from Philips USB sticks in general?

- Stay away from USB 3.0 sticks in general?

- Stay away from Filesystem in User Space using a non-stable 5.10 or
5.11 kernel (currently I'm using stable 5.4.97)?

- Stay away from Gentoo?

- Stay away from Linux in general and go back to OTOS (aka the Only
True Operating System aka Windoze)?

- ...?

Any ideas and comments welcome ...

Sincerely,
Rainer
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
Dr Rainer Woitok wrote:
>
> Well, it's been quite a while, due to my being almost permanently con-
> fronted with more pressing tasks ... :-(
>
> To sum up my experience with my new 128 GB Philips USB 3.0 sticks: while
> the Philips sticks are significantly faster for reading operations than
> my old 64 GB Verbatim ones (probably USB 2.0), writing operations to the
> Philips sticks are unbearably slow, regardless of whether I created a
> normal unencrypted NTFS filesystem on them or an encrypted NTFS filesys-
> tem using VeraCrypt. Writing to the USB stick while at the same time
> reading from it in a different terminal window caused commands like "cd"
> or "ls" to simply stall. Thus while running
>
> $ cp --preserve=timestamps -ru $source_dir .
>
> in one terminal window, I ran
>
> $ while true
> > do n=$(ps -ef|g 'cp --preserve'|g -v grep)
> > if [[ "$n" = "${o-}" ]]
> > then sleep 10
> > else o="$n"
> > echo "$n"
> > fi
> > done
>
> in another, to get the wall clock times when copying a new file began.
> That way I found that copying a 30 MB file took about 40 minutes.
>
> So what are my options?
>
> - Stay away from Philips USB 3.0 sticks?
>
> - Stay away from Philips USB sticks in general?
>
> - Stay away from USB 3.0 sticks in general?
>
> - Stay away from Filesystem in User Space using a non-stable 5.10 or
> 5.11 kernel (currently I'm using stable 5.4.97)?
>
> - Stay away from Gentoo?
>
> - Stay away from Linux in general and go back to OTOS (aka the Only
> True Operating System aka Windoze)?
>
> - ...?
>
> Any ideas and comments welcome ...
>
> Sincerely,
> Rainer
>
>


Have you tried using dd to test the speed?  It doesn't even need a file
system as it writes directly to the device.  I've done this in the past
and it tells if it is a file system issue or a hardware issue.  One
thing here, it will destroy ANY and ALL data on it if you let it run
until it finishes.  Make sure you have nothing you want to save on it
when doing this.  I might add, I like this over rsync --progress because
it doesn't have a file system in the middle. 

Commands I use to test this. 

dd if=/dev/zero of=/dev/sd bs=4k conv=notrunc oflag=direct  #disables cache

Don't forget to complete the disk device in the "of=" section and you
can leave off the comment as well when typing it in.  You may also want
to adjust the other settings or leave out some to test things.  See if
you can figure out a way to tweek it a bit.  Sometimes you can add those
to the mount options to make it work better, so I've read anyway.

To monitor the speed, I found this years ago and it still worked a few
months ago when I was testing a stick that was giving me problems. 
Turned out, the stick was dying a slow death.  The dd test finished it,
with errors. 

watch -n 10 kill -USR1 <pid of process>

I use Konsole here which has tabs but you may use something similar that
will work just as well.  I start the dd command in one tab, then go to
other tab and find the process number for dd there with ps and grep.  I
then use the above and replace "<pid of process>" with the number, leave
out the <> as well, just the number itself.  When you go to the tab
where dd is running, it updates about every ten seconds with speed and
other info.  I've found it normal to start out fast and then slow down
as it goes. Since it has no moving parts, no clue why it does that. 

Someone else may see this and have a much better method but that has
worked for me in the past.  As we know, some USB stick and other memory
type cards can get fussy or go bad. 

Hope that helps or at least gives some ideas.

Dale

:-)  :-)

P. S.  I just copy what folks post, I don't claim to understand the
inner workings of this.  LOL  I do know, it destroys data tho.  Always
be careful with dd and if and 0. 
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On March 30, 2021 10:11:56 AM PDT, Dr Rainer Woitok <rainer.woitok@gmail.com> wrote:
>On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote:
>
>("> >" refers to Michael <confabulate@kintzios.com>)
>
>> Michael,
>>
>> On Friday, 2020-11-27 19:07:17 +0000, you wrote:
>>
>> > ...
>> > A 4k block size is recommended for ntfs-3g which is the default
>sector created
>> > by fdisk and friends on Linux these days. This will align your
>partition
>> > optimally. In addition, mkfs.ntfs will use 4096 bytes as the
>default cluster
>> > size, so you should be good in that respect.
>> >
>> > Another setting you may want to try is mounting the USB with
>'big_writes' -
>> > check the man page. This should help particularly with large
>files, which
>> > will use larger blocks up to 128KB when copying data to the NTFS.
>>
>> Both, the VeraCrypt command line (--fs-options=big_writes) and the
>Vera-
>> Crypt GUI (under "Settings --> Preferences") allow setting this
>mount
>> option. But
>>
>> $ mount | grep veracrypt
>>
>> never shows it, initially causing me to erroneously believe it
>wasn't
>> set and to try finding on the web another way of setting it. By
>pure
>> chance I finally found out that
>>
>> $ ps -ef | grep veracrypt
>>
>> lists a "/usr/sbin/mount.ntfs" task which shows the options really
>in
>> effect. However, I haven't yet had the time to test the effect of
>this
>> option when writing plenty of really big files. I will report on
>that
>> later.
>
>Well, it's been quite a while, due to my being almost permanently
>con-
>fronted with more pressing tasks ... :-(
>
>To sum up my experience with my new 128 GB Philips USB 3.0 sticks:
>while
>the Philips sticks are significantly faster for reading operations
>than
>my old 64 GB Verbatim ones (probably USB 2.0), writing operations to
>the
>Philips sticks are unbearably slow, regardless of whether I created
>a
>normal unencrypted NTFS filesystem on them or an encrypted NTFS
>filesys-
>tem using VeraCrypt. Writing to the USB stick while at the same
>time
>reading from it in a different terminal window caused commands like
>"cd"
>or "ls" to simply stall. Thus while running
>
> $ cp --preserve=timestamps -ru $source_dir .
>
>in one terminal window, I ran
>
> $ while true
> > do n=$(ps -ef|g 'cp --preserve'|g -v grep)
> > if [[ "$n" = "${o-}" ]]
> > then sleep 10
> > else o="$n"
> > echo "$n"
> > fi
> > done
>
>in another, to get the wall clock times when copying a new file
>began.
>That way I found that copying a 30 MB file took about 40 minutes.
>
>So what are my options?
>
> - Stay away from Philips USB 3.0 sticks?
>
> - Stay away from Philips USB sticks in general?
>
> - Stay away from USB 3.0 sticks in general?
>
> - Stay away from Filesystem in User Space using a non-stable 5.10 or
> 5.11 kernel (currently I'm using stable 5.4.97)?
>
> - Stay away from Gentoo?
>
> - Stay away from Linux in general and go back to OTOS (aka the Only
> True Operating System aka Windoze)?
>
> - ...?
>
>Any ideas and comments welcome ...
>
>Sincerely,
> Rainer

There are a number of things which might be going on here.
To start with, you can get the kernel, user, and wall clock run times for commands by prefixing it with "time". So:

time cp <large file> <USB mountpoint>

Will get you more precise answers with much less effort.

As for the performance of the USB drive in question, there are a few things that might be tripping it up.

Firstly, writing flash memory is significantly slower than reading it. Some drives deal with this by having some kind of internal cache mechanism. Many deal with it by using a pile of smaller chips instead of one big one and striping the writes. If the Phillips drive just used a few large chips instead, then it's just slow to write to and there isn't much you can do about it. I've seen a lot of cheaper drives that are that way.

Double check that the alignment and block size are correct for the drive's internal structure. That can cause some pretty massive performance hits if it's incorrect.

You can also check the output of the dmesg command for any errors the system is encountering with regard to the drive.

I don't know of any reason to stay away from usb 3.0 on Linux, but if you have USB 3 devices on both ends and try to hook them together with a USB 2.0 or 1.1 rated cable that could easily cause some problems... I assume you're plugging the drive straight into the machine's socket. If you're using the front panel though try one of the ones on the back. There may be something up with the case wiring.

I've never had a Phillips USB stick, so maybe do some tests with another brand of stick and see if it has the same problem. Kingston or SanDisk or something. One of the ones where memory is their primary focus.

You could definitely check performance on a different OS. There may be driver-related performance issues on this model of drive or even this specific drive.

Instead of NTFS you could also try UDF. It's supported on both Linux and Windows Vista and newer. (And even XP I think, but only an older revision of the filesystem.) Being an open standard it is quite a bit more stable on Linux than NTFS, doesn't need a userspace driver, and still supports larger drives and files. You can find Linux and Windows formatting instructions on the web pretty easily. Big thing is that on Windows it's available via the format command, but not in the GUI.

That's all I can think of at the moment. Hope it helps.

LMP
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Tuesday, 30 March 2021 18:11:56 BST Dr Rainer Woitok wrote:
> On Saturday, 2020-12-05 19:07:51 +0100, I myself wrote:
>
> ("> >" refers to Michael <confabulate@kintzios.com>)
>
> > Michael,
> >
> > On Friday, 2020-11-27 19:07:17 +0000, you wrote:
> > > ...
> > > A 4k block size is recommended for ntfs-3g which is the default sector
> > > created by fdisk and friends on Linux these days. This will align your
> > > partition optimally. In addition, mkfs.ntfs will use 4096 bytes as the
> > > default cluster size, so you should be good in that respect.
> > >
> > > Another setting you may want to try is mounting the USB with
> > > 'big_writes' -
> > > check the man page. This should help particularly with large files,
> > > which
> > > will use larger blocks up to 128KB when copying data to the NTFS.
> >
> > Both, the VeraCrypt command line (--fs-options=big_writes) and the Vera-
> > Crypt GUI (under "Settings --> Preferences") allow setting this mount
> > option. But
> >
> > $ mount | grep veracrypt
> >
> > never shows it, initially causing me to erroneously believe it wasn't
> > set and to try finding on the web another way of setting it. By pure
> > chance I finally found out that
> >
> > $ ps -ef | grep veracrypt
> >
> > lists a "/usr/sbin/mount.ntfs" task which shows the options really in
> > effect. However, I haven't yet had the time to test the effect of this
> > option when writing plenty of really big files. I will report on that
> > later.
>
> Well, it's been quite a while, due to my being almost permanently con-
> fronted with more pressing tasks ... :-(
>
> To sum up my experience with my new 128 GB Philips USB 3.0 sticks: while
> the Philips sticks are significantly faster for reading operations than
> my old 64 GB Verbatim ones (probably USB 2.0), writing operations to the
> Philips sticks are unbearably slow, regardless of whether I created a
> normal unencrypted NTFS filesystem on them or an encrypted NTFS filesys-
> tem using VeraCrypt. Writing to the USB stick while at the same time
> reading from it in a different terminal window caused commands like "cd"
> or "ls" to simply stall. Thus while running
>
> $ cp --preserve=timestamps -ru $source_dir .
>
> in one terminal window, I ran
>
> $ while true
>
> > do n=$(ps -ef|g 'cp --preserve'|g -v grep)
> >
> > if [[ "$n" = "${o-}" ]]
> > then sleep 10
> > else o="$n"
> >
> > echo "$n"
> >
> > fi
> >
> > done
>
> in another, to get the wall clock times when copying a new file began.
> That way I found that copying a 30 MB file took about 40 minutes.

OK, unless you made a typo and the "minutes" were meant to say seconds, this
is ridiculously slow.

You could run some tests to see what is causing the delay. The veracrypt
algos & cipher iterations, the fuse based ntfs-3g, or the USB stick's
controller.

However if, as I understand it, all other variables are the same and the only
change was to replace your Verbatim 64G USB 2.0 sticks with Philips 128G USB
3.0 sticks, then the slow writes point to the Philips devices being the
culprit.

Some years ago I tested some USB 2.0 sticks of various sizes, from 256M up to
32G and recall the smaller the USB stick the faster the write performance, so
differences in writing speed are normal. The writing speed you're describing
however is a clear indication of something being wrong.


> So what are my options?
>
> - Stay away from Philips USB 3.0 sticks?
>
> - Stay away from Philips USB sticks in general?

Without knowing the internals, a brand may offer only an unwarranted
assumption of performance. We saw Western Digital disks being sold as CMR,
while having SMR internals. A brand could switch OEM suppliers, or
components, making benchmarking unreliable.


> - Stay away from USB 3.0 sticks in general?

USB 3.0 is faster and USB 3.2 when available will be even faster. So use
whatever the USB ports on your PC offer.


> - Stay away from Filesystem in User Space using a non-stable 5.10 or
> 5.11 kernel (currently I'm using stable 5.4.97)?
>
> - Stay away from Gentoo?
>
> - Stay away from Linux in general and go back to OTOS (aka the Only
> True Operating System aka Windoze)?
>
> - ...?

In-kernel fs drivers are measurably faster than fuse based fs for well
understood reasons. However, if needs must and the fs you require is not
available on Linux, then some compromise will be required.


> Any ideas and comments welcome ...
>
> Sincerely,
> Rainer

You may want to run some tests on the sticks you have, if only to bottom out
what their performance is on different PCs and USB ports:

dd if=/dev/zero of=/run/media/<USER>/<STICK_LABEL>/TESTFILE bs=512
count=600000 oflag=direct conv=notrunc,fsync status=progress

Use a large enough file to make sure the USB controller cache gets saturated.

You could use a ramdisk/tmpfs as an input file.

If you write directly to the device as Dale suggested it will wipe data, so
keep a backup of anything you need first.

You can experiment with different filesystems and in the first opportunity
with a different make of USB 3.0 stick. You'd soon be able to determine how
good the real world performance can get and if the Philips or something else
is causing the problem you've experienced.

A note about UDF: it works and it is versatile - but ensuring
interoperability between different OSs can be tricky. Check some suggestions
here:

https://askubuntu.com/questions/27936/can-and-should-udf-be-used-as-a-hard-drive-format

and here:

https://superuser.com/questions/39942/using-udf-on-a-usb-flash-drive

Ultimately such tests are an attempt to eliminate methodically all other
factors, until you isolate the cause of the problem you are experiencing.
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
Michael wrote:
> You may want to run some tests on the sticks you have, if only to bottom out
> what their performance is on different PCs and USB ports:
>
> dd if=/dev/zero of=/run/media/<USER>/<STICK_LABEL>/TESTFILE bs=512
> count=600000 oflag=direct conv=notrunc,fsync status=progress
>
> Use a large enough file to make sure the USB controller cache gets saturated.
>
> You could use a ramdisk/tmpfs as an input file.
>
> If you write directly to the device as Dale suggested it will wipe data, so
> keep a backup of anything you need first.


I checked the man page, I see nothing about the show progress option. 
You can bet I'll try that next time tho.  I saw a video of someone else
using it and it is a lot easier than having to switch Konsoles and type
in more commands. 

Thanks for sharing that option, that isn't in the freaking manual I
might add.  This is one time where telling someone to read the manual
wouldn't work. ROFL 

Awesome!!

Dale

:-)  :_) 
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Wednesday, 31 March 2021 12:37:49 BST Dale wrote:

> I checked the man page, I see nothing about the show progress option.

Are you sure?

This is what I see here on line 47:

"status=LEVEL
The LEVEL of information to print to stderr; 'none' suppresses
everything but error messages, 'noxfer' suppresses the final transfer
statistics, 'progress' shows periodic transfer statistics"

I find this useful in seeing the transfer speed drop in real time as the cache
gets saturated.
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
Michael wrote:
> On Wednesday, 31 March 2021 12:37:49 BST Dale wrote:
>
>> I checked the man page, I see nothing about the show progress option.
> Are you sure?
>
> This is what I see here on line 47:
>
> "status=LEVEL
> The LEVEL of information to print to stderr; 'none' suppresses
> everything but error messages, 'noxfer' suppresses the final transfer
> statistics, 'progress' shows periodic transfer statistics"
>
> I find this useful in seeing the transfer speed drop in real time as the cache
> gets saturated.


Ahhh, I didn't see the status part.  It's sort of hiding in a
subsection.  At least I know now that the version I have installed has
this option. 

I wish I could view man pages like I used to in Konqueror.  It displays
like a webpage and is much easier to search through.

Thanks for pointing that out. 

Dale

:-)  :-) 
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Wednesday, 31 March 2021 13:09:03 BST Dale wrote:
> Michael wrote:
> > On Wednesday, 31 March 2021 12:37:49 BST Dale wrote:
> >> I checked the man page, I see nothing about the show progress option.
> >
> > Are you sure?
> >
> > This is what I see here on line 47:
> >
> > "status=LEVEL
> >
> > The LEVEL of information to print to stderr; 'none' suppresses
> > everything but error messages, 'noxfer' suppresses the final
> > transfer
> > statistics, 'progress' shows periodic transfer statistics"
> >
> > I find this useful in seeing the transfer speed drop in real time as the
> > cache gets saturated.
>
> Ahhh, I didn't see the status part. It's sort of hiding in a
> subsection. At least I know now that the version I have installed has
> this option.
>
> I wish I could view man pages like I used to in Konqueror. It displays
> like a webpage and is much easier to search through.
>
> Thanks for pointing that out.
>
> Dale
>
> :-) :-)

Yes, I also liked the old Konqueror interface. Searching for keywords e.g.
"progress" within man pages works if you preface the keyword with "/":

/progress

will find it and "n" or "Shift+n" will jump forward and backward to any other
instances in the man page.
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
Michael wrote:
> On Wednesday, 31 March 2021 13:09:03 BST Dale wrote:
>> Michael wrote:
>>> On Wednesday, 31 March 2021 12:37:49 BST Dale wrote:
>>>> I checked the man page, I see nothing about the show progress option.
>>> Are you sure?
>>>
>>> This is what I see here on line 47:
>>>
>>> "status=LEVEL
>>>
>>> The LEVEL of information to print to stderr; 'none' suppresses
>>> everything but error messages, 'noxfer' suppresses the final
>>> transfer
>>> statistics, 'progress' shows periodic transfer statistics"
>>>
>>> I find this useful in seeing the transfer speed drop in real time as the
>>> cache gets saturated.
>> Ahhh, I didn't see the status part. It's sort of hiding in a
>> subsection. At least I know now that the version I have installed has
>> this option.
>>
>> I wish I could view man pages like I used to in Konqueror. It displays
>> like a webpage and is much easier to search through.
>>
>> Thanks for pointing that out.
>>
>> Dale
>>
>> :-) :-)
> Yes, I also liked the old Konqueror interface. Searching for keywords e.g.
> "progress" within man pages works if you preface the keyword with "/":
>
> /progress
>
> will find it and "n" or "Shift+n" will jump forward and backward to any other
> instances in the man page.

That doesn't work here.  I can type in /progress but it just shows up at
the bottom.  If I try "n" or shift+n I just get a n or N.  Maybe my man
page uses something different. 

That said, I haven't tried Konqueror in a while so I found it, it gives
a error but I can type in man:dd and it shows up.  It has a search
tool.  Last time I tried it, wouldn't even come up.  That was a while
ago tho.  Guess it got fixed.

Dale

:-)  :-) 
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Wednesday, 31 March 2021 13:23:27 BST Dale wrote:
> Michael wrote:
> > On Wednesday, 31 March 2021 13:09:03 BST Dale wrote:
> >> Michael wrote:
> >>> On Wednesday, 31 March 2021 12:37:49 BST Dale wrote:
> >>>> I checked the man page, I see nothing about the show progress option.
> >>>
> >>> Are you sure?
> >>>
> >>> This is what I see here on line 47:
> >>>
> >>> "status=LEVEL
> >>>
> >>> The LEVEL of information to print to stderr; 'none' suppresses
> >>> everything but error messages, 'noxfer' suppresses the final
> >>> transfer
> >>> statistics, 'progress' shows periodic transfer statistics"
> >>>
> >>> I find this useful in seeing the transfer speed drop in real time as the
> >>> cache gets saturated.
> >>
> >> Ahhh, I didn't see the status part. It's sort of hiding in a
> >> subsection. At least I know now that the version I have installed has
> >> this option.
> >>
> >> I wish I could view man pages like I used to in Konqueror. It displays
> >> like a webpage and is much easier to search through.
> >>
> >> Thanks for pointing that out.
> >>
> >> Dale
> >>
> >> :-) :-)
> >
> > Yes, I also liked the old Konqueror interface. Searching for keywords
> > e.g.
> > "progress" within man pages works if you preface the keyword with "/":
> >
> > /progress
> >
> > will find it and "n" or "Shift+n" will jump forward and backward to any
> > other instances in the man page.
>
> That doesn't work here. I can type in /progress but it just shows up at
> the bottom.

Yes, it shows at the bottom until you hit enter to execute the search. Then
it highlights the next instance of the searched string. Just like Vim/Vi
does. Hmm ... I wonder if I have set up some special environment parameter on
my systems and forgotten about it. :-/
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
Michael wrote:
> On Wednesday, 31 March 2021 13:23:27 BST Dale wrote:
>> Michael wrote:
>>> On Wednesday, 31 March 2021 13:09:03 BST Dale wrote:
>>>> Michael wrote:
>>>>> On Wednesday, 31 March 2021 12:37:49 BST Dale wrote:
>>>>>> I checked the man page, I see nothing about the show progress option.
>>>>> Are you sure?
>>>>>
>>>>> This is what I see here on line 47:
>>>>>
>>>>> "status=LEVEL
>>>>>
>>>>> The LEVEL of information to print to stderr; 'none' suppresses
>>>>> everything but error messages, 'noxfer' suppresses the final
>>>>> transfer
>>>>> statistics, 'progress' shows periodic transfer statistics"
>>>>>
>>>>> I find this useful in seeing the transfer speed drop in real time as the
>>>>> cache gets saturated.
>>>> Ahhh, I didn't see the status part. It's sort of hiding in a
>>>> subsection. At least I know now that the version I have installed has
>>>> this option.
>>>>
>>>> I wish I could view man pages like I used to in Konqueror. It displays
>>>> like a webpage and is much easier to search through.
>>>>
>>>> Thanks for pointing that out.
>>>>
>>>> Dale
>>>>
>>>> :-) :-)
>>> Yes, I also liked the old Konqueror interface. Searching for keywords
>>> e.g.
>>> "progress" within man pages works if you preface the keyword with "/":
>>>
>>> /progress
>>>
>>> will find it and "n" or "Shift+n" will jump forward and backward to any
>>> other instances in the man page.
>> That doesn't work here. I can type in /progress but it just shows up at
>> the bottom.
> Yes, it shows at the bottom until you hit enter to execute the search. Then
> it highlights the next instance of the searched string. Just like Vim/Vi
> does. Hmm ... I wonder if I have set up some special environment parameter on
> my systems and forgotten about it. :-/
>
>

*cough cough*  I didn't hit enter.  Just did and it worked.  Now to get
that info to stick in this old dog's brain for next time.  ROFL

Dale

:-)  :-)
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Wed, 31 Mar 2021 07:23:27 -0500, Dale wrote:

> > Yes, I also liked the old Konqueror interface. Searching for
> > keywords e.g. "progress" within man pages works if you preface the
> > keyword with "/":
> >
> > /progress
> >
> > will find it and "n" or "Shift+n" will jump forward and backward to
> > any other instances in the man page.
>
> That doesn't work here.  I can type in /progress but it just shows up at
> the bottom.  If I try "n" or shift+n I just get a n or N.  Maybe my man
> page uses something different. 

It's a feature of whichever pager you use, rather than man itself. Less
does it, as does most, but others may differ.


--
Neil Bothwick

Having children will turn you into your parents.
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On Wed, 31 Mar 2021 07:09:03 -0500, Dale wrote:

> I wish I could view man pages like I used to in Konqueror.  It displays
> like a webpage and is much easier to search through.

I miss that too. I use mankier.com these days, which gives similar
benefits. I have a shortcut set up in chromium so typing "man dd" opens
the page in mankier.com.


--
Neil Bothwick

Life's a cache, and then you flush...
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On 2021-03-31, Neil Bothwick <neil@digimed.co.uk> wrote:
> On Wed, 31 Mar 2021 07:09:03 -0500, Dale wrote:
>
>> I wish I could view man pages like I used to in Konqueror.  It displays
>> like a webpage and is much easier to search through.
>
> I miss that too. I use mankier.com these days, which gives similar
> benefits. I have a shortcut set up in chromium so typing "man dd"
> opens the page in mankier.com.

Many years ago, there was an X11 man page and gnu info viewer that I
used to use, but I can't remember the name of it. This was probably
20+ years ago (pre GTK and Qt), so the chances that it's still around
are small...

--
Grant
Re: Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On 2021.03.31 16:28, Grant Edwards wrote:
> On 2021-03-31, Neil Bothwick <neil@digimed.co.uk> wrote:
> > On Wed, 31 Mar 2021 07:09:03 -0500, Dale wrote:
> >
> >> I wish I could view man pages like I used to in Konqueror.? It
> displays
> >> like a webpage and is much easier to search through.
> >
> > I miss that too. I use mankier.com these days, which gives similar
> > benefits. I have a shortcut set up in chromium so typing "man dd"
> > opens the page in mankier.com.
>
> Many years ago, there was an X11 man page and gnu info viewer that I
> used to use, but I can't remember the name of it. This was probably
> 20+ years ago (pre GTK and Qt), so the chances that it's still around
> are small...
Perhaps x11-apps/xman? I remember it as you do, but have not used it
in years, so I'm not sure if this is it or not. The info command also
seems to have access to man pages, but it doesn't seem the most obvious
way to get there.

Jack
Re: Is there a way to misconfigure USB ports in the kernel? [ In reply to ]
On 2021-03-31, Jack <ostroffjh@users.sourceforge.net> wrote:
> On 2021.03.31 16:28, Grant Edwards wrote:
>
>> Many years ago, there was an X11 man page and gnu info viewer that I
>> used to use, but I can't remember the name of it. This was probably
>> 20+ years ago (pre GTK and Qt), so the chances that it's still around
>> are small...
>
> Perhaps x11-apps/xman?

I do remember using xman closer to 30 years ago, but what I was
thinking of was something a bit newer and fancier than that -- I
recall it also being able to navigate info pages. Or I may be
conflating two different apps: one for man and one for info.

It may have been tkinfo or tkman

http://math-www.uni-paderborn.de/~axel/tkinfo/
https://sourceforge.net/projects/tkman/


> I remember it as you do, but have not used it in years, so I'm not
> sure if this is it or not. The info command also seems to have
> access to man pages, but it doesn't seem the most obvious way to get
> there.