Mailing List Archive

On Thu, Oct 29, 2020 at 5:43 AM Jared Mauch <jared@puck.nether.net> wrote:

> I have all UBNT at home for wireless and periodically have some
> random
> issues which I can't explain, but for the most part have things tuned to
> ensure
> there's little to no interference.
>

All UBNT at home? Ouch.

They're on my banned list after one of their POE devices caught on fire
after being in service for 11 months.
Then they went round and round for a week saying they weren't going to pay
for a shipping label. I wasn't going to pay for one because I didn't want
their gear back.

Finally someone with a bit of common sense sent a shipping label so they
could figure out why it caught on fire.
They ended up sending a replacement back that was obviously used. Instead
of letting it go to waste, I installed it.
It died two weeks later. When I contacted them, they said the original
purchase was over a year ago so they wouldn't RMA it.

Then a second device (plugged into an entirely different switch in a
different building) started smoking and emitting an electrical smell. I
pulled all of them and tossed them in the dumpster.

They are an absolutely atrocious company to deal with. I'm betting some
day real soon they'll be sued into oblivion when their crap burns down
someone's home or office building.

Friends don't let friends buy UniFi.

-A

I'd need more data than your anecdotal experience with a POE device to
throw out my Unifi gear and ban the company. But I'm dealing with 2
devices: a Security Gateway and a single Access Point (plus the Controller
software running on my Mac).

There are some quirky things about Unifi that can be annoying, but it is
mostly around common stuff like running a DNS Caching server on the
Security Gateway or force-pushing a DDNS update.

It's been way better than the ad-hoc varied brand of network I was running
before, and I get to see and manage a lot more as well, quite reliably.

We all know that hardware failures happen, and you definitely had a bad
experience, and that sucks.

I'll take all of your Unifi gear, PM me for an address. :-)

Beckman

On Thu, 29 Oct 2020, Aaron C. de Bruyn via NANOG wrote:

> On Thu, Oct 29, 2020 at 5:43 AM Jared Mauch <jared@puck.nether.net> wrote:
>
>> I have all UBNT at home for wireless and periodically have some
>> random
>> issues which I can't explain, but for the most part have things tuned to
>> ensure
>> there's little to no interference.
>>
>
> All UBNT at home? Ouch.
>
> They're on my banned list after one of their POE devices caught on fire
> after being in service for 11 months.
> Then they went round and round for a week saying they weren't going to pay
> for a shipping label. I wasn't going to pay for one because I didn't want
> their gear back.
>
> Finally someone with a bit of common sense sent a shipping label so they
> could figure out why it caught on fire.
> They ended up sending a replacement back that was obviously used. Instead
> of letting it go to waste, I installed it.
> It died two weeks later. When I contacted them, they said the original
> purchase was over a year ago so they wouldn't RMA it.
>
> Then a second device (plugged into an entirely different switch in a
> different building) started smoking and emitting an electrical smell. I
> pulled all of them and tossed them in the dumpster.
>
> They are an absolutely atrocious company to deal with. I'm betting some
> day real soon they'll be sued into oblivion when their crap burns down
> someone's home or office building.
>
> Friends don't let friends buy UniFi.
>
> -A
>

---------------------------------------------------------------------------
Peter Beckman Internet Guy
beckman@angryox.com http://www.angryox.com/
---------------------------------------------------------------------------

I also have all Ubiquiti stuff at home, and I’ve deployed it in large installations of up to 100 APs. Beyond that it seems to hit some communication bottleneck in its spectrum allocation protocols, so I usually go to Aruba or Ruckus for networks larger than that.

Ubiquiti is pretty reliable, but you have to stay away from the latest release, just like anything. Its SDN implementation is quite impressive for what it does, especially for the fact that you get it for free (i’m talking to you, Cisco DNA :-). The only major downside is that it’s all Ubiquiti all the time: they don’t interoperate with anything else.

-mel via cell

> On Oct 29, 2020, at 12:24 PM, Peter Beckman <beckman@angryox.com> wrote:
>
> ?I'd need more data than your anecdotal experience with a POE device to
> throw out my Unifi gear and ban the company. But I'm dealing with 2
> devices: a Security Gateway and a single Access Point (plus the Controller
> software running on my Mac).
>
> There are some quirky things about Unifi that can be annoying, but it is
> mostly around common stuff like running a DNS Caching server on the
> Security Gateway or force-pushing a DDNS update.
>
> It's been way better than the ad-hoc varied brand of network I was running
> before, and I get to see and manage a lot more as well, quite reliably.
>
> We all know that hardware failures happen, and you definitely had a bad
> experience, and that sucks.
>
> I'll take all of your Unifi gear, PM me for an address. :-)
>
> Beckman
>
>> On Thu, 29 Oct 2020, Aaron C. de Bruyn via NANOG wrote:
>>
>>> On Thu, Oct 29, 2020 at 5:43 AM Jared Mauch <jared@puck.nether.net> wrote:
>>>
>>> I have all UBNT at home for wireless and periodically have some
>>> random
>>> issues which I can't explain, but for the most part have things tuned to
>>> ensure
>>> there's little to no interference.
>>>
>>
>> All UBNT at home? Ouch.
>>
>> They're on my banned list after one of their POE devices caught on fire
>> after being in service for 11 months.
>> Then they went round and round for a week saying they weren't going to pay
>> for a shipping label. I wasn't going to pay for one because I didn't want
>> their gear back.
>>
>> Finally someone with a bit of common sense sent a shipping label so they
>> could figure out why it caught on fire.
>> They ended up sending a replacement back that was obviously used. Instead
>> of letting it go to waste, I installed it.
>> It died two weeks later. When I contacted them, they said the original
>> purchase was over a year ago so they wouldn't RMA it.
>>
>> Then a second device (plugged into an entirely different switch in a
>> different building) started smoking and emitting an electrical smell. I
>> pulled all of them and tossed them in the dumpster.
>>
>> They are an absolutely atrocious company to deal with. I'm betting some
>> day real soon they'll be sued into oblivion when their crap burns down
>> someone's home or office building.
>>
>> Friends don't let friends buy UniFi.
>>
>> -A
>>
>
> ---------------------------------------------------------------------------
> Peter Beckman Internet Guy
> beckman@angryox.com http://www.angryox.com/
> ---------------------------------------------------------------------------

On Thu, Oct 29, 2020 at 12:22 PM Peter Beckman <beckman@angryox.com> wrote:

> I'll take all of your Unifi gear, PM me for an address. :-)
>

I'd send it your way in a heartbeat, but you wouldn't get much use out of
it.
I have an old CloudKey that mysteriously doesn't seem to be getting updates
anymore.
I have an old CloudKey Gen2 Plus that hasn't received updates in about a
year but the HDD died 8 months in.
I have a UDM Pro that's useless because the WAN port seems to have a
manufacturing defect.
I have a 4 old UVC-G3 cameras that died at about 13 months (not eligible
for warranty repair)
I have 3 UVC-G3-Flex cameras that have bad SD cards that I fortunately
bought from a vendor instead of UniFi directly and they just said "here are
replacements, don't bother shipping that crap back--toss them".
I have a US-24-250 and a US-48-500 switch that have dead power supplies.
One was DOA, the other died about 3 months in. UniFi won't fix it, and I'm
not going to pay for shipping labels to fix their mistakes. (I paid for a
working switch, and I got a dead switch--why should I pay more to get what
I ordered?)
I have a dead VIEWPORT and two that 'stutter' badly when displaying 4
cameras and they reboot several times per hour.
I have a partially melted UA-HUB.

Fortunately I switched to a vendor in the last 6 months that charges about
5% less than the list price and they actually replace stuff at no cost.
Dealing with Ubiquity directly is a nightmare.

There are some quirky things about Unifi that can be annoying, but it is
> mostly around common stuff like running a DNS Caching server on the
> Security Gateway or force-pushing a DDNS update.
>

Quirky things: the lack of customer service, the lack of communication
with customers, the buggy software, releasing hardware to production that
is completely unusable for 6 months (UDM Pro), etc...

Anyways, not to get too down on them, their wireless gear and switches are
pretty wonderful. I've never had a WAP die on me, and only a small handful
of switches...but the cost is hard to beat.
There has been talk in the forums about various Apple gear having problems,
and I think a recent update (possibly still in beta) appears to have fixed
it. I don't own any Apple gear, so I can't confirm.

-A

On 10/29/2020 1:42 PM, Aaron C. de Bruyn via NANOG wrote:
> I have an old CloudKey that mysteriously doesn't seem to be getting
> updates anymore.

https://community.ui.com/releases/UniFi-Cloud-Key-Firmware-1-1-13/733dfc55-b61b-483b-afc1-77d7f2c1e032

4 months ago, if you want to stick strictly to stable... More recent if
you run a beta release.


> I have an old CloudKey Gen2 Plus that hasn't received updates in about a
> year but the HDD died 8 months in.

Updates are from same link as above, and there's new builds based on
their new OS that integrates a bunch of separate controllers if you
don't mind beta...

Not sure about hard disk dying.


> I have a UDM Pro that's useless because the WAN port seems to have a
> manufacturing defect.


Its actually a software bug - I had it too on releases around 1.8.0.
It's actually fixed in 1.8.2-5 and later.


The other stuff, i can't really comment on. But, I figured I'd correct
several statements of what I had knowledge on. :-)

One thing I will note - I've had only one device out of hundreds go bad
in the last 10 years of using their products. Usually, when you have
lots of failing hardware 'for some reason', its a good idea to look into
the reason 'why?'. Could be bad grounds, could be dirty power...

--
Brielle Bruns
The Summit Open Source Development Group
http://www.sosdg.org / http://www.ahbl.org

On Thu, Oct 29, 2020 at 12:54 PM Brielle <bruns@2mbit.com> wrote:

> On 10/29/2020 1:42 PM, Aaron C. de Bruyn via NANOG wrote:
> > I have an old CloudKey that mysteriously doesn't seem to be getting
> > updates anymore.
>
>
> https://community.ui.com/releases/UniFi-Cloud-Key-Firmware-1-1-13/733dfc55-b61b-483b-afc1-77d7f2c1e032
>
> 4 months ago, if you want to stick strictly to stable... More recent if
> you run a beta release.


Not to drag this out any further, but that was well after I stopped using
it because it hadn't had updates for nearly a year.

Same with the other issues mentioned. They spent a good 8-12 months not
communicating with customers, not addressing issues, and not releasing
firmware while they developed their new UniFi OS.

-A

Sorry--accidental premature send.

On Thu, Oct 29, 2020 at 12:54 PM Brielle <bruns@2mbit.com> wrote:

> Updates are from same link as above, and there's new builds based on
> their new OS that integrates a bunch of separate controllers if you
> don't mind beta...
>

With most companies I wouldn't mind.
But with UniFi, Alpha is Beta, and Beta is launched into production.


> > I have a UDM Pro that's useless because the WAN port seems to have a
> > manufacturing defect.
>
> Its actually a software bug - I had it too on releases around 1.8.0.
> It's actually fixed in 1.8.2-5 and later.
>

That's what I saw originally too. It kept it sitting on the shelf for
months.
It's fixed in their beta software, but not on one of my appliances.


> One thing I will note - I've had only one device out of hundreds go bad
> in the last 10 years of using their products. Usually, when you have
> lots of failing hardware 'for some reason', its a good idea to look into
> the reason 'why?'. Could be bad grounds, could be dirty power...


Their switches and WAPs are pretty solid for me. I have about 150 of each
deployed with only 2 or 3 failures.

I've had electricians check power in locations where things died and they
find no issues. Plus the sites use double-conversion UPS units, so I
highly doubt it's power issues.

-A

On 10/29/20 20:14, David Curado wrote:

> I was curious, so poked at this... my results from a macbook pro 2019
> running Catalina 10.15.3
>
> sudo /usr/local/sbin/mtr -r 10.200.200.200
>
> Start: 2020-10-29T14:09:08-0400
> HOST: bos-mp36c                   Loss%   Snt Last   Avg  Best  Wrst StDev
>   1.|-- 10.200.200.200             0.0%    10 11.9  63.7   9.0 340.2 104.1
>
> sudo sysctl net.link.generic.system.hwcksum_tx=0
> net.link.generic.system.hwcksum_tx: 1 -> 0
> sudo sysctl net.link.generic.system.hwcksum_rx=0
> net.link.generic.system.hwcksum_rx: 1 -> 0
> sudo ifconfig en0 -rxcsum
>
> sudo /usr/local/sbin/mtr -r 10.200.200.200
>
> Start: 2020-10-29T14:09:43-0400
> HOST: bos-mp36c                   Loss%   Snt Last   Avg  Best  Wrst StDev
>   1.|-- 10.200.200.200             0.0%    10 19.8  13.6   9.8  20.9   4.5
>
> So, seems to make things better. =-)

I tried it again this morning, on the off-chance that the stars didn't
align yesterday. Same deal:

sh-3.2# sysctl net.link.generic.system.hwcksum_tx=0
net.link.generic.system.hwcksum_tx: 1 -> 0
sh-3.2# sysctl net.link.generic.system.hwcksum_rx=0
net.link.generic.system.hwcksum_rx: 1 -> 0
sh-3.2# ifconfig en0 -rxcsum

Marks-MacBook-Pro.local (172.16.0.239) 2020-10-30T08:39:44+0200
Keys:  Help   Display mode   Restart statistics   Order of fields quit
Packets               Pings
 Host Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 172.16.0.254 0.0%    28    1.9  28.7   1.9 218.9  56.1

Mark.

Hi all.

So I may have fixed this for my end, and hopefully others may be able to
use the same fix.

After a tip from Karl Auerbach and this link:

    https://developer.apple.com/forums/thread/97805

... I was able to fix the problem by disabling Bluetooth.

However, disabling Bluetooth was not enough. I also had to disable all
Location Services.

After that, I re-enabled Location Services and only allowed for two
features:

    - NetSpot
    - Find My Mac

With just those two location services, as well as Bluetooth disabled, I
have no more high jitter.

App performance like Zoom and Youtube uploads are now crisp, with 0.0%
packet loss.

So looks like that Bluetooth is a huge problem. Confirmed by opening the
"Console" app, and adding "scan" in the filter bar, top right.

A peak latency of 13.5ms after 300 packets:

Marks-MacBook-Pro.local (172.16.0.239) 2020-10-30T21:06:05+0200
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
Packets               Pings
 Host Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. 172.16.0.254 0.0%   300    3.1   4.8   2.2  13.5   1.9

Mark.

Hi Mark,

I'm running a MacBook Pro (Retina, 15-inch, Mid 2015) with Mojave 10.14.6 (latest).
I've always had location services off (including all system services within).

I haven't seen any jitter issues on my end.

Along with, no matter if I have bluetooth turned on with my wireless mouse and keyboard connected or not, I see no consistent change in jitter.

Even further, enabling location services (including all system services within) and bluetooth enabled, I'm not seeing any change in jitter.

I know you said earlier this issue seems to have started somewhere _around_ Mojave so just my two cents.

-Aaron

Oct 30, 2020, 12:08 by mark.tinka@seacom.com:

> Hi all.
>
> So I may have fixed this for my end, and hopefully others may be able to use the same fix.
>
> After a tip from Karl Auerbach and this link:
>
>     > https://developer.apple.com/forums/thread/97805
>
> ... I was able to fix the problem by disabling Bluetooth.
>
> However, disabling Bluetooth was not enough. I also had to disable all Location Services.
>
> After that, I re-enabled Location Services and only allowed for two features:
>
>     - NetSpot
>     - Find My Mac
>
> With just those two location services, as well as Bluetooth disabled, I have no more high jitter.
>
> App performance like Zoom and Youtube uploads are now crisp, with 0.0% packet loss.
>
> So looks like that Bluetooth is a huge problem. Confirmed by opening the "Console" app, and adding "scan" in the filter bar, top right.
>
> A peak latency of 13.5ms after 300 packets:
>
> Marks-MacBook-Pro.local(172.16.0.239)                                                                                                                                                                                       2020-10-30T21:06:05+0200
> Keys:  Help   Display mode   Restart statistics   Order of fields   quit
>                                                                                                                                                                                                              Packets               Pings
>  Host                                                                                                                                                                                                      Loss%   Snt   Last   Avg  Best  Wrst StDev
>  1.172.16.0.254                                                                                                                                                                                            0.0%   300    3.1   4.8   2.2  13.5   1.9
>
> Mark.
>
>

I would hesitate to blame BT. I have a macbook pro from ~1 year ago, on
Catalina, and I use BT extensively ... mouse, keyboard, and headset. I
do have location services trimmed down to just find my mac.

I ran: ping -c 1000 -i 0.1 <router address>

1000 packets transmitted, 998 packets received, 0.2% packet loss
round-trip min/avg/max/stddev = 1.255/2.378/9.095/0.634 ms

One thing that may contribute to blaming BT however is if you are using
wifi on 2.4G only, and/or preferring it, as BT operates in the same
frequency range neighborhood. My macbook is connected using 5G.

Happy to compare other settings if there is interest.

Doug


On 10/30/20 12:08 PM, Mark Tinka wrote:
> Hi all.
>
> So I may have fixed this for my end, and hopefully others may be able to
> use the same fix.
>
> After a tip from Karl Auerbach and this link:
>
> https://developer.apple.com/forums/thread/97805
>
> ... I was able to fix the problem by disabling Bluetooth.
>
> However, disabling Bluetooth was not enough. I also had to disable all
> Location Services.
>
> After that, I re-enabled Location Services and only allowed for two
> features:
>
>     - NetSpot
>     - Find My Mac
>
> With just those two location services, as well as Bluetooth disabled, I
> have no more high jitter.
>
> App performance like Zoom and Youtube uploads are now crisp, with 0.0%
> packet loss.
>
> So looks like that Bluetooth is a huge problem. Confirmed by opening the
> "Console" app, and adding "scan" in the filter bar, top right.
>
> A peak latency of 13.5ms after 300 packets:
>
> Marks-MacBook-Pro.local (172.16.0.239) 2020-10-30T21:06:05+0200
> Keys:  Help   Display mode   Restart statistics   Order of fields   quit
> Packets               Pings
>  Host Loss%   Snt   Last   Avg  Best  Wrst StDev
>  1. 172.16.0.254 0.0%   300    3.1   4.8   2.2  13.5   1.9
>
> Mark.
>
>

Heya ! Doug!

Yeah I wouldn’t put this on BT either. On the other hand it seems that whether the scheduler is newreno or cubic that this situation persists pasts my previous suggestions. Seems tho that when you put strain on an upload that the jitter gets considerably worse... 90m out of a 100m link it starts to get to 1000+ms

--
J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a lot about anticipated traffic volume.

> On Oct 30, 2020, at 16:59, Doug Barton <dougb@dougbarton.us> wrote:
> ?I would hesitate to blame BT. I have a macbook pro from ~1 year ago, on Catalina, and I use BT extensively ... mouse, keyboard, and headset. I do have location services trimmed down to just find my mac.
>
> I ran: ping -c 1000 -i 0.1 <router address>
>
> 1000 packets transmitted, 998 packets received, 0.2% packet loss
> round-trip min/avg/max/stddev = 1.255/2.378/9.095/0.634 ms
>
> One thing that may contribute to blaming BT however is if you are using wifi on 2.4G only, and/or preferring it, as BT operates in the same frequency range neighborhood. My macbook is connected using 5G.
>
> Happy to compare other settings if there is interest.
>
> Doug
>
>
>> On 10/30/20 12:08 PM, Mark Tinka wrote:
>> Hi all.
>> So I may have fixed this for my end, and hopefully others may be able to use the same fix.
>> After a tip from Karl Auerbach and this link:
>> https://developer.apple.com/forums/thread/97805
>> ... I was able to fix the problem by disabling Bluetooth.
>> However, disabling Bluetooth was not enough. I also had to disable all Location Services.
>> After that, I re-enabled Location Services and only allowed for two features:
>> - NetSpot
>> - Find My Mac
>> With just those two location services, as well as Bluetooth disabled, I have no more high jitter.
>> App performance like Zoom and Youtube uploads are now crisp, with 0.0% packet loss.
>> So looks like that Bluetooth is a huge problem. Confirmed by opening the "Console" app, and adding "scan" in the filter bar, top right.
>> A peak latency of 13.5ms after 300 packets:
>> Marks-MacBook-Pro.local (172.16.0.239) 2020-10-30T21:06:05+0200
>> Keys: Help Display mode Restart statistics Order of fields quit
>> Packets Pings
>> Host Loss% Snt Last Avg Best Wrst StDev
>> 1. 172.16.0.254 0.0% 300 3.1 4.8 2.2 13.5 1.9
>> Mark.

On 10/30/20 23:57, Doug Barton wrote:

> I would hesitate to blame BT. I have a macbook pro from ~1 year ago,
> on Catalina, and I use BT extensively ... mouse, keyboard, and
> headset. I do have location services trimmed down to just find my mac.
>
> I ran: ping -c 1000 -i 0.1 <router address>
>
> 1000 packets transmitted, 998 packets received, 0.2% packet loss
> round-trip min/avg/max/stddev = 1.255/2.378/9.095/0.634 ms
>
> One thing that may contribute to blaming BT however is if you are
> using wifi on 2.4G only, and/or preferring it, as BT operates in the
> same frequency range neighborhood. My macbook is connected using 5G.
>
> Happy to compare other settings if there is interest.

What I do know is that there has been more than one fix for most people
that have had this issue. For some, it has been a dodgy app, and for
others, it has just been disabling some Location Services.

In my case, what worked was disabling Bluetooth, in combination with as
many Location Services as I could afford to not use.

I am able to reliably reproduce this issue, so in my case, I can
certainly blame Bluetooth. Pity, since I do use some Handoff services
with my other Apple devices, but I won't give up wi-fi stability for that.

And yes, this happens both on 2.4GHz and 5GHz. As you may know, macOS
will default to 5GHz provided RSSI is at -68dB or better. My laptop only
ever connects on 2.4GHz if I am outside my house (which is hardly).
Inside the house, there is always a 5GHz transmitter within -38dB of me.

Mark.

On 10/31/20 01:53, J. Hellenthal via NANOG wrote:

> Heya ! Doug!
>
> Yeah I wouldn’t put this on BT either. On the other hand it seems that whether the scheduler is newreno or cubic that this situation persists pasts my previous suggestions. Seems tho that when you put strain on an upload that the jitter gets considerably worse... 90m out of a 100m link it starts to get to 1000+ms

I noticed this too... when I had the issue, uploads were severely
impacted (hence my problems pushing files to Youtube).

However, I also noticed the same problem if I am stressing my 100Mbps
FTTH link pushing a 24GB file to Youtube, as a matter of course. So it
would make sense that the high jitter caused by Bluetooth, in my case,
affects uploads more because the wi-fi and Bluetooth chips on the laptop
are aggressively scanning the environment, which would seem like a Tx
direction to me.

Mark.

Let me jump in and add a bit more information.

I am not an RF guy - I stopped playing with radios [and TV] in the days
when they used vacuum tubes (yes, really.)

Many laptops share radio and antenna resources between WiFi and bluetooth.

Bluetooth lives on the 2.4ghz band.  Wifi presently uses both that band
and also a 5ghz band. Different antennas might be used for each.

I encountered Wi-Fi/Bluetooth contention issues a couple of years back....

My home wifi has (or rather had) distinct SSIDs for Wifi on the 2.4 and
5ghz bands.  It was a rough attempt at manual load and distance balancing.

(Our house is in a relatively quiet area, RF wise, so there's not really
any seriously competing wi-fi - or for that matter cell signal,
broadcast TV, or FM radio.)

I began to notice that when I had one of my laptops on the 5ghz WiFi and
was listening to music via some bluetooth speakers that my remote
terminal keystrokes sometimes had that sluggish feel that is familiar
when doing remote terminal command-line stuff over long paths with a lot
of latency/jitter.  And at the same time the music via Bluetooth often
broke up or stuttered.  There was a clear correlation between the two
problems.

I had heard from some Linux kernel developers that deep down in the
Linux kernel the simultaneous use of Wifi on a 5ghz channel and
bluetooth on 2.4 causes a lot of thrashing and flogging of the the radio
system.  I don't know, but I suspect that as a result there are queues
of outbound traffic waiting for the radio or antennas to become
operational on the channel they need.  I have no idea what happens to
inbound frames when the radio system is tuned elsewhere - I never
measured whether the frames are lost or delayed.

I suspect similar issues are present in *BSD, MacOS, and Windows kernels.

So I did some simple empirical testing to compare life with the laptop
coerced to use an SSID present only on the 2.4ghz band. The problems
went away.

I went back to the laptop, but coerced onto the 5ghz band for WiFi and,
voila, there was trouble.

I've done this with a MacBook Pro (circa 2015 model) using various
versions of MacOS and with my rather newer Linux laptops (mostly Dell
XPS units with Fedora.)  Same sorts of behavior.

These were all i5 based units with 2 or 4 cores - plenty of CPU power to
simultaneously handle an SSH remote console client and a music player.

I did not test with mobile phone or tablet platforms.

I do not know if the single radio issue is the result of cost savings or
some radio-engineering or antenna issue.  I do suspect that these things
could become more troublesome as WiFi 6 and/or 5G start to use some of
the higher frequency allocations around 5.9 and 6ghz.)

(A few weeks ago we switched our home WiFi to a WiFi 6 [Netgear Orbi-6]
mesh system that does not appear to allow separate SSIDs for the 2.4 and
5ghz bands, so I can not repeat these tests without constructing a test
network with the now unused access points.  BTW, I did encounter the
hell that is known as "reconfiguring dozens upon dozens of different
kinds of IoT devices to use a different SSID".)

Looking somewhat off topic - it is my sense that we will be seeing a lot
more latency/jitter (and packet resequencing) issues in the future as
radio systems become more agile and as we begin to use shorter
(millimeter) wavelength frequencies with reduced ability to penetrate
walls that, in turn, cause more frequent access-point transitions (with
possibly distinctly different backhaul characteristics).  I've observed
that these things can cause trouble for some TCP stacks and some non-TCP
based VoIP and streaming applications.

        --karl--

On 10/30/20 12:08 PM, Mark Tinka wrote:
> Hi all.
>
> So I may have fixed this for my end, and hopefully others may be able
> to use the same fix.
>
> After a tip from Karl Auerbach and this link:
>
> https://developer.apple.com/forums/thread/97805
>
> ... I was able to fix the problem by disabling Bluetooth.

Thanks for the input, Karl.

Hopefully someone from Apple is around here and can get some ideas on
how to fix this particular problem set.

Mark.

On 10/31/20 11:37, Karl Auerbach wrote:
>
> Let me jump in and add a bit more information.
>
> I am not an RF guy - I stopped playing with radios [and TV] in the
> days when they used vacuum tubes (yes, really.)
>
> Many laptops share radio and antenna resources between WiFi and bluetooth.
>
> Bluetooth lives on the 2.4ghz band.  Wifi presently uses both that
> band and also a 5ghz band. Different antennas might be used for each.
>
> I encountered Wi-Fi/Bluetooth contention issues a couple of years back....
>
> My home wifi has (or rather had) distinct SSIDs for Wifi on the 2.4
> and 5ghz bands.  It was a rough attempt at manual load and distance
> balancing.
>
> (Our house is in a relatively quiet area, RF wise, so there's not
> really any seriously competing wi-fi - or for that matter cell signal,
> broadcast TV, or FM radio.)
>
> I began to notice that when I had one of my laptops on the 5ghz WiFi
> and was listening to music via some bluetooth speakers that my remote
> terminal keystrokes sometimes had that sluggish feel that is familiar
> when doing remote terminal command-line stuff over long paths with a
> lot of latency/jitter.  And at the same time the music via Bluetooth
> often broke up or stuttered.  There was a clear correlation between
> the two problems.
>
> I had heard from some Linux kernel developers that deep down in the
> Linux kernel the simultaneous use of Wifi on a 5ghz channel and
> bluetooth on 2.4 causes a lot of thrashing and flogging of the the
> radio system.  I don't know, but I suspect that as a result there are
> queues of outbound traffic waiting for the radio or antennas to become
> operational on the channel they need.  I have no idea what happens to
> inbound frames when the radio system is tuned elsewhere - I never
> measured whether the frames are lost or delayed.
>
> I suspect similar issues are present in *BSD, MacOS, and Windows kernels.
>
> So I did some simple empirical testing to compare life with the laptop
> coerced to use an SSID present only on the 2.4ghz band. The problems
> went away.
>
> I went back to the laptop, but coerced onto the 5ghz band for WiFi
> and, voila, there was trouble.
>
> I've done this with a MacBook Pro (circa 2015 model) using various
> versions of MacOS and with my rather newer Linux laptops (mostly Dell
> XPS units with Fedora.)  Same sorts of behavior.
>
> These were all i5 based units with 2 or 4 cores - plenty of CPU power
> to simultaneously handle an SSH remote console client and a music player.
>
> I did not test with mobile phone or tablet platforms.
>
> I do not know if the single radio issue is the result of cost savings
> or some radio-engineering or antenna issue.  I do suspect that these
> things could become more troublesome as WiFi 6 and/or 5G start to use
> some of the higher frequency allocations around 5.9 and 6ghz.)
>
> (A few weeks ago we switched our home WiFi to a WiFi 6 [Netgear
> Orbi-6] mesh system that does not appear to allow separate SSIDs for
> the 2.4 and 5ghz bands, so I can not repeat these tests without
> constructing a test network with the now unused access points.  BTW, I
> did encounter the hell that is known as "reconfiguring dozens upon
> dozens of different kinds of IoT devices to use a different SSID".)
>
> Looking somewhat off topic - it is my sense that we will be seeing a
> lot more latency/jitter (and packet resequencing) issues in the future
> as radio systems become more agile and as we begin to use shorter
> (millimeter) wavelength frequencies with reduced ability to penetrate
> walls that, in turn, cause more frequent access-point transitions
> (with possibly distinctly different backhaul characteristics).  I've
> observed that these things can cause trouble for some TCP stacks and
> some non-TCP based VoIP and streaming applications.
>
>         --karl--
>
> On 10/30/20 12:08 PM, Mark Tinka wrote:
>> Hi all.
>>
>> So I may have fixed this for my end, and hopefully others may be able
>> to use the same fix.
>>
>> After a tip from Karl Auerbach and this link:
>>
>> https://developer.apple.com/forums/thread/97805
>>
>> ... I was able to fix the problem by disabling Bluetooth.

Just an update on this re: the Bluetooth.

I had my AirPods paired previously for single use. I don't use them on
the laptop (there is some latency), so I prefer the wired earphones. But
it seems like Bluetooth was aggressively scanning for them. After
removing them from the system, the scanning remained, but reduced
significantly.

So looking at Console again, every so often, Bluetooth is scanning the
network on behalf of the "sharingd" process.

sharingd is a sharing daemon that supports features such as AirDrop,
Handoff, Instant Hotspot, Shared Computers and Remote Disc in Finder.

Still keeping Bluetooth off, however.

Mark.

Hey Mark,

Went through a bunch of tests here. Seems I’ve cleared up the matter on this macOS[1] Big Sur at least by disabling Wi-Fi Networking under “Location Services -> System Services -> Wi-Fi Networking [2]”. It seems at least from perspective that something changed there and causes the Mac to scan more aggressively when more than one access point (generally speaking your SSID & SSID + 5G) has been logged at a location as accessible. This same thing could be observed at least on my system while having those settings turned off, bluetooth on and location services enabled and opening (Wi-Fi Explorer[3]) which puts the interface into “monitor mode” which seems to be causing the contention somewhere. After those changes keep in mind I had to restart from a full shutdown to get to some real clean ms traffic to the router and I prefer to be connected to 5Ghz before 2.4Ghz.


1. Darwin Kernel Version 20.1.0: Thu Oct 29 05:35:40 PDT 2020; root:xnu-7195.50.5~4/RELEASE_X86_64
2. https://www.dropbox.com/s/m3xm3fpoziwe01d/Screen%20Shot%202020-11-10%20at%2008.20.08.png?dl=0
3. https://apps.apple.com/us/app/wifi-explorer/id494803304?mt=12


> On Nov 5, 2020, at 00:43, Mark Tinka <mark.tinka@seacom.com> wrote:
>
> Just an update on this re: the Bluetooth.
>
> I had my AirPods paired previously for single use. I don't use them on the laptop (there is some latency), so I prefer the wired earphones. But it seems like Bluetooth was aggressively scanning for them. After removing them from the system, the scanning remained, but reduced significantly.
>
> So looking at Console again, every so often, Bluetooth is scanning the network on behalf of the "sharingd" process.
>
> sharingd is a sharing daemon that supports features such as AirDrop, Handoff, Instant Hotspot, Shared Computers and Remote Disc in Finder.
>
> Still keeping Bluetooth off, however.
>
> Mark.


--

J. Hellenthal

The fact that there's a highway to Hell but only a stairway to Heaven says a lot about anticipated traffic volume.

Thanks, J.

So I did test this a few times as well. The only thing I had enabled
(during the first test) in "System Services" was "Find My Mac".
Everything else was turned off, and the issue remained.

Just to confirm that this was a clean install of Catalina, so the only
wi-fi AP in my history is my home one, as I haven't worked in a cafe,
airport, or hotel since then :-).

Since disabling Bluetooth and the system stabilizing, I've re-enabled
"Location-Based Suggestions" and "Significant Locations" only.
Everything else under "System Services" is still turned off, including
"Wi-Fi Networking".

That said, you did mention that you've fixed this on Big Sur. I am still
running Catalina. Considering Apple's history of dodgy initial releases
of a new OS, I'll give Big Sur a few months (or a year) before I feel
it's safe to upgrade. I'm already having to deal with Catalina as it is,
which is why I have High Sierra installed on my old laptop for my
weekend DJ streaming habit :-). OBS Studio seems to like High Sierra
better than Catalina, hehe.

Many thanks for working on this - it's most appreciated!

Mark.

On 11/10/20 16:30, J. Hellenthal wrote:
> Hey Mark,
>
> Went through a bunch of tests here. Seems I’ve cleared up the matter on this macOS[1] Big Sur at least by disabling Wi-Fi Networking under “Location Services -> System Services -> Wi-Fi Networking [2]”. It seems at least from perspective that something changed there and causes the Mac to scan more aggressively when more than one access point (generally speaking your SSID & SSID + 5G) has been logged at a location as accessible. This same thing could be observed at least on my system while having those settings turned off, bluetooth on and location services enabled and opening (Wi-Fi Explorer[3]) which puts the interface into “monitor mode” which seems to be causing the contention somewhere. After those changes keep in mind I had to restart from a full shutdown to get to some real clean ms traffic to the router and I prefer to be connected to 5Ghz before 2.4Ghz.
>
>
> 1. Darwin Kernel Version 20.1.0: Thu Oct 29 05:35:40 PDT 2020; root:xnu-7195.50.5~4/RELEASE_X86_64
> 2. https://www.dropbox.com/s/m3xm3fpoziwe01d/Screen%20Shot%202020-11-10%20at%2008.20.08.png?dl=0
> 3. https://apps.apple.com/us/app/wifi-explorer/id494803304?mt=12
>
>
>> On Nov 5, 2020, at 00:43, Mark Tinka <mark.tinka@seacom.com> wrote:
>>
>> Just an update on this re: the Bluetooth.
>>
>> I had my AirPods paired previously for single use. I don't use them on the laptop (there is some latency), so I prefer the wired earphones. But it seems like Bluetooth was aggressively scanning for them. After removing them from the system, the scanning remained, but reduced significantly.
>>
>> So looking at Console again, every so often, Bluetooth is scanning the network on behalf of the "sharingd" process.
>>
>> sharingd is a sharing daemon that supports features such as AirDrop, Handoff, Instant Hotspot, Shared Computers and Remote Disc in Finder.
>>
>> Still keeping Bluetooth off, however.
>>
>> Mark.
>

I'm not generally into conspiracy, but as I keep trying to work out the
issue I described in this thread, I came across this:

    https://sneak.berlin/20201112/your-computer-isnt-yours/

Might explain quite a lot, actually, (particularly at the FAQ section
under "When did this start?") and why it feels like macOS has taken a
huge dive (that I couldn't explain) since Mojave.

I still have High Sierra installed on my 2017 15-inch Mac, and will
likely keep it that way forever... if for nothing else, but as a
reference of what once was.

Staying away from Big Sur for as long as I can.

Mark.

On Tue, 17 Nov 2020 at 09:17, Mark Tinka <mark.tinka@seacom.com> wrote:

> https://sneak.berlin/20201112/your-computer-isnt-yours/
https://support.apple.com/en-us/HT202491

I am not trying to make any argument, just wanted to add context.

--
++ytti

On 11/17/20 09:26, Saku Ytti wrote:

> https://support.apple.com/en-us/HT202491
> I am not trying to make any argument, just wanted to add context.

Yes, saw that too, and that post by Apple is also highlighted (and
explained) in the same report.

The Gatekeeper OCSP checks remain unencrypted.

It still leave two glaring issues:

* Apple are still not saying anything about their OS apps bypassing
local firewalls and leaking our IP address and location past any
VPN's we may be running on Big Sur.

* The backdoor in iMessage's encryption that allows Apple and other
"interested parties" to view our iMessage texts.

Mark.

On Tue, 17 Nov 2020 at 09:35, Mark Tinka <mark.tinka@seacom.com> wrote:

> Yes, saw that too, and that post by Apple is also highlighted (and explained) in the same report.

Aah, I had not seen the updated version of it, thanks.

--
++ytti

The leaking past the VPN thing is pretty obnoxious. There are people who may be subject to policy and/or regulatory requirements that don’t permit split tunnels (even if supposedly not in userspace), so it will be interesting to see what burdens the use of an OS that intentionally leaks data will place on certain companies. In contrast, it’s pretty funny that while they let their own data collection apps leak past a tunnel to call home, they do not let the link local ipv6 traffic that Sidecar uses leak past a non-split VPN; i.e. if I’m on corporate VPN, I can no longer connect my tablet as a Sidecar monitor to my Macbook because that traffic is blocked.


From: NANOG <nanog-bounces+dhubbard=dino.hostasaurus.com@nanog.org> on behalf of Mark Tinka <mark.tinka@seacom.com>
Organization: SEACOM
Date: Tuesday, November 17, 2020 at 2:37 AM
To: Saku Ytti <saku@ytti.fi>
Cc: North American Network Operators Group <nanog@nanog.org>
Subject: Re: Apple Catalina Appears to Introduce Massive Jitter - SOLVED!


On 11/17/20 09:26, Saku Ytti wrote:
https://support.apple.com/en-us/HT202491

I am not trying to make any argument, just wanted to add context.

Yes, saw that too, and that post by Apple is also highlighted (and explained) in the same report.

The Gatekeeper OCSP checks remain unencrypted.

It still leave two glaring issues:

* Apple are still not saying anything about their OS apps bypassing local firewalls and leaking our IP address and location past any VPN's we may be running on Big Sur.

* The backdoor in iMessage's encryption that allows Apple and other "interested parties" to view our iMessage texts.
Mark.

So after installing Little Snitch and basically denying "trustd" any
kind of Internet access, I have been seeing reasonably normal jitter
with Bluetooth enabled.

It's not that Bluetooth stops scanning, but it's not scanning as
aggressively. So after a few minutes, there will be very high jitter
when Bluetooth scans the environment, but it would affect only a single
packet. It's easily reduced its chattiness by 99%.

I don't have any empirical data to support the claim that Little Snitch
has anything to do with it (and I am too lazy to dig further into it),
but the reduction in jitter is massively noticeable since Little Snitch.
Which means I can now run Catalina with Bluetooth enabled and not have
any wi-fi problems.

Just FYI, for the archives :-).

Mark.