Mailing List Archive

> On Sep 10, 2021, at 9:31 AM, Jason Kuehl <jason.w.kuehl@gmail.com> wrote:
>
> For whatever reason Comcast Xfinity is blocking my VPN URL. I've started the process to unblock, and I'm trying to get a hold of their security team to resolve this. I've been bounced around all morning.
>
> Does anyone have a contact at Comcast that can whitelist a URL or get me to a team that can understand what is going on for the block to happen?

Why is Comcast blocking things? That seems like it’s out of scope for an ISP.

—Chris

On Fri, 2021-09-10 at 10:31 -0400, Jason Kuehl wrote:
> For whatever reason Comcast Xfinity is blocking my VPN URL.

Not certain that this applies, but Concast Advanced Security (setup in
your Comcast gateway) only allows outbound VPN connections to UDP ports
500, 4500, and 62515 and TCP port 1723.

-Jim P.

By default, the cable modems from Comcast have Xfi Advanced
security-enabled which is a layer 3 URL blocker.

We can access our URL via that IP fine, but the URL fails.

The fix we're telling users is to 1st allow to unblock the URL in the APP,
then disable the service. Which does fix the issue.

I'm trying to find out why Comcast why they did the block to start with and
how to white list.

On Fri, Sep 10, 2021 at 10:57 AM Chris Boyd <cboyd@gizmopartners.com> wrote:

>
>
> > On Sep 10, 2021, at 9:31 AM, Jason Kuehl <jason.w.kuehl@gmail.com>
> wrote:
> >
> > For whatever reason Comcast Xfinity is blocking my VPN URL. I've started
> the process to unblock, and I'm trying to get a hold of their security team
> to resolve this. I've been bounced around all morning.
> >
> > Does anyone have a contact at Comcast that can whitelist a URL or get me
> to a team that can understand what is going on for the block to happen?
>
> Why is Comcast blocking things? That seems like it’s out of scope for an
> ISP.
>
> —Chris



--
Sincerely,

Jason W Kuehl
Cell 920-419-8983
jason.w.kuehl@gmail.com

This is an SSL VPN that is being blocked. This is what failure looks like.
Curl is the same.

Once we disable the Xfi Advanced Security everyone can connect.

[image: image.png]

On Fri, Sep 10, 2021 at 11:01 AM Jim Popovitch via NANOG <nanog@nanog.org>
wrote:

> On Fri, 2021-09-10 at 10:31 -0400, Jason Kuehl wrote:
> > For whatever reason Comcast Xfinity is blocking my VPN URL.
>
> Not certain that this applies, but Concast Advanced Security (setup in
> your Comcast gateway) only allows outbound VPN connections to UDP ports
> 500, 4500, and 62515 and TCP port 1723.
>
> -Jim P.
>
>

--
Sincerely,

Jason W Kuehl
Cell 920-419-8983
jason.w.kuehl@gmail.com

Could it be related to the many FortiNet devices being exploited? About 45k
credentials were dumped two days ago. Many are still working.


On Fri, Sep 10, 2021 at 10:56 AM Chris Boyd <cboyd@gizmopartners.com> wrote:

>
>
> > On Sep 10, 2021, at 9:31 AM, Jason Kuehl <jason.w.kuehl@gmail.com>
> wrote:
> >
> > For whatever reason Comcast Xfinity is blocking my VPN URL. I've started
> the process to unblock, and I'm trying to get a hold of their security team
> to resolve this. I've been bounced around all morning.
> >
> > Does anyone have a contact at Comcast that can whitelist a URL or get me
> to a team that can understand what is going on for the block to happen?
>
> Why is Comcast blocking things? That seems like it’s out of scope for an
> ISP.
>
> —Chris

I know this is not a solution to your problem, but I have found myself more
often running the public interface of openvpn systems on port 443. Any
sufficiently advanced DPI setup will be able to tell that it's not quite
normal https traffic.

But 99% of the time it seems to serve the purpose of defeating
heavily-restricted "free" wifi in airports, hotels, random guest/amenity
wifi stuff, which obviously can't block https/443 to the world these days.

On Fri, Sep 10, 2021 at 11:08 AM Jason Kuehl <jason.w.kuehl@gmail.com>
wrote:

> This is an SSL VPN that is being blocked. This is what failure looks like.
> Curl is the same.
>
> Once we disable the Xfi Advanced Security everyone can connect.
>
> [image: image.png]
>
> On Fri, Sep 10, 2021 at 11:01 AM Jim Popovitch via NANOG <nanog@nanog.org>
> wrote:
>
>> On Fri, 2021-09-10 at 10:31 -0400, Jason Kuehl wrote:
>> > For whatever reason Comcast Xfinity is blocking my VPN URL.
>>
>> Not certain that this applies, but Concast Advanced Security (setup in
>> your Comcast gateway) only allows outbound VPN connections to UDP ports
>> 500, 4500, and 62515 and TCP port 1723.
>>
>> -Jim P.
>>
>>
>
> --
> Sincerely,
>
> Jason W Kuehl
> Cell 920-419-8983
> jason.w.kuehl@gmail.com
>

First thing I do with any cable modem is convert it to bridge mode.

The fewer “smarts” in the cable modem doing odd things to my traffic, the better.

Owen


> On Sep 10, 2021, at 10:40 , Eric Kuhnke <eric.kuhnke@gmail.com> wrote:
>
> I know this is not a solution to your problem, but I have found myself more often running the public interface of openvpn systems on port 443. Any sufficiently advanced DPI setup will be able to tell that it's not quite normal https traffic.
>
> But 99% of the time it seems to serve the purpose of defeating heavily-restricted "free" wifi in airports, hotels, random guest/amenity wifi stuff, which obviously can't block https/443 to the world these days.
>
> On Fri, Sep 10, 2021 at 11:08 AM Jason Kuehl <jason.w.kuehl@gmail.com <mailto:jason.w.kuehl@gmail.com>> wrote:
> This is an SSL VPN that is being blocked. This is what failure looks like. Curl is the same.
>
> Once we disable the Xfi Advanced Security everyone can connect.
>
>
>
> On Fri, Sep 10, 2021 at 11:01 AM Jim Popovitch via NANOG <nanog@nanog.org <mailto:nanog@nanog.org>> wrote:
> On Fri, 2021-09-10 at 10:31 -0400, Jason Kuehl wrote:
> > For whatever reason Comcast Xfinity is blocking my VPN URL.
>
> Not certain that this applies, but Concast Advanced Security (setup in
> your Comcast gateway) only allows outbound VPN connections to UDP ports
> 500, 4500, and 62515 and TCP port 1723.
>
> -Jim P.
>
>
>
> --
> Sincerely,
>
> Jason W Kuehl
> Cell 920-419-8983
> jason.w.kuehl@gmail.com <mailto:jason.w.kuehl@gmail.com>

Ideally being your own customer owned cable modem that meets specs (Comcast
does allow this in some regions) that will function as a layer 2 bridge.

On Fri, Sep 10, 2021, 1:46 PM Owen DeLong <owen@delong.com> wrote:

> First thing I do with any cable modem is convert it to bridge mode.
>
> The fewer “smarts” in the cable modem doing odd things to my traffic, the
> better.
>
> Owen
>
>
> On Sep 10, 2021, at 10:40 , Eric Kuhnke <eric.kuhnke@gmail.com> wrote:
>
> I know this is not a solution to your problem, but I have found myself
> more often running the public interface of openvpn systems on port 443. Any
> sufficiently advanced DPI setup will be able to tell that it's not quite
> normal https traffic.
>
> But 99% of the time it seems to serve the purpose of defeating
> heavily-restricted "free" wifi in airports, hotels, random guest/amenity
> wifi stuff, which obviously can't block https/443 to the world these days.
>
> On Fri, Sep 10, 2021 at 11:08 AM Jason Kuehl <jason.w.kuehl@gmail.com>
> wrote:
>
>> This is an SSL VPN that is being blocked. This is what failure looks
>> like. Curl is the same.
>>
>> Once we disable the Xfi Advanced Security everyone can connect.
>>
>> [image: image.png]
>>
>> On Fri, Sep 10, 2021 at 11:01 AM Jim Popovitch via NANOG <nanog@nanog.org>
>> wrote:
>>
>>> On Fri, 2021-09-10 at 10:31 -0400, Jason Kuehl wrote:
>>> > For whatever reason Comcast Xfinity is blocking my VPN URL.
>>>
>>> Not certain that this applies, but Concast Advanced Security (setup in
>>> your Comcast gateway) only allows outbound VPN connections to UDP ports
>>> 500, 4500, and 62515 and TCP port 1723.
>>>
>>> -Jim P.
>>>
>>>
>>
>> --
>> Sincerely,
>>
>> Jason W Kuehl
>> Cell 920-419-8983
>> jason.w.kuehl@gmail.com
>>
>
>

We ran into this same issue for the first time yesterday too. Xfi Advanced Security started blocking our websockets endpoint, websocket.carsandbids.com/carsandbids. Our logs just showed a couple users failing to make the connection. We never would have figured it out except that one of our employees was getting blocked and was able to help us debug.

We submitted the url to https://spa.xfinity.com/report to get whitelisted, but haven’t heard back yet, the form said estimated three business days.

Matt Goldman
Backend Engineer
matt at carsandbids dot com

https://spa.xfinity.com should have a form to request removal. Note they say resolution time can be up to three business days

--
Alex Brotman
Sr. Engineer, Anti-Abuse & Messaging Policy
Comcast

From: NANOG <nanog-bounces+alex_brotman=comcast.com@nanog.org> On Behalf Of Jason Kuehl
Sent: Friday, September 10, 2021 10:31 AM
To: NANOG <nanog@nanog.org>
Subject: Xfi Advances Security (comcast)

For whatever reason Comcast Xfinity is blocking my VPN URL. I've started the process to unblock, and I'm trying to get a hold of their security team to resolve this. I've been bounced around all morning.

Does anyone have a contact at Comcast that can whitelist a URL or get me to a team that can understand what is going on for the block to happen?

--
Sincerely,

Jason W Kuehl
Cell 920-419-8983
jason.w.kuehl@gmail.com<mailto:jason.w.kuehl@gmail.com>

Yes, I own my own modem even though comcast now charges me $5/month more than if I rented their equipment for this privilege.

Owen


> On Sep 10, 2021, at 15:49 , Eric Kuhnke <eric.kuhnke@gmail.com> wrote:
>
> Ideally being your own customer owned cable modem that meets specs (Comcast does allow this in some regions) that will function as a layer 2 bridge.
>
> On Fri, Sep 10, 2021, 1:46 PM Owen DeLong <owen@delong.com <mailto:owen@delong.com>> wrote:
> First thing I do with any cable modem is convert it to bridge mode.
>
> The fewer “smarts” in the cable modem doing odd things to my traffic, the better.
>
> Owen
>
>
>> On Sep 10, 2021, at 10:40 , Eric Kuhnke <eric.kuhnke@gmail.com <mailto:eric.kuhnke@gmail.com>> wrote:
>>
>> I know this is not a solution to your problem, but I have found myself more often running the public interface of openvpn systems on port 443. Any sufficiently advanced DPI setup will be able to tell that it's not quite normal https traffic.
>>
>> But 99% of the time it seems to serve the purpose of defeating heavily-restricted "free" wifi in airports, hotels, random guest/amenity wifi stuff, which obviously can't block https/443 to the world these days.
>>
>> On Fri, Sep 10, 2021 at 11:08 AM Jason Kuehl <jason.w.kuehl@gmail.com <mailto:jason.w.kuehl@gmail.com>> wrote:
>> This is an SSL VPN that is being blocked. This is what failure looks like. Curl is the same.
>>
>> Once we disable the Xfi Advanced Security everyone can connect.
>>
>>
>>
>> On Fri, Sep 10, 2021 at 11:01 AM Jim Popovitch via NANOG <nanog@nanog.org <mailto:nanog@nanog.org>> wrote:
>> On Fri, 2021-09-10 at 10:31 -0400, Jason Kuehl wrote:
>> > For whatever reason Comcast Xfinity is blocking my VPN URL.
>>
>> Not certain that this applies, but Concast Advanced Security (setup in
>> your Comcast gateway) only allows outbound VPN connections to UDP ports
>> 500, 4500, and 62515 and TCP port 1723.
>>
>> -Jim P.
>>
>>
>>
>> --
>> Sincerely,
>>
>> Jason W Kuehl
>> Cell 920-419-8983
>> jason.w.kuehl@gmail.com <mailto:jason.w.kuehl@gmail.com>

On 9/10/21, 10:58, "NANOG on behalf of Chris Boyd" <nanog-bounces+jason_livingood=cable.comcast.com@nanog.org on behalf of cboyd@gizmopartners.com> wrote:

> Why is Comcast blocking things? That seems like it’s out of scope for an ISP.

For Internet access, sure. But ISPs also have value added protection services and this part of an optional content filtering service that is integrated into the leased Comcast gateways. Users can turn on things like parental controls, including time limit and time-of-day boundaries for certain devices (e.g. cut off kid's game console Internet access at midnight on school nights). See https://www.xfinity.com/support/articles/using-xfinity-xfi-advanced-security

Jason

As Alex said, you can submit a request to review a block at https://spa.xfinity.com<https://urldefense.com/v3/__https:/spa.xfinity.com__;!!CQl3mcHX2A!VFRCR2r6w4y6BDhy4gmaIa2JdxJVoUxgzRD48A1CG_X6a9Nq8gN2Qjie7Yzk8C5y_XSXg-Dd$>. Note that this service relies substantially on 3rd party list sources – so if any IP/FQDN appears on other lists (e.g. webroot and similar) then it may be here as well. So you may want to take a look more broadly, especially if you rely on any virtual infrastructure.

Thanks
Jason

From: NANOG <nanog-bounces+jason_livingood=cable.comcast.com@nanog.org> on behalf of Jason Kuehl <jason.w.kuehl@gmail.com>
Date: Friday, September 10, 2021 at 11:10
To: Jim Popovitch <jimpop@domainmail.org>
Cc: NANOG <nanog@nanog.org>
Subject: Re: Xfi Advances Security (comcast)

This is an SSL VPN that is being blocked. This is what failure looks like. Curl is the same.

Once we disable the Xfi Advanced Security everyone can connect.

[cid:ii_ktehov470]

On Fri, Sep 10, 2021 at 11:01 AM Jim Popovitch via NANOG <nanog@nanog.org<mailto:nanog@nanog.org>> wrote:
On Fri, 2021-09-10 at 10:31 -0400, Jason Kuehl wrote:
> For whatever reason Comcast Xfinity is blocking my VPN URL.

Not certain that this applies, but Concast Advanced Security (setup in
your Comcast gateway) only allows outbound VPN connections to UDP ports
500, 4500, and 62515 and TCP port 1723.

-Jim P.


--
Sincerely,

Jason W Kuehl
Cell 920-419-8983
jason.w.kuehl@gmail.com<mailto:jason.w.kuehl@gmail.com>

> On Sep 13, 2021, at 07:56 , Livingood, Jason via NANOG <nanog@nanog.org> wrote:
>
> On 9/10/21, 10:58, "NANOG on behalf of Chris Boyd" <nanog-bounces+jason_livingood=cable.comcast.com@nanog.org on behalf of cboyd@gizmopartners.com> wrote:
>
>> Why is Comcast blocking things? That seems like it’s out of scope for an ISP.
>
> For Internet access, sure. But ISPs also have value added protection services and this part of an optional content filtering service that is integrated into the leased Comcast gateways. Users can turn on things like parental controls, including time limit and time-of-day boundaries for certain devices (e.g. cut off kid's game console Internet access at midnight on school nights). See https://www.xfinity.com/support/articles/using-xfinity-xfi-advanced-security
>
> Jason
>
>

Yes, but it’s tragically opt-out instead of opt-in as it should be. That means that anyone whose site happens to get miscategorized by them gets the added costs of dealing with the user complaints instead of Comcast having to bear the costs of their error.

It’s a classic example of the toxic polluter business model. Do something stupid while making sure that the costs of your errors fall on someone else.

Owen

On 9/13/21, 12:02, "Owen DeLong" <owen@delong.com> wrote:
> Yes, but it’s tragically opt-out instead of opt-in as it should be.

It is not a default for an Internet access service. It comes bundled as one of several features in an optional add on service. See https://www.xfinity.com/learn/internet-service/modems-and-routers for details. This is targeted at the average consumer, particularly those that may want parental controls, mesh WiFi, a voice port, and so on - so not really targeted at NANOG list subs like us. ;-) That said, I have an XB7 modem at home and really like it a lot - especially the new AQM feature that dramatically lowered working latency.

> That means that anyone whose site happens to get miscategorized by them gets the added costs of dealing with the user complaints instead of Comcast having to bear the costs of their error.

As my other reply noted, this service uses a bunch of 3rd party services and it is those 3rd parties that maintain the lists (a la anti-spam and anti-phishing email list vendors). So if an IP/FQDN/URL happens to be on "our" list it is very likely getting filtered/blocked in a lot of network places because it is on a well-known independent list.

BUT, how do we know that was even the case here? Do we have a traceroute or a screen shot of an error or block message? We seem to have concluded it was blocked by a content filter but what technical evidence do we have (that can help troubleshoot)? I know you are not the OP (it is Chris) - but I'd love to know more technical detail and I am in communication off-list with the OP (along with my colleague Tony Tauber, who was the first to reach out to Chris 1:1).

Jason