Mailing List Archive

I have run into this before using 1 Gigabit SFP optics in the uplink module
on EX switches. We disabled auto-negotiation on the uplink and didn't an
issue. See https://kb.juniper.net/InfoCenter/index?page=content&id=KB34060

It's definitely not my preferred way of handling the situation, but it
worked, and as long as it's documented it shouldn't become an problem in
the future.

On Tue, May 5, 2020 at 11:55 AM Norman Elton <normelton@gmail.com> wrote:

> Hello!
>
> We've got a sizable deployment of EX-3400s on campus, and have noticed
> that after a reboot, some switches fail to bring their uplink
> connection online. Connections between two EX switches tend to work
> fine, connections between an EX-3400 and an MX or SRX fail much more
> frequently. In some cases, it fails every single time. In all cases,
> we can down the interface, bring it back up, and things are fine. It
> seems to only fail immediately after a reboot. We've tried changing
> fiber, changing SFPs, etc. Nothing.
>
> We've worked with JTAC, and our account team, on a resolution. In both
> cases, the answer from engineering is that this is a hardware chipset
> problem, and not resolvable in software. The workaround is to disable
> autonegotiation on uplink interfaces.
>
> Has anyone else experienced this phenomenon? If it's a chipset issue,
> I would think it's present in other vendor's products?
>
> We've long standardized on running autonegotiation, everywhere. Do
> others actually disable autonegotiation on "managed" connections,
> where you're configuring both ends of the link?
>
> Thanks!
>
> Norman
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>


--
- Anthony

http://www.linkedin.com/in/anthonyjohnson
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Norman,

Curious what Junos versions you hit this on, and what speed the
uplinks were? When you say you standardize on running autonegotiation,
are you adding this non-default portion to the config:

set interfaces xe-0/2/0 ether-options auto-negotiation

And Juniper's recommendation is below, or to simply not add above?

set interfaces xe-0/2/0 ether-options no-auto-negotiation

We don't have any of these in production but have one as a test for a
possible project, hence the curiosity. We've had a lab one running
18.2R3-S2.9 with LR SFP+ uplink with no issues- we have no
ether-options set at all.

Cheers,
Chris
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi,

On Tue, May 05, 2020 at 12:51:45PM -0400, Norman Elton wrote:
> We've got a sizable deployment of EX-3400s on campus, and have noticed
> that after a reboot, some switches fail to bring their uplink
> connection online.

Never seen a problem with our 3400s (3rd party SFP+ modules, uplinked
to Aristas mostly, some to Cisco ASR9k).

18.1R3.3 and 15.1X53-D59.4 here.

> We've long standardized on running autonegotiation, everywhere. Do
> others actually disable autonegotiation on "managed" connections,
> where you're configuring both ends of the link?

Devices or Carriers disabling autoneg are a major PITA and need to
rot in hell. It's a mandatory part of the standard and should be
treated as such.

(So, no, we never disable autoneg unless talking to something stupid)

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

Hi,

On Tue, May 05, 2020 at 08:42:20PM +0200, Gert Doering wrote:
> > We've long standardized on running autonegotiation, everywhere. Do
> > others actually disable autonegotiation on "managed" connections,
> > where you're configuring both ends of the link?
>
> Devices or Carriers disabling autoneg are a major PITA and need to
> rot in hell. It's a mandatory part of the standard and should be
> treated as such.
>
> (So, no, we never disable autoneg unless talking to something stupid)

Actually it's only mandatory on 1000Base-TX.

The thing is that auto-negotiation on 1000Base-X includes link fault
signaling, which is great... and not a mandatory part of the 1000Base-X
standard, albeit often implemented.

It is not an issue as long as the remote party implements
auto-negotiation on 1000Base-X (which is optional) and link fault
signaling (which is an optional part of auto-negotiation) and doesn't
use an SGMII-based transceiver as they don't support link fault signaling
(I don't know about RGMII-based transceivers but I wouldn't bet on them
having the management interface for it).

In the unlikely (right?) event that the aforementioned conditions are
not met, for the link to go up, one solution is to disable
auto-negotiation entirely. Another one is to play with the
`auto-negotiation remote-fault` knob if you're using a MX/ACX device
(good luck understanding the documentation on that one).

I am not advocating for disabling auto-neg (and link fault signaling) on
1G fibre links, on the contrary, but there are good and valid reasons to
do so.

Benjamin
--
Benjamin Collet
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Chris,

>> Curious what Junos versions you hit this on, and what speed the
>> uplinks were?

We're running 18.2R3-S3.11.

>> When you say you standardize on running autonegotiation,
>> are you adding this non-default portion to the config:

Nope. These are 1G SFP connections, no configuration modifications to
the default autonegotiation settings.

Though your note leads me to believe that this might be a 1G problem.
Where possible, upgrading to 10G uplinks would get us out of the
problem. Fiberstore to the rescue!

Norman

On Tue, May 5, 2020 at 1:53 PM Chris Wopat <me@falz.net> wrote:
>
> Norman,
>
> Curious what Junos versions you hit this on, and what speed the
> uplinks were? When you say you standardize on running autonegotiation,
> are you adding this non-default portion to the config:
>
> set interfaces xe-0/2/0 ether-options auto-negotiation
>
> And Juniper's recommendation is below, or to simply not add above?
>
> set interfaces xe-0/2/0 ether-options no-auto-negotiation
>
> We don't have any of these in production but have one as a test for a
> possible project, hence the curiosity. We've had a lab one running
> 18.2R3-S2.9 with LR SFP+ uplink with no issues- we have no
> ether-options set at all.
>
> Cheers,
> Chris
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Benjamin,

Your note made me realize there are multiple levels of autonegotiation
that I had no idea even existed. I see a wikipedia spiral in my
future!

Thanks for the recap.

Norman

On Sun, May 10, 2020 at 9:55 AM Benjamin Collet <juniper-nsp@clt.tf> wrote:
>
> Hi,
>
> On Tue, May 05, 2020 at 08:42:20PM +0200, Gert Doering wrote:
> > > We've long standardized on running autonegotiation, everywhere. Do
> > > others actually disable autonegotiation on "managed" connections,
> > > where you're configuring both ends of the link?
> >
> > Devices or Carriers disabling autoneg are a major PITA and need to
> > rot in hell. It's a mandatory part of the standard and should be
> > treated as such.
> >
> > (So, no, we never disable autoneg unless talking to something stupid)
>
> Actually it's only mandatory on 1000Base-TX.
>
> The thing is that auto-negotiation on 1000Base-X includes link fault
> signaling, which is great... and not a mandatory part of the 1000Base-X
> standard, albeit often implemented.
>
> It is not an issue as long as the remote party implements
> auto-negotiation on 1000Base-X (which is optional) and link fault
> signaling (which is an optional part of auto-negotiation) and doesn't
> use an SGMII-based transceiver as they don't support link fault signaling
> (I don't know about RGMII-based transceivers but I wouldn't bet on them
> having the management interface for it).
>
> In the unlikely (right?) event that the aforementioned conditions are
> not met, for the link to go up, one solution is to disable
> auto-negotiation entirely. Another one is to play with the
> `auto-negotiation remote-fault` knob if you're using a MX/ACX device
> (good luck understanding the documentation on that one).
>
> I am not advocating for disabling auto-neg (and link fault signaling) on
> 1G fibre links, on the contrary, but there are good and valid reasons to
> do so.
>
> Benjamin
> --
> Benjamin Collet
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp