Mailing List Archive

Link establishment issues with 1Gbps SX/LX SFPs on QFX5110
Hi All,

We've been battling with a number of issues getting 1G Base-X links to work on a batch of new QFX5110-48S switches, in use as Fusion devices connected to MX204.

No problems with a variety of 10G and 100G connections. A lot of 1G works fine but we seem to have issues with auto-negotiate with both SX and LX optics. JTAC are a bit lost and we've tried a variety of firmwares with varying success. Turning auto-negotiate off on both sides seems to fix most of this, but that's not viable in a lot of cases where there's a 3rd party involved.

Anyone seen similar and aware of a fix?

Regards,

Tim
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
Hi Tim

Can confirm I've seen exactly the same issue on QFX5110 18.4R1.8 with the only way I can get fs.com LX 1G transceivers working being turning off auto-neg.

Have also seen strange issues with 1G copper transceivers just not showing up in chassis hardware n some versions, and working fine in older versions.

Shaffi

-----Original Message-----
From: juniper-nsp <juniper-nsp-bounces@puck.nether.net> On Behalf Of Timothy Creswick
Sent: Tuesday, 25 June 2019 7:50 AM
To: 'juniper-nsp@puck.nether.net' <juniper-nsp@puck.nether.net>
Subject: [j-nsp] Link establishment issues with 1Gbps SX/LX SFPs on QFX5110

Hi All,

We've been battling with a number of issues getting 1G Base-X links to work on a batch of new QFX5110-48S switches, in use as Fusion devices connected to MX204.

No problems with a variety of 10G and 100G connections. A lot of 1G works fine but we seem to have issues with auto-negotiate with both SX and LX optics. JTAC are a bit lost and we've tried a variety of firmwares with varying success. Turning auto-negotiate off on both sides seems to fix most of this, but that's not viable in a lot of cases where there's a 3rd party involved.

Anyone seen similar and aware of a fix?

Regards,

Tim
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
> Can confirm I've seen exactly the same issue on QFX5110 18.4R1.8 with the only way I can get fs.com LX 1G
> transceivers working being turning off auto-neg.

Thanks. We've been looking at the Broadcom tools on the QFX and note some apparently quite major differences in the way that the interfaces negotiate across versions.

In the older satellite verions (e.g. 2.0R1.1), GE interfaces appear to have the correct auto neg state set on the chipset. This seems to coincide with it using GMII between the SFP and the board. This reflects the way the ports are configured in Junos:

ena/ speed/ link auto STP lrn inter max loop
port link duplex scan neg? state pause discrd ops face frame back
xe4 up 1G FD HW Yes Forward None F GMII 1604
xe6 up 1G FD HW No Forward None F GMII 1522
xe10 up 1G FD HW Yes Forward None F GMII 1526
xe11 up 1G FD HW Yes Forward None F GMII 1526
xe12 up 1G FD HW Yes Forward None F GMII 1526

On newer versions (e.g. 3.5R2.2), GE interfaces are always shown as auto-neg disabled on the chipset:

ena/ speed/ link auto STP lrn inter max loop
port link duplex scan neg? state pause discrd ops face frame back
ge0( 3) up 1G FD HW No Forward None F SGMII 1518
ge1( 4) up 1G FD HW No Forward None F SGMII 1518
ge2( 6) down 1G FD HW No Forward None F SGMII 1518
ge3( 12) !ena 1G FD HW No Forward None F SGMII 1518
ge4( 13) up 1G FD HW No Forward None F SGMII 1526
ge5( 15) up 1G FD HW No Forward None F SGMII 1526

They will show as up here and in Junos but not on the remote device.

Forcefully setting the interface to '[gig]ether-options auto-negotiate' does not change the bcm setting.

If you forcefully change the bcm setting (i.e. tell the chipset to auto-neg), the port goes down both in the bcm shell and in Junos.

Interestingly, some of these ports work, others do not. We haven't been able to identify a pattern and JTAC are still looking.

We also can't explain why these ports indicate they are SGMII.

At the behest of JTAC we put these on 3.5R1-S4.0. This has the characteristic that in the following example we have a MM (port 1) and SM (port 3) transceiver. Both are Flexoptix correctly coded (we've tried others). Port 1 gets link successfully, Port 3 does not. Note that now the working port is back to GMII:

ena/ speed/ link auto STP lrn inter max loop
port link duplex scan neg? state pause discrd ops face frame back
ge0( 1) up 1G FD HW Yes Forward None F GMII 1518
ge1( 3) down 10M FD HW Yes Forward None F SGMII 1518

Can anyone shed any more light on this?

Regards,

Tim
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
What version of code are you using? When I worked at Juniper I delt with a
ISP that had this issue and I believe 17.3R3 did not have the issue at all.

There was an issue with BCM and the Juniper code. You can manually force
the port up in the BCM shell (not that is a viable long term solution).

FPC0( vty)# set dcbcm bcmshell "ps 45"

HW (unit 0)
ena/ speed/ link auto STP lrn
inter max loop
port link duplex scan neg? state pause discrd ops
face frame back
ge0( 45) down 1G FD HW Yes Block None FA
SGMII 1518

FPC0( vty)# set dcbcm bcmshell "port 45 an=1 e=t"

HW (unit 0)

FPC0( vty)# set dcbcm bcmshell "ps 45"

HW (unit 0)
ena/ speed/ link auto STP lrn
inter max loop
port link duplex scan neg? state pause discrd ops
face frame back
ge0( 45) !ena 1G FD None No Block None D
SGMII 9412




On Tue, Jun 25, 2019 at 1:15 AM Timothy Creswick <timothy@creswick.eu>
wrote:

> > Can confirm I've seen exactly the same issue on QFX5110 18.4R1.8 with
> the only way I can get fs.com LX 1G
> > transceivers working being turning off auto-neg.
>
> Thanks. We've been looking at the Broadcom tools on the QFX and note some
> apparently quite major differences in the way that the interfaces negotiate
> across versions.
>
> In the older satellite verions (e.g. 2.0R1.1), GE interfaces appear to
> have the correct auto neg state set on the chipset. This seems to coincide
> with it using GMII between the SFP and the board. This reflects the way the
> ports are configured in Junos:
>
> ena/ speed/ link auto STP lrn inter
> max loop
> port link duplex scan neg? state pause discrd ops face
> frame back
> xe4 up 1G FD HW Yes Forward None F GMII
> 1604
> xe6 up 1G FD HW No Forward None F GMII
> 1522
> xe10 up 1G FD HW Yes Forward None F GMII
> 1526
> xe11 up 1G FD HW Yes Forward None F GMII
> 1526
> xe12 up 1G FD HW Yes Forward None F GMII
> 1526
>
> On newer versions (e.g. 3.5R2.2), GE interfaces are always shown as
> auto-neg disabled on the chipset:
>
> ena/ speed/ link auto STP lrn
> inter max loop
> port link duplex scan neg? state pause discrd ops
> face frame back
> ge0( 3) up 1G FD HW No Forward None F
> SGMII 1518
> ge1( 4) up 1G FD HW No Forward None F
> SGMII 1518
> ge2( 6) down 1G FD HW No Forward None F
> SGMII 1518
> ge3( 12) !ena 1G FD HW No Forward None F
> SGMII 1518
> ge4( 13) up 1G FD HW No Forward None F
> SGMII 1526
> ge5( 15) up 1G FD HW No Forward None F
> SGMII 1526
>
> They will show as up here and in Junos but not on the remote device.
>
> Forcefully setting the interface to '[gig]ether-options auto-negotiate'
> does not change the bcm setting.
>
> If you forcefully change the bcm setting (i.e. tell the chipset to
> auto-neg), the port goes down both in the bcm shell and in Junos.
>
> Interestingly, some of these ports work, others do not. We haven't been
> able to identify a pattern and JTAC are still looking.
>
> We also can't explain why these ports indicate they are SGMII.
>
> At the behest of JTAC we put these on 3.5R1-S4.0. This has the
> characteristic that in the following example we have a MM (port 1) and SM
> (port 3) transceiver. Both are Flexoptix correctly coded (we've tried
> others). Port 1 gets link successfully, Port 3 does not. Note that now the
> working port is back to GMII:
>
> ena/ speed/ link auto STP lrn
> inter max loop
> port link duplex scan neg? state pause discrd ops
> face frame back
> ge0( 1) up 1G FD HW Yes Forward None F
> GMII 1518
> ge1( 3) down 10M FD HW Yes Forward None F
> SGMII 1518
>
> Can anyone shed any more light on this?
>
> Regards,
>
> Tim
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
Hi,

> What version of code are you using? When I worked at Juniper I delt with a ISP that had this issue and I believe
> 17.3R3 did not have the issue at all.

It's basically a 18.x release, we think (hard to be sure because they are SNOS / Satellite releases).

> There was an issue with BCM and the Juniper code. You can manually force the port up in the BCM shell (not that is a
> viable long term solution).
>
> FPC0( vty)# set dcbcm bcmshell "port 45 an=1 e=t"

Yes, was aware of that and have been using this in our testing. It's not a problem to force the link up, the issue is that the other side still sees the link as down so there is no received traffic. We aren't in a position to always disable auto-neg on the other side.
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
The same problem exists on 18.x on EX4300 standalone or VC (not Fusion).

On Mon, Jun 24, 2019 at 09:49:45PM +0000, Timothy Creswick wrote:
> Hi All,
>
> We've been battling with a number of issues getting 1G Base-X links to work on a batch of new QFX5110-48S switches, in use as Fusion devices connected to MX204.
>
> No problems with a variety of 10G and 100G connections. A lot of 1G works fine but we seem to have issues with auto-negotiate with both SX and LX optics. JTAC are a bit lost and we've tried a variety of firmwares with varying success. Turning auto-negotiate off on both sides seems to fix most of this, but that's not viable in a lot of cases where there's a 3rd party involved.
>
> Anyone seen similar and aware of a fix?
>
> Regards,
>
> Tim
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
Here is a sampling of PRs (there have been a _lot_ in the PR subscription
emails recently):


(EX4300)
PR1420343
<https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1420343&actp=SUBSCRIPTION>
When
using 1G SFP in 10G ports, neet to configure interface as no-autoneg

PR1422958
<https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1422958&actp=SUBSCRIPTION>
QFX5100-48T
10G interface might be auto-negotiated at 1G speed instead of 10G

PR1411852
<https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1411852&actp=SUBSCRIPTION>
V44
QFX5110 : auto-negotiation is not disabled in hardware after setting
no-auto-negotiation option in cli

--
Eldon

On Tue, Jun 25, 2019, 09:13 Anderson, Charles R <cra@wpi.edu> wrote:

> The same problem exists on 18.x on EX4300 standalone or VC (not Fusion).
>
> On Mon, Jun 24, 2019 at 09:49:45PM +0000, Timothy Creswick wrote:
> > Hi All,
> >
> > We've been battling with a number of issues getting 1G Base-X links to
> work on a batch of new QFX5110-48S switches, in use as Fusion devices
> connected to MX204.
> >
> > No problems with a variety of 10G and 100G connections. A lot of 1G
> works fine but we seem to have issues with auto-negotiate with both SX and
> LX optics. JTAC are a bit lost and we've tried a variety of firmwares with
> varying success. Turning auto-negotiate off on both sides seems to fix most
> of this, but that's not viable in a lot of cases where there's a 3rd party
> involved.
> >
> > Anyone seen similar and aware of a fix?
> >
> > Regards,
> >
> > Tim
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
And a TSB:

TSB17538
<https://kb.juniper.net/InfoCenter/index?page=content&id=TSB17538&actp=SUBSCRIPTION>
Traffic
drop is seen on EX4300 when 10G Fiber port is using 1 Gigabit Ethernet SFP
optics with Auto Negotiation enabled

--
Eldon


On Tue, Jun 25, 2019, 23:01 Eldon Koyle <ekoyle+puck.nether.net@gmail.com>
wrote:

> Here is a sampling of PRs (there have been a _lot_ in the PR subscription
> emails recently):
>
>
> (EX4300)
> PR1420343
> <https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1420343&actp=SUBSCRIPTION> When
> using 1G SFP in 10G ports, neet to configure interface as no-autoneg
>
> PR1422958
> <https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1422958&actp=SUBSCRIPTION> QFX5100-48T
> 10G interface might be auto-negotiated at 1G speed instead of 10G
>
> PR1411852
> <https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1411852&actp=SUBSCRIPTION> V44
> QFX5110 : auto-negotiation is not disabled in hardware after setting
> no-auto-negotiation option in cli
>
> --
> Eldon
>
> On Tue, Jun 25, 2019, 09:13 Anderson, Charles R <cra@wpi.edu> wrote:
>
>> The same problem exists on 18.x on EX4300 standalone or VC (not Fusion).
>>
>> On Mon, Jun 24, 2019 at 09:49:45PM +0000, Timothy Creswick wrote:
>> > Hi All,
>> >
>> > We've been battling with a number of issues getting 1G Base-X links to
>> work on a batch of new QFX5110-48S switches, in use as Fusion devices
>> connected to MX204.
>> >
>> > No problems with a variety of 10G and 100G connections. A lot of 1G
>> works fine but we seem to have issues with auto-negotiate with both SX and
>> LX optics. JTAC are a bit lost and we've tried a variety of firmwares with
>> varying success. Turning auto-negotiate off on both sides seems to fix most
>> of this, but that's not viable in a lot of cases where there's a 3rd party
>> involved.
>> >
>> > Anyone seen similar and aware of a fix?
>> >
>> > Regards,
>> >
>> > Tim
>> _______________________________________________
>> juniper-nsp mailing list juniper-nsp@puck.nether.net
>> https://puck.nether.net/mailman/listinfo/juniper-nsp
>>
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
For anyone interested in this, JTAC released to us 3.5R1-S4.1 for the satellite devices (I don't believe this will be generally available until tomorrow). This addresses the issue and confirms that there were 2 or 3 different PRs relating to the Broadcom chipset being incorrectly programmed by Junos.

Some output, testing with a variety of 1Gbps optics (coded SX, LX, uncoded BX, LX and SX).

> show chassis hardware satellite
Hardware inventory:
Item Version Part number Serial number Description
FPC 100 REV 27 650-061152 WS37183----X QFX5110-48S-4C
PIC 0 REV 27 650-061152 WS37183----X 48x10G-4x100G
Xcvr 0 850nm 740-011782 F----------X SFP-SX
Xcvr 2 1310nm 740-011783 F----------X SFP-LX10
Xcvr 4 NON-JNPR VS51208----X SFP-1000BASE-BX10-D
Xcvr 6 NON-JNPR VS60930----X SFP-LX10
Xcvr 8 NON-JNPR VS50930----X SFP-SX

> show interfaces ge-100/0/[02468] terse
Interface Admin Link Proto Local Remote
ge-100/0/0 up up
ge-100/0/2 up up
ge-100/0/4 up up
ge-100/0/6 up up
ge-100/0/8 up up

> show configuration interfaces
ge-100/0/0 {
unit 0;
}
ge-100/0/2 {
unit 0;
}
ge-100/0/4 {
unit 0;
}
ge-100/0/6 {
unit 0;
}
ge-100/0/8 {
unit 0;
}

# bcmshell ps 1,3,5,7,9

HW (unit 0)
ena/ speed/ link auto STP lrn inter max loop
port link duplex scan neg? state pause discrd ops face frame back
ge0( 1) up 1G FD HW Yes Forward None F GMII 1518
ge1( 3) up 1G FD HW Yes Forward None F GMII 1518
ge2( 5) up 1G FD HW Yes Forward None F GMII 1518
ge3( 7) up 1G FD HW Yes Forward None F GMII 1518
ge4( 9) up 1G FD HW Yes Forward None F GMII 1518

We note that now changing the auto-negotiate state in Junos correctly updates the underlying chipset and link negotiation works correctly. Interface is shown as GMII which we understand to be correct also.

All-in, pretty terrible that - as far as we can tell - 1Gbps ports have *never worked* on QFX5110 properly in all the four basic combinations (i.e. SX and LX at both auto- and no-auto negotiation) until this release. Quite how Juniper have been shipping the hardware for so long in that state is a mystery to me.


Tim
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
I have the same problem on QFX5120 which is running the new Trident chipset.
Model: qfx5120-48y-8c
Junos: 18.4R1-S2.4

I have some 1310nm LR and different wavelength 1G SFP's from FS.com.
I tried setting autoneg on and off under etheroptions on juniper side, aswell as manual speed etc..
None of the config combinations I tried on Juniper side got auto negotiation working for me. However, Juniper side always reported link as UP, but remote DOWN.

The solution for me was to set autoneg=no on remote side (Mikrotik Routerboard).


Olof


________________________________
From: juniper-nsp <juniper-nsp-bounces@puck.nether.net> on behalf of Timothy Creswick <timothy@creswick.eu>
Sent: Friday, June 28, 2019 9:22 PM
To: Juniper List
Subject: Re: [j-nsp] Link establishment issues with 1Gbps SX/LX SFPs on QFX5110

For anyone interested in this, JTAC released to us 3.5R1-S4.1 for the satellite devices (I don't believe this will be generally available until tomorrow). This addresses the issue and confirms that there were 2 or 3 different PRs relating to the Broadcom chipset being incorrectly programmed by Junos.

Some output, testing with a variety of 1Gbps optics (coded SX, LX, uncoded BX, LX and SX).

> show chassis hardware satellite
Hardware inventory:
Item Version Part number Serial number Description
FPC 100 REV 27 650-061152 WS37183----X QFX5110-48S-4C
PIC 0 REV 27 650-061152 WS37183----X 48x10G-4x100G
Xcvr 0 850nm 740-011782 F----------X SFP-SX
Xcvr 2 1310nm 740-011783 F----------X SFP-LX10
Xcvr 4 NON-JNPR VS51208----X SFP-1000BASE-BX10-D
Xcvr 6 NON-JNPR VS60930----X SFP-LX10
Xcvr 8 NON-JNPR VS50930----X SFP-SX

> show interfaces ge-100/0/[02468] terse
Interface Admin Link Proto Local Remote
ge-100/0/0 up up
ge-100/0/2 up up
ge-100/0/4 up up
ge-100/0/6 up up
ge-100/0/8 up up

> show configuration interfaces
ge-100/0/0 {
unit 0;
}
ge-100/0/2 {
unit 0;
}
ge-100/0/4 {
unit 0;
}
ge-100/0/6 {
unit 0;
}
ge-100/0/8 {
unit 0;
}

# bcmshell ps 1,3,5,7,9

HW (unit 0)
ena/ speed/ link auto STP lrn inter max loop
port link duplex scan neg? state pause discrd ops face frame back
ge0( 1) up 1G FD HW Yes Forward None F GMII 1518
ge1( 3) up 1G FD HW Yes Forward None F GMII 1518
ge2( 5) up 1G FD HW Yes Forward None F GMII 1518
ge3( 7) up 1G FD HW Yes Forward None F GMII 1518
ge4( 9) up 1G FD HW Yes Forward None F GMII 1518

We note that now changing the auto-negotiate state in Junos correctly updates the underlying chipset and link negotiation works correctly. Interface is shown as GMII which we understand to be correct also.

All-in, pretty terrible that - as far as we can tell - 1Gbps ports have *never worked* on QFX5110 properly in all the four basic combinations (i.e. SX and LX at both auto- and no-auto negotiation) until this release. Quite how Juniper have been shipping the hardware for so long in that state is a mystery to me.


Tim
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
Do you know when this will be fixed in the mainline releases? I have no
clue what 3.5R1-S4.1 is?

On Fri, Jun 28, 2019 at 2:22 PM Timothy Creswick <timothy@creswick.eu>
wrote:

> For anyone interested in this, JTAC released to us 3.5R1-S4.1 for the
> satellite devices (I don't believe this will be generally available until
> tomorrow). This addresses the issue and confirms that there were 2 or 3
> different PRs relating to the Broadcom chipset being incorrectly programmed
> by Junos.
>
> Some output, testing with a variety of 1Gbps optics (coded SX, LX, uncoded
> BX, LX and SX).
>
> > show chassis hardware satellite
> Hardware inventory:
> Item Version Part number Serial number Description
> FPC 100 REV 27 650-061152 WS37183----X QFX5110-48S-4C
> PIC 0 REV 27 650-061152 WS37183----X 48x10G-4x100G
> Xcvr 0 850nm 740-011782 F----------X SFP-SX
> Xcvr 2 1310nm 740-011783 F----------X SFP-LX10
> Xcvr 4 NON-JNPR VS51208----X
> SFP-1000BASE-BX10-D
> Xcvr 6 NON-JNPR VS60930----X SFP-LX10
> Xcvr 8 NON-JNPR VS50930----X SFP-SX
>
> > show interfaces ge-100/0/[02468] terse
> Interface Admin Link Proto Local Remote
> ge-100/0/0 up up
> ge-100/0/2 up up
> ge-100/0/4 up up
> ge-100/0/6 up up
> ge-100/0/8 up up
>
> > show configuration interfaces
> ge-100/0/0 {
> unit 0;
> }
> ge-100/0/2 {
> unit 0;
> }
> ge-100/0/4 {
> unit 0;
> }
> ge-100/0/6 {
> unit 0;
> }
> ge-100/0/8 {
> unit 0;
> }
>
> # bcmshell ps 1,3,5,7,9
>
> HW (unit 0)
> ena/ speed/ link auto STP lrn
> inter max loop
> port link duplex scan neg? state pause discrd ops
> face frame back
> ge0( 1) up 1G FD HW Yes Forward None F
> GMII 1518
> ge1( 3) up 1G FD HW Yes Forward None F
> GMII 1518
> ge2( 5) up 1G FD HW Yes Forward None F
> GMII 1518
> ge3( 7) up 1G FD HW Yes Forward None F
> GMII 1518
> ge4( 9) up 1G FD HW Yes Forward None F
> GMII 1518
>
> We note that now changing the auto-negotiate state in Junos correctly
> updates the underlying chipset and link negotiation works correctly.
> Interface is shown as GMII which we understand to be correct also.
>
> All-in, pretty terrible that - as far as we can tell - 1Gbps ports have
> *never worked* on QFX5110 properly in all the four basic combinations (i.e.
> SX and LX at both auto- and no-auto negotiation) until this release. Quite
> how Juniper have been shipping the hardware for so long in that state is a
> mystery to me.
>
>
> Tim
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
That is a fusion satellite device software version for an 18.x release.

--
Eldon


On Tue, Jul 2, 2019, 14:05 Colton Conor <colton.conor@gmail.com> wrote:

> Do you know when this will be fixed in the mainline releases? I have no
> clue what 3.5R1-S4.1 is?
>
> On Fri, Jun 28, 2019 at 2:22 PM Timothy Creswick <timothy@creswick.eu>
> wrote:
>
> > For anyone interested in this, JTAC released to us 3.5R1-S4.1 for the
> > satellite devices (I don't believe this will be generally available until
> > tomorrow). This addresses the issue and confirms that there were 2 or 3
> > different PRs relating to the Broadcom chipset being incorrectly
> programmed
> > by Junos.
> >
> > Some output, testing with a variety of 1Gbps optics (coded SX, LX,
> uncoded
> > BX, LX and SX).
> >
> > > show chassis hardware satellite
> > Hardware inventory:
> > Item Version Part number Serial number Description
> > FPC 100 REV 27 650-061152 WS37183----X QFX5110-48S-4C
> > PIC 0 REV 27 650-061152 WS37183----X 48x10G-4x100G
> > Xcvr 0 850nm 740-011782 F----------X SFP-SX
> > Xcvr 2 1310nm 740-011783 F----------X SFP-LX10
> > Xcvr 4 NON-JNPR VS51208----X
> > SFP-1000BASE-BX10-D
> > Xcvr 6 NON-JNPR VS60930----X SFP-LX10
> > Xcvr 8 NON-JNPR VS50930----X SFP-SX
> >
> > > show interfaces ge-100/0/[02468] terse
> > Interface Admin Link Proto Local Remote
> > ge-100/0/0 up up
> > ge-100/0/2 up up
> > ge-100/0/4 up up
> > ge-100/0/6 up up
> > ge-100/0/8 up up
> >
> > > show configuration interfaces
> > ge-100/0/0 {
> > unit 0;
> > }
> > ge-100/0/2 {
> > unit 0;
> > }
> > ge-100/0/4 {
> > unit 0;
> > }
> > ge-100/0/6 {
> > unit 0;
> > }
> > ge-100/0/8 {
> > unit 0;
> > }
> >
> > # bcmshell ps 1,3,5,7,9
> >
> > HW (unit 0)
> > ena/ speed/ link auto STP lrn
> > inter max loop
> > port link duplex scan neg? state pause discrd ops
> > face frame back
> > ge0( 1) up 1G FD HW Yes Forward None F
> > GMII 1518
> > ge1( 3) up 1G FD HW Yes Forward None F
> > GMII 1518
> > ge2( 5) up 1G FD HW Yes Forward None F
> > GMII 1518
> > ge3( 7) up 1G FD HW Yes Forward None F
> > GMII 1518
> > ge4( 9) up 1G FD HW Yes Forward None F
> > GMII 1518
> >
> > We note that now changing the auto-negotiate state in Junos correctly
> > updates the underlying chipset and link negotiation works correctly.
> > Interface is shown as GMII which we understand to be correct also.
> >
> > All-in, pretty terrible that - as far as we can tell - 1Gbps ports have
> > *never worked* on QFX5110 properly in all the four basic combinations
> (i.e.
> > SX and LX at both auto- and no-auto negotiation) until this release.
> Quite
> > how Juniper have been shipping the hardware for so long in that state is
> a
> > mystery to me.
> >
> >
> > Tim
> > _______________________________________________
> > juniper-nsp mailing list juniper-nsp@puck.nether.net
> > https://puck.nether.net/mailman/listinfo/juniper-nsp
> >
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: Link establishment issues with 1Gbps SX/LX SFPs on QFX5110 [ In reply to ]
Colton, EX Access switches used in Campus Fusion, do NOT run Junos. The run what we call SNOS (Satellite Network OS) instead. These SW releases, at least from a numbering standpoint, are completely different and independent of Junos releases. You can find Satellite SW via this direct link below. Something wrong with Support Web as I am not able to get there via normal login and choosing Downloads. Will look into this.

https://support.juniper.net/support/downloads/?f=fusion

HTH, Rich

Richard McGovern
Sr Sales Engineer, Juniper Networks
978-618-3342

I’d rather be lucky than good, as I know I am not good
I don’t make the news, I just report it


?On 7/3/19, 12:54 AM, "Eldon Koyle" <ekoyle+puck.nether.net@gmail.com> wrote:

That is a fusion satellite device software version for an 18.x release.

--
Eldon


On Tue, Jul 2, 2019, 14:05 Colton Conor <colton.conor@gmail.com> wrote:

> Do you know when this will be fixed in the mainline releases? I have no
> clue what 3.5R1-S4.1 is?
>
> On Fri, Jun 28, 2019 at 2:22 PM Timothy Creswick <timothy@creswick.eu>
> wrote:
>
> > For anyone interested in this, JTAC released to us 3.5R1-S4.1 for the
> > satellite devices (I don't believe this will be generally available until
> > tomorrow). This addresses the issue and confirms that there were 2 or 3
> > different PRs relating to the Broadcom chipset being incorrectly
> programmed
> > by Junos.
> >
> > Some output, testing with a variety of 1Gbps optics (coded SX, LX,
> uncoded
> > BX, LX and SX).
> >
> > > show chassis hardware satellite
> > Hardware inventory:
> > Item Version Part number Serial number Description
> > FPC 100 REV 27 650-061152 WS37183----X QFX5110-48S-4C
> > PIC 0 REV 27 650-061152 WS37183----X 48x10G-4x100G
> > Xcvr 0 850nm 740-011782 F----------X SFP-SX
> > Xcvr 2 1310nm 740-011783 F----------X SFP-LX10
> > Xcvr 4 NON-JNPR VS51208----X
> > SFP-1000BASE-BX10-D
> > Xcvr 6 NON-JNPR VS60930----X SFP-LX10
> > Xcvr 8 NON-JNPR VS50930----X SFP-SX
> >
> > > show interfaces ge-100/0/[02468] terse
> > Interface Admin Link Proto Local Remote
> > ge-100/0/0 up up
> > ge-100/0/2 up up
> > ge-100/0/4 up up
> > ge-100/0/6 up up
> > ge-100/0/8 up up
> >
> > > show configuration interfaces
> > ge-100/0/0 {
> > unit 0;
> > }
> > ge-100/0/2 {
> > unit 0;
> > }
> > ge-100/0/4 {
> > unit 0;
> > }
> > ge-100/0/6 {
> > unit 0;
> > }
> > ge-100/0/8 {
> > unit 0;
> > }
> >
> > # bcmshell ps 1,3,5,7,9
> >
> > HW (unit 0)
> > ena/ speed/ link auto STP lrn
> > inter max loop
> > port link duplex scan neg? state pause discrd ops
> > face frame back
> > ge0( 1) up 1G FD HW Yes Forward None F
> > GMII 1518
> > ge1( 3) up 1G FD HW Yes Forward None F
> > GMII 1518
> > ge2( 5) up 1G FD HW Yes Forward None F
> > GMII 1518
> > ge3( 7) up 1G FD HW Yes Forward None F
> > GMII 1518
> > ge4( 9) up 1G FD HW Yes Forward None F
> > GMII 1518
> >
> > We note that now changing the auto-negotiate state in Junos correctly
> > updates the underlying chipset and link negotiation works correctly.
> > Interface is shown as GMII which we understand to be correct also.
> >
> > All-in, pretty terrible that - as far as we can tell - 1Gbps ports have
> > *never worked* on QFX5110 properly in all the four basic combinations
> (i.e.
> > SX and LX at both auto- and no-auto negotiation) until this release.
> Quite
> > how Juniper have been shipping the hardware for so long in that state is
> a
> > mystery to me.
> >
> >
> > Tim
> > _______________________________________________
> > juniper-nsp mailing list juniper-nsp@puck.nether.net
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__puck.nether.net_mailman_listinfo_juniper-2Dnsp&d=DwIBaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=cViNvWbwxCvdnmDGDIbWYLiUsu8nisqLYXmd-x445bc&m=T6w2urFHFvXBxD29C0gFgfVkJh4B5x15Sv7j8BgLnFE&s=Ju91vnB092BAzzug7RbgIgb5ILmj9TrY-o3J9GQLoSM&e=
> >
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://urldefense.proofpoint.com/v2/url?u=https-3A__puck.nether.net_mailman_listinfo_juniper-2Dnsp&d=DwIBaQ&c=HAkYuh63rsuhr6Scbfh0UjBXeMK-ndb3voDTXcWzoCI&r=cViNvWbwxCvdnmDGDIbWYLiUsu8nisqLYXmd-x445bc&m=T6w2urFHFvXBxD29C0gFgfVkJh4B5x15Sv7j8BgLnFE&s=Ju91vnB092BAzzug7RbgIgb5ILmj9TrY-o3J9GQLoSM&e=
>



_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp