Mailing List Archive

We use them on MX304 at 10g, primarily to get DWDM SFP+ to work. MTU is
fine, it's 9k as a part of LACP on a recent deployment. The adapter simply
passes through lane :0 to the port when configured as QSFP+. If you insert
the adapter and no optic, the device is unaware of its existence - you only
see the SFP+ info.

We haven't tested in prod on 1g but i think we did in the lab. can probably
toss something in there if you're really curious. I think the juniper
supported optic page lists QSA adapter support or not. I thought it was
generally supported with Junos 20+ nowadays.

me@mx304> show chassis pic fpc-slot 0 pic-slot 1 | match 14
14 10GBASE ZR SM FLEXOPTIX P.1596.80 1550 nm
0.0 1550nm SFF-8472 ver 10.2


me@mx304> show interfaces terse | match xe-0/0/14
xe-0/0/14:0 up up
xe-0/0/14:0.0 up up aenet --> ae5.0


me@mx304> show interfaces ae5 | match mtu | match roto
Protocol inet, MTU: 9000
Protocol inet6, MTU: 9000
Protocol mpls, MTU: 9166, Maximum labels: 3
Protocol multiservice, MTU: Unlimited

Cheers
Chris


On Fri, Nov 3, 2023 at 8:01?AM Ola Thoresen via juniper-nsp <
juniper-nsp@puck.nether.net> wrote:

> Hi all,
>
> I am currently investigating the use of 1G and 10G optics on the MX304.
>
> This requires a QSA adapter -
>
> https://mellanox.my.site.com/mellanoxcommunity/s/article/all-about-qsa-adapter
> - That is fine.
>
> But then I hear people telling me that there is a known limitation when
> using these adapters that you can not have a higher MTU than ~2000 Bytes
> at _least_ when using 1G SFPs, but possibly also with 10G. However, I
> can not find _any_ information about that anywhere. Not from Juniper,
> not from Mellanox (Nvidia), nor from any user guides or datasheets I
> have found for the adapter from various vendors, and not from any mail
> archives, forums or anywhere else my Google-foo is taking me.
>
> Those who raised the issue are quite insistent that it is real, but they
> don't have any links or documents backing them up. Is it just FUD?
>
> * Has anyone ever used these adapters?
>
> * In a Juniper device (they are supported on different platforms)?
>
> * With 1G or 10G SFPs?
>
> * With (or without) adjusting the MTU?
>
>
> /Ola (T)
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On 03.11.2023 16:04, Chris Wopat wrote:

> We use them on MX304 at 10g, primarily to get DWDM SFP+ to work. MTU
> is fine, it's 9k as a part of LACP on a recent deployment. The adapter
> simply passes through lane :0 to the port when configured as QSFP+. If
> you insert the adapter and no optic, the device is unaware of its
> existence - you only see the SFP+ info.
>
This is the same use case as for us, and my understanding is exactly
that they are just passing a single lane through, and should not really
know anything about packets or ethernet frames or anything.

But still - people more knowledgeable than me - assure me that there is
a limit of 2008 Bytes MTU - at least for 1G.

I just can't find this documented anywhere, and I would have thought
that more people would have made more fuss about it when they start
using it if this is a real issue.


> We haven't tested in prod on 1g but i think we did in the lab. can
> probably toss something in there if you're really curious. I think the
> juniper supported optic page lists QSA adapter support or not. I
> thought it was generally supported with Junos 20+ nowadays.
>
Yes. They list QSA adapter as supported, also for 1G optics, and don't
write anything about any MTU limitations in the Hardware Compatibility Tool.


/Ola (T)

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Hi,

On Fri, Nov 03, 2023 at 04:15:58PM +0100, Ola Thoresen via juniper-nsp wrote:
> But still - people more knowledgeable than me - assure me that there is a
> limit of 2008 Bytes MTU - at least for 1G.

MTU is a very ill-understood topic... as is auto-negotiation...

If these people can't point to any formal documentation, I'd put it
to "folklore" - maybe they tested with an 1G copper device on the other
end that claimed 9000 but failed to actually do more than 2008...

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

Hi,
Here is the public documentation of MX304 mentioning the MTU limitation for 1G:
https://www.juniper.net/documentation/us/en/software/junos/interfaces-ethernet/topics/ref/statement/speed-gigether-options.html


* On MX304 for 1G interface, the maximum MTU supported is 2000 bytes. Hence, the maximum MRU supported is 2008 bytes.

Br.,
--
Eduardo Haro



Juniper Business Use Only

From: juniper-nsp <juniper-nsp-bounces@puck.nether.net> on behalf of Ola Thoresen via juniper-nsp <juniper-nsp@puck.nether.net>
Date: Friday, 3 November 2023 12:16
To: juniper-nsp@puck.nether.net <juniper-nsp@puck.nether.net>
Subject: Re: [j-nsp] QSA adapters and MTU
[External Email. Be cautious of content]


On 03.11.2023 16:04, Chris Wopat wrote:

> We use them on MX304 at 10g, primarily to get DWDM SFP+ to work. MTU
> is fine, it's 9k as a part of LACP on a recent deployment. The adapter
> simply passes through lane :0 to the port when configured as QSFP+. If
> you insert the adapter and no optic, the device is unaware of its
> existence - you only see the SFP+ info.
>
This is the same use case as for us, and my understanding is exactly
that they are just passing a single lane through, and should not really
know anything about packets or ethernet frames or anything.

But still - people more knowledgeable than me - assure me that there is
a limit of 2008 Bytes MTU - at least for 1G.

I just can't find this documented anywhere, and I would have thought
that more people would have made more fuss about it when they start
using it if this is a real issue.


> We haven't tested in prod on 1g but i think we did in the lab. can
> probably toss something in there if you're really curious. I think the
> juniper supported optic page lists QSA adapter support or not. I
> thought it was generally supported with Junos 20+ nowadays.
>
Yes. They list QSA adapter as supported, also for 1G optics, and don't
write anything about any MTU limitations in the Hardware Compatibility Tool.


/Ola (T)

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://urldefense.com/v3/__https://puck.nether.net/mailman/listinfo/juniper-nsp__;!!NEt6yMaO-gk!DaqgnffXeN_5g5MT4MX8vxG5xhtjRSbu8HSfaHQe_k3qfrESOCfV3YpaMzwMEdc7T8gs6JHvNYaXKmVMAI16rgk$<https://urldefense.com/v3/__https:/puck.nether.net/mailman/listinfo/juniper-nsp__;!!NEt6yMaO-gk!DaqgnffXeN_5g5MT4MX8vxG5xhtjRSbu8HSfaHQe_k3qfrESOCfV3YpaMzwMEdc7T8gs6JHvNYaXKmVMAI16rgk$>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

I can test this on our lab box, but cant get 'supported platform' in the
chassis config for

port 15 {
##
## Warning: statement ignored: unsupported platform (mx304)
##
speed 1G;
}

even though it's passing on https://apps.juniper.net/port-checker/mx304/
¯\_(?)_/¯

could perhaps work harder on getting that config right later. it
certainly lets me commit a ge config with that MTU though. when set as
speed 10g, I can see the 1g optic inserted, as well as DOM, but it won't
link.

--Chris



On Fri, Nov 3, 2023 at 10:16?AM Ola Thoresen via juniper-nsp <
juniper-nsp@puck.nether.net> wrote:

> On 03.11.2023 16:04, Chris Wopat wrote:
>
> > We use them on MX304 at 10g, primarily to get DWDM SFP+ to work. MTU
> > is fine, it's 9k as a part of LACP on a recent deployment. The adapter
> > simply passes through lane :0 to the port when configured as QSFP+. If
> > you insert the adapter and no optic, the device is unaware of its
> > existence - you only see the SFP+ info.
> >
> This is the same use case as for us, and my understanding is exactly
> that they are just passing a single lane through, and should not really
> know anything about packets or ethernet frames or anything.
>
> But still - people more knowledgeable than me - assure me that there is
> a limit of 2008 Bytes MTU - at least for 1G.
>
> I just can't find this documented anywhere, and I would have thought
> that more people would have made more fuss about it when they start
> using it if this is a real issue.
>
>
> > We haven't tested in prod on 1g but i think we did in the lab. can
> > probably toss something in there if you're really curious. I think the
> > juniper supported optic page lists QSA adapter support or not. I
> > thought it was generally supported with Junos 20+ nowadays.
> >
> Yes. They list QSA adapter as supported, also for 1G optics, and don't
> write anything about any MTU limitations in the Hardware Compatibility
> Tool.
>
>
> /Ola (T)
>
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Actually 1G ports are «10G ports operating at 1G speed».
So, configured as 10G ports on chassis side, giga-ether speed 1G on
interface side.

Le ven. 3 nov. 2023 à 16:53, Chris Wopat via juniper-nsp <
juniper-nsp@puck.nether.net> a écrit :

> I can test this on our lab box, but cant get 'supported platform' in the
> chassis config for
>
> port 15 {
> ##
> ## Warning: statement ignored: unsupported platform (mx304)
> ##
> speed 1G;
> }
>
> even though it's passing on https://apps.juniper.net/port-checker/mx304/
> ¯\_(?)_/¯
>
> could perhaps work harder on getting that config right later. it
> certainly lets me commit a ge config with that MTU though. when set as
> speed 10g, I can see the 1g optic inserted, as well as DOM, but it won't
> link.
>
> --Chris
>
>
>
> On Fri, Nov 3, 2023 at 10:16?AM Ola Thoresen via juniper-nsp <
> juniper-nsp@puck.nether.net> wrote:
>
> > On 03.11.2023 16:04, Chris Wopat wrote:
> >
> > > We use them on MX304 at 10g, primarily to get DWDM SFP+ to work. MTU
> > > is fine, it's 9k as a part of LACP on a recent deployment. The adapter
> > > simply passes through lane :0 to the port when configured as QSFP+. If
> > > you insert the adapter and no optic, the device is unaware of its
> > > existence - you only see the SFP+ info.
> > >
> > This is the same use case as for us, and my understanding is exactly
> > that they are just passing a single lane through, and should not really
> > know anything about packets or ethernet frames or anything.
> >
> > But still - people more knowledgeable than me - assure me that there is
> > a limit of 2008 Bytes MTU - at least for 1G.
> >
> > I just can't find this documented anywhere, and I would have thought
> > that more people would have made more fuss about it when they start
> > using it if this is a real issue.
> >
> >
> > > We haven't tested in prod on 1g but i think we did in the lab. can
> > > probably toss something in there if you're really curious. I think the
> > > juniper supported optic page lists QSA adapter support or not. I
> > > thought it was generally supported with Junos 20+ nowadays.
> > >
> > Yes. They list QSA adapter as supported, also for 1G optics, and don't
> > write anything about any MTU limitations in the Hardware Compatibility
> > Tool.
> >
>

--
*Ce message et toutes les pièces jointes (ci-après le "message") sont
établis à l’intention exclusive des destinataires désignés. Il contient des
informations confidentielles et pouvant être protégé par le secret
professionnel. Si vous recevez ce message par erreur, merci d'en avertir
immédiatement l'expéditeur et de détruire le message. Toute utilisation de
ce message non conforme à sa destination, toute diffusion ou toute
publication, totale ou partielle, est interdite, sauf autorisation expresse
de l'émetteur*
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

I recall the MX204 being like that… an XE interface with a 1g speed command on the interface

Aaron

> On Nov 3, 2023, at 11:00 AM, Olivier Benghozi via juniper-nsp <juniper-nsp@puck.nether.net> wrote:
>
> ?Actually 1G ports are «10G ports operating at 1G speed».
> So, configured as 10G ports on chassis side, giga-ether speed 1G on
> interface side.
>
>> Le ven. 3 nov. 2023 à 16:53, Chris Wopat via juniper-nsp <
>> juniper-nsp@puck.nether.net> a écrit :
>>
>> I can test this on our lab box, but cant get 'supported platform' in the
>> chassis config for
>>
>> port 15 {
>> ##
>> ## Warning: statement ignored: unsupported platform (mx304)
>> ##
>> speed 1G;
>> }
>>
>> even though it's passing on https://apps.juniper.net/port-checker/mx304/
>> ¯\_(?)_/¯
>>
>> could perhaps work harder on getting that config right later. it
>> certainly lets me commit a ge config with that MTU though. when set as
>> speed 10g, I can see the 1g optic inserted, as well as DOM, but it won't
>> link.
>>
>> --Chris
>>
>>
>>
>> On Fri, Nov 3, 2023 at 10:16?AM Ola Thoresen via juniper-nsp <
>> juniper-nsp@puck.nether.net> wrote:
>>
>>>> On 03.11.2023 16:04, Chris Wopat wrote:
>>>
>>>> We use them on MX304 at 10g, primarily to get DWDM SFP+ to work. MTU
>>>> is fine, it's 9k as a part of LACP on a recent deployment. The adapter
>>>> simply passes through lane :0 to the port when configured as QSFP+. If
>>>> you insert the adapter and no optic, the device is unaware of its
>>>> existence - you only see the SFP+ info.
>>>>
>>> This is the same use case as for us, and my understanding is exactly
>>> that they are just passing a single lane through, and should not really
>>> know anything about packets or ethernet frames or anything.
>>>
>>> But still - people more knowledgeable than me - assure me that there is
>>> a limit of 2008 Bytes MTU - at least for 1G.
>>>
>>> I just can't find this documented anywhere, and I would have thought
>>> that more people would have made more fuss about it when they start
>>> using it if this is a real issue.
>>>
>>>
>>>> We haven't tested in prod on 1g but i think we did in the lab. can
>>>> probably toss something in there if you're really curious. I think the
>>>> juniper supported optic page lists QSA adapter support or not. I
>>>> thought it was generally supported with Junos 20+ nowadays.
>>>>
>>> Yes. They list QSA adapter as supported, also for 1G optics, and don't
>>> write anything about any MTU limitations in the Hardware Compatibility
>>> Tool.
>>>
>>
>
> --
> *Ce message et toutes les pièces jointes (ci-après le "message") sont
> établis à l’intention exclusive des destinataires désignés. Il contient des
> informations confidentielles et pouvant être protégé par le secret
> professionnel. Si vous recevez ce message par erreur, merci d'en avertir
> immédiatement l'expéditeur et de détruire le message. Toute utilisation de
> ce message non conforme à sa destination, toute diffusion ou toute
> publication, totale ou partielle, est interdite, sauf autorisation expresse
> de l'émetteur*
> _______________________________________________
> juniper-nsp mailing list juniper-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/juniper-nsp

_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Once upon a time, Aaron1 via juniper-nsp <juniper-nsp@puck.nether.net> said:
> I recall the MX204 being like that… an XE interface with a 1g speed command on the interface

There it's done under "interface xe-x/y/z gigether-options". You set
"chassis fpc x pic y port z speed 10g" first, then speed 1g in the
interface.

--
Chris Adams <cma@cmadams.net>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

On 03.11.2023 16:37, Eduardo Lopes de Haro wrote:

> Hi,
>
> Here is the public documentation of MX304 mentioning the MTU
> limitation for 1G:
>
> https://www.juniper.net/documentation/us/en/software/junos/interfaces-ethernet/topics/ref/statement/speed-gigether-options.html
>
> * On MX304 for 1G interface, the maximum MTU supported is 2000
> bytes. Hence, the maximum MRU supported is 2008 bytes.
>

Thank you for the link.  That answered my question, and was - in good
Juniper tradition - well hidden.

No explanation of why such a limitation exists, though.  But at least I
have something official to reference.


/Ola (T)
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp

Couple of things regarding this thread.

#1 ? MX304 MTU: The MTU restriction ONLY applies when the QSA adapter is used with 1G Optic in an MX304. For 10G there is no MTU limitation. The reason being:

The 2K MTU limit is very specific to MX304. The MX304 uses the YT trio PFE, which does not have native support for a 1GE MAC. Workaround was to implement the 1GE in the PHY framer on the board, and this has a very shallow buffer ? hence the MTU limit.

Advice is to take a simple EX as satellite and use that to aggregate 1G. No MTU restriction then.

#2 ? MX204 1G speed. I thought MX204 interfaces were SFP+, which in Juniper terms equals 1/10 support. I thought the interface should auto-sense the installed optic to determine the speed. At the same time, for MX and SRX (but not EX/QFX) 1/10 SFP+ capable interfaces always use an interface naming designation as xe-. There is no support for ge- interface naming. This was an internal Juniper decision for ease of implementation, as original MX (and therefore SRX) interfaces were NOT SFP+ (1/10 support) originally. Instead they were SFP, which is 1G or 10G support, but not both. Both EX and QFX (and others) interfaces were SFP+ from day 1, due to time period they were built and the improved technology. To get 1G to work properly in a SFP+ interface, you may need to disable auto-neg on that interface. It really depends upon model/SW and I do not know all the combinations and details. If not working as expected, try this.

Hopefully this helps some and can clear up some things. I am just the messenger, . . .

FYI only, Rich

Richard McGovern
Sr Sales Engineer, Juniper Networks
978-618-3342

I?d rather be lucky than good, as I know I am not good
I don?t make the news, I just report it




Juniper Business Use Only

On 11/3/23, 2:53 PM, "Chris Adams" <cma@cmadams.net> wrote:
Once upon a time, Aaron1 via juniper-nsp <juniper-nsp@puck.nether.net<mailto:juniper-nsp@puck.nether.net>> said:
> I recall the MX204 being like that? an XE interface with a 1g speed command on the interface

There it's done under "interface xe-x/y/z gigether-options". You set
"chassis fpc x pic y port z speed 10g" first, then speed 1g in the
interface.

--
Chris Adams <cma@cmadams.net<mailto:cma@cmadams.net>>


_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp