Mailing List Archive

Re: [EXT] EX4300: Framing error with macsec enabled
Yes, I see CRC errors on EX3400s with MACsec termination, but only on one side.

Here is my topology:

From A to B:

[EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B]
MACsec L2 connection L2 xconnect MACsec

From B to A:

[EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B]
MACsec L2 connection L2 xconnect MACsec

I also have a redundant path with EX3400-C (different local switch) and EX3400-B (same remote switch).

I see the CRC errors increasing at a rate of about 2-3 per minute, but only on EX3400-A and EXX3400-C.

All EX3400s were initially running 15.1X53-D57. Now A and C are running 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been consistent throughout all releases, no improvement with upgrades.

I wonder if something the carrier's ASR9k is sending down the VLAN towards EX3400-A and -C is causing this? If not, maybe it is the MX480s sending something locally to EX3400-A and -C?

The following PRs don't seem relevant--I'm not doing anywhere close to 60% utilization:

https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1261567

And I'm not seeing "runts":

https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1469663

I'm only seeing Framing errors (CRC/Align errors):

admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791
Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
CRC/Align errors 227911 0

A few seconds later, it increased to 227913:

MAC statistics: Receive Transmit
Total octets 17953647117156 3221741316352
Total packets 13200126465 7010832956
Unicast packets 13194022205 7004785539
Broadcast packets 5272 0
Multicast packets 6098988 6047417
CRC/Align errors 227913 0
FIFO errors 0 0
MAC control frames 0 0
MAC pause frames 0 0
Oversized frames 0
Jabber frames 0
Fragment frames 0
VLAN tagged frames 13196813130
Code violations 0

Rate is only 24 Mbps, 2200 pps:

Manager@sw-gp1-macsec-1> show interfaces extensive xe-0/2/0 |match "bps|pps"
Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None,
Input bytes : 17954037433802 24242136 bps
Output bytes : 3221749625049 498504 bps
Input packets: 13200416854 2190 pps
Output packets: 7010941872 830 pps



On Sun, Apr 19, 2020 at 09:37:23AM +0200, james list wrote:
> Dear experts,
> I've an EX4300 (Junos 17.3R3-S3.3) which have a constant Framing error
> counter increase also if the traffic is very low.
> Interface is connected to a WAN link from a carrier and bw is 1 Gbs but
> traffic max is actually 100 Mbs and on average 10 Mbs.
> On this interface I've enabled macsec, if I disable macsec the issue is not
> in place but unfortunately macsec is mandatory to be kept enabled.
>
> I cannot sniff since the packet is encrypted but to me it seems that
> traffic is not lost, if I have 100 Mbs inside from WAN I see 100 Mbs
> outside to DataCenter.
>
> Due to the fact that monitoring system contantly raise an alert, I'd like
> to know how to fix it or at least let the EX4300 do not raise the counter
> increase.
>
> I've opened a JTAC case but they found a PR which is currently related to a
> Broadcom chipset raising framing errors during spikes (ie 70% of the
> interface bandwidth).
>
> https://kb.juniper.net/InfoCenter/index?page=content&id=KB32264&actp=METADATA
>
> Also enabling flow-control as described in the KB do not change the
> behaviour.
>
> I'm wondering if there could the option we're receiving some sort on
> "unknown protocol" from the carrier (I remeber Cisco has something like
> that) or could be an harware issue..
>
> On the other side of the link, the other EX4300 (side B) do not experience
> the same issue but the traffic is mostly from side B to side A.
>
> Here an example of the output, statistics cleared and after 1 minute I get
> 12 framing errors with 2 Mbs running on the WAN link:
>
> @EX4300-A> show interfaces ae0 extensive
> Physical interface: ae0, Enabled, Physical link is Up
> Interface index: 220, SNMP ifIndex: 549, Generation: 131
> Description: xxx
> Link-level type: Ethernet, MTU: 9192, Speed: 1Gbps, BPDU Error: None,
> Ethernet-Switching Error: None, MAC-REWRITE Error: None,
> Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled,
> Minimum links needed: 1, Minimum bandwidth needed: 1bps
> Device flags : Present Running
> Interface flags: SNMP-Traps Internal: 0x0
> Current address: cc:e5:94:11:43:23, Hardware address: cc:e5:94:11:43:23
> Last flapped : 2020-04-19 02:05:05 CEST (06:50:45 ago)
> Statistics last cleared: 2020-04-19 09:11:22 CEST (00:01:00 ago)
> Traffic statistics:
> Input bytes : 10014863 2205456 bps
> Output bytes : 4095720 582456 bps
> Input packets: 33292 624 pps
> Output packets: 33023 568 pps
> IPv6 transit statistics:
> Input bytes : 0
> Output bytes : 0
> Input packets: 0
> Output packets: 0
> Input errors:
> Errors: 12, Drops: 0, Framing errors: 12, Runts: 0, Giants: 0, Policed
> discards: 0, Resource errors: 0
> Output errors:
> Carrier transitions: 0, Errors: 0, Drops: 0, MTU errors: 0, Resource
> errors: 0
> Egress queues: 12 supported, 11 in use
>
>
> @EX4300-A> show interfaces ge-0/0/0 extensive
> Physical interface: ge-0/0/0, Enabled, Physical link is Up
> Interface index: 649, SNMP ifIndex: 509, Generation: 140
> Description: WAN link
> Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Link-mode:
> Full-duplex, Speed: 1000mbps, BPDU Error: None, Loop Detect PDU Error: None,
> Ethernet-Switching Error: None, Source filtering: Disabled
> Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback:
> Disabled, Flow control: Enabled, Auto-negotiation: Enabled,
> Remote fault: Online, Media type: Copper, IEEE 802.3az Energy Efficient
> Ethernet: Disabled, Auto-MDIX: Enabled
> Device flags : Present Running
> Interface flags: SNMP-Traps Internal: 0x0
> Link flags : None
> CoS queues : 12 supported, 12 maximum usable queues
> Hold-times : Up 0 ms, Down 0 ms
> Current address: cc:e5:94:11:43:23, Hardware address: cc:e5:94:11:43:23
> Last flapped : 2020-03-28 18:43:04 CET (3w0d 13:30 ago)
> Statistics last cleared: 2020-04-19 09:11:18 CEST (00:02:18 ago)
> Traffic statistics:
> Input bytes : 21782579 932296 bps
> Output bytes : 17898068 498704 bps
> Input packets: 76844 569 pps
> Output packets: 82594 590 pps
> IPv6 transit statistics:
> Input bytes : 0
> Output bytes : 0
> Input packets: 0
> Output packets: 0
> Input errors:
> Errors: 28, Drops: 0, Framing errors: 28, Runts: 0, Policed discards:
> 0, L3 incompletes: 0, L2 channel errors: 0,
> L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
>
>
> Here part of the config:
>
> @EX4300-A> show configuration interfaces ge-0/0/0 | display set
> set interfaces ge-0/0/0 ether-options auto-negotiation
> set interfaces ge-0/0/0 ether-options flow-control
> set interfaces ge-0/0/0 ether-options 802.3ad ae0
>
>
> @EX4300-A> show configuration interfaces ae0 | display set
> set interfaces ae0 mtu 9192
> set interfaces ae0 aggregated-ether-options flow-control
> set interfaces ae0 aggregated-ether-options lacp active
> set interfaces ae0 aggregated-ether-options lacp periodic fast
> set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
> set interfaces ae0 unit 0 family ethernet-switching vlan members 2228
> set interfaces ae0 unit 0 family ethernet-switching vlan members 2552-2553
> set interfaces ae0 unit 0 family ethernet-switching filter input QOS
>
>
> @EX4300-A> show configuration security macsec | display set
> set security macsec connectivity-association MAC security-mode static-cak
> set security macsec connectivity-association MAC pre-shared-key ckn xxxx
> set security macsec connectivity-association MAC pre-shared-key cak
> "tttttvvvv"
> set security macsec connectivity-association MAC exclude-protocol lldp
> set security macsec connectivity-association MAC exclude-protocol lacp
> set security macsec interfaces ge-0/0/0 connectivity-association MAC
>
>
> Dear all, an help is appreciated and welcomme, please let me thank in
> advance anyone will give an hint.
>
> Cheers
> James
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [EXT] EX4300: Framing error with macsec enabled [ In reply to ]
Well, that was an easy fix on my MX480s:

set protocols lldp interface xe-0/0/1 disable

Now I'm not seeing CRC errors incrementing 2-3 times per minute on the EX3400s connected directly to the MX480s.

I'm not excluding any protocols from MACsec--LLDP runs end-to-end between the EX3400s just fine.

Check if the carrier is running LLDP or CDP or similar.


On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote:
> Yes, I see CRC errors on EX3400s with MACsec termination, but only on one side.
>
> Here is my topology:
>
> From A to B:
>
> [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B]
> MACsec L2 connection L2 xconnect MACsec
>
> From B to A:
>
> [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B]
> MACsec L2 connection L2 xconnect MACsec
>
> I also have a redundant path with EX3400-C (different local switch) and EX3400-B (same remote switch).
>
> I see the CRC errors increasing at a rate of about 2-3 per minute, but only on EX3400-A and EXX3400-C.
>
> All EX3400s were initially running 15.1X53-D57. Now A and C are running 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been consistent throughout all releases, no improvement with upgrades.
>
> I wonder if something the carrier's ASR9k is sending down the VLAN towards EX3400-A and -C is causing this? If not, maybe it is the MX480s sending something locally to EX3400-A and -C?
>
> The following PRs don't seem relevant--I'm not doing anywhere close to 60% utilization:
>
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1261567
>
> And I'm not seeing "runts":
>
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1469663
>
> I'm only seeing Framing errors (CRC/Align errors):
>
> admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791
> Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
> CRC/Align errors 227911 0
>
> A few seconds later, it increased to 227913:
>
> MAC statistics: Receive Transmit
> Total octets 17953647117156 3221741316352
> Total packets 13200126465 7010832956
> Unicast packets 13194022205 7004785539
> Broadcast packets 5272 0
> Multicast packets 6098988 6047417
> CRC/Align errors 227913 0
> FIFO errors 0 0
> MAC control frames 0 0
> MAC pause frames 0 0
> Oversized frames 0
> Jabber frames 0
> Fragment frames 0
> VLAN tagged frames 13196813130
> Code violations 0
>
> Rate is only 24 Mbps, 2200 pps:
>
> admin@ex3400-a> show interfaces extensive xe-0/2/0 |match "bps|pps"
> Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None,
> Input bytes : 17954037433802 24242136 bps
> Output bytes : 3221749625049 498504 bps
> Input packets: 13200416854 2190 pps
> Output packets: 7010941872 830 pps
>
>
>
> On Sun, Apr 19, 2020 at 09:37:23AM +0200, james list wrote:
> > Dear experts,
> > I've an EX4300 (Junos 17.3R3-S3.3) which have a constant Framing error
> > counter increase also if the traffic is very low.
> > Interface is connected to a WAN link from a carrier and bw is 1 Gbs but
> > traffic max is actually 100 Mbs and on average 10 Mbs.
> > On this interface I've enabled macsec, if I disable macsec the issue is not
> > in place but unfortunately macsec is mandatory to be kept enabled.
> >
> > I cannot sniff since the packet is encrypted but to me it seems that
> > traffic is not lost, if I have 100 Mbs inside from WAN I see 100 Mbs
> > outside to DataCenter.
> >
> > Due to the fact that monitoring system contantly raise an alert, I'd like
> > to know how to fix it or at least let the EX4300 do not raise the counter
> > increase.
> >
> > I've opened a JTAC case but they found a PR which is currently related to a
> > Broadcom chipset raising framing errors during spikes (ie 70% of the
> > interface bandwidth).
> >
> > https://kb.juniper.net/InfoCenter/index?page=content&id=KB32264&actp=METADATA
> >
> > Also enabling flow-control as described in the KB do not change the
> > behaviour.
> >
> > I'm wondering if there could the option we're receiving some sort on
> > "unknown protocol" from the carrier (I remeber Cisco has something like
> > that) or could be an harware issue..
> >
> > On the other side of the link, the other EX4300 (side B) do not experience
> > the same issue but the traffic is mostly from side B to side A.
> >
> > Here an example of the output, statistics cleared and after 1 minute I get
> > 12 framing errors with 2 Mbs running on the WAN link:
> >
> > @EX4300-A> show interfaces ae0 extensive
> > Physical interface: ae0, Enabled, Physical link is Up
> > Interface index: 220, SNMP ifIndex: 549, Generation: 131
> > Description: xxx
> > Link-level type: Ethernet, MTU: 9192, Speed: 1Gbps, BPDU Error: None,
> > Ethernet-Switching Error: None, MAC-REWRITE Error: None,
> > Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled,
> > Minimum links needed: 1, Minimum bandwidth needed: 1bps
> > Device flags : Present Running
> > Interface flags: SNMP-Traps Internal: 0x0
> > Current address: cc:e5:94:11:43:23, Hardware address: cc:e5:94:11:43:23
> > Last flapped : 2020-04-19 02:05:05 CEST (06:50:45 ago)
> > Statistics last cleared: 2020-04-19 09:11:22 CEST (00:01:00 ago)
> > Traffic statistics:
> > Input bytes : 10014863 2205456 bps
> > Output bytes : 4095720 582456 bps
> > Input packets: 33292 624 pps
> > Output packets: 33023 568 pps
> > IPv6 transit statistics:
> > Input bytes : 0
> > Output bytes : 0
> > Input packets: 0
> > Output packets: 0
> > Input errors:
> > Errors: 12, Drops: 0, Framing errors: 12, Runts: 0, Giants: 0, Policed
> > discards: 0, Resource errors: 0
> > Output errors:
> > Carrier transitions: 0, Errors: 0, Drops: 0, MTU errors: 0, Resource
> > errors: 0
> > Egress queues: 12 supported, 11 in use
> >
> >
> > @EX4300-A> show interfaces ge-0/0/0 extensive
> > Physical interface: ge-0/0/0, Enabled, Physical link is Up
> > Interface index: 649, SNMP ifIndex: 509, Generation: 140
> > Description: WAN link
> > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Link-mode:
> > Full-duplex, Speed: 1000mbps, BPDU Error: None, Loop Detect PDU Error: None,
> > Ethernet-Switching Error: None, Source filtering: Disabled
> > Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback:
> > Disabled, Flow control: Enabled, Auto-negotiation: Enabled,
> > Remote fault: Online, Media type: Copper, IEEE 802.3az Energy Efficient
> > Ethernet: Disabled, Auto-MDIX: Enabled
> > Device flags : Present Running
> > Interface flags: SNMP-Traps Internal: 0x0
> > Link flags : None
> > CoS queues : 12 supported, 12 maximum usable queues
> > Hold-times : Up 0 ms, Down 0 ms
> > Current address: cc:e5:94:11:43:23, Hardware address: cc:e5:94:11:43:23
> > Last flapped : 2020-03-28 18:43:04 CET (3w0d 13:30 ago)
> > Statistics last cleared: 2020-04-19 09:11:18 CEST (00:02:18 ago)
> > Traffic statistics:
> > Input bytes : 21782579 932296 bps
> > Output bytes : 17898068 498704 bps
> > Input packets: 76844 569 pps
> > Output packets: 82594 590 pps
> > IPv6 transit statistics:
> > Input bytes : 0
> > Output bytes : 0
> > Input packets: 0
> > Output packets: 0
> > Input errors:
> > Errors: 28, Drops: 0, Framing errors: 28, Runts: 0, Policed discards:
> > 0, L3 incompletes: 0, L2 channel errors: 0,
> > L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
> >
> >
> > Here part of the config:
> >
> > @EX4300-A> show configuration interfaces ge-0/0/0 | display set
> > set interfaces ge-0/0/0 ether-options auto-negotiation
> > set interfaces ge-0/0/0 ether-options flow-control
> > set interfaces ge-0/0/0 ether-options 802.3ad ae0
> >
> >
> > @EX4300-A> show configuration interfaces ae0 | display set
> > set interfaces ae0 mtu 9192
> > set interfaces ae0 aggregated-ether-options flow-control
> > set interfaces ae0 aggregated-ether-options lacp active
> > set interfaces ae0 aggregated-ether-options lacp periodic fast
> > set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
> > set interfaces ae0 unit 0 family ethernet-switching vlan members 2228
> > set interfaces ae0 unit 0 family ethernet-switching vlan members 2552-2553
> > set interfaces ae0 unit 0 family ethernet-switching filter input QOS
> >
> >
> > @EX4300-A> show configuration security macsec | display set
> > set security macsec connectivity-association MAC security-mode static-cak
> > set security macsec connectivity-association MAC pre-shared-key ckn xxxx
> > set security macsec connectivity-association MAC pre-shared-key cak
> > "tttttvvvv"
> > set security macsec connectivity-association MAC exclude-protocol lldp
> > set security macsec connectivity-association MAC exclude-protocol lacp
> > set security macsec interfaces ge-0/0/0 connectivity-association MAC
> >
> >
> > Dear all, an help is appreciated and welcomme, please let me thank in
> > advance anyone will give an hint.
> >
> > Cheers
> > James
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [EXT] EX4300: Framing error with macsec enabled [ In reply to ]
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [EXT] EX4300: Framing error with macsec enabled [ In reply to ]
Hi Richard
lldp and lacp are excluded:

> > @EX4300-A> show configuration security macsec | display set
> > set security macsec connectivity-association MAC security-mode
static-cak
> > set security macsec connectivity-association MAC pre-shared-key ckn
xxxx
> > set security macsec connectivity-association MAC pre-shared-key cak
> > "tttttvvvv"
> > set security macsec connectivity-association MAC exclude-protocol
lldp
> > set security macsec connectivity-association MAC exclude-protocol
lacp
> > set security macsec interfaces ge-0/0/0 connectivity-association
MAC

I did not catch the connection with Framing errors counter...

Please detail if you can.

Cheers


Il giorno mar 21 apr 2020 alle ore 15:27 Richard McGovern <
rmcgovern@juniper.net> ha scritto:

> Chuck, I thought you were running both LLDP and LACP outside the MACSEC
> tunnel, no?
>
> (Optional) Exclude a protocol from MACsec:
> [edit security macsec connectivity-association
> connectivity-association-name]
> user@switch# set exclude-protocol protocol-name
> For instance, if you did not want Link Level Discovery Protocol (LLDP) to
> be secured using MACsec:
>
> [edit security macsec connectivity-association ca-dynamic1]
> user@switch# set exclude-protocol lldp
> When this option is enabled, MACsec is disabled for all packets of the
> specified protocol—in this case, LLDP—that are sent or received on the link.
>
> BEST PRACTICEWe recommend that any protocol other than MACsec being used
> on the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing
> protocols, should be excluded and moved outside of the MACsec tunnel.
>
> Is this not working properly for LLDP?
>
> Rich
>
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
>
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
>
>
> ?On 4/19/20, 7:31 PM, "Chuck Anderson" <cra@WPI.EDU> wrote:
>
> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on the
> EX3400s connected directly to the MX480s.
>
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end
> between the EX3400s just fine.
>
> Check if the carrier is running LLDP or CDP or similar.
>
>
> On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote:
> > Yes, I see CRC errors on EX3400s with MACsec termination, but only
> on one side.
> >
> > Here is my topology:
> >
> > From A to B:
> >
> > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2
> vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B]
> > MACsec L2 connection L2
> xconnect MACsec
> >
> > From B to A:
> >
> > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2
> vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B]
> > MACsec L2 connection L2
> xconnect MACsec
> >
> > I also have a redundant path with EX3400-C (different local switch)
> and EX3400-B (same remote switch).
> >
> > I see the CRC errors increasing at a rate of about 2-3 per minute,
> but only on EX3400-A and EXX3400-C.
> >
> > All EX3400s were initially running 15.1X53-D57. Now A and C are
> running 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been
> consistent throughout all releases, no improvement with upgrades.
> >
> > I wonder if something the carrier's ASR9k is sending down the VLAN
> towards EX3400-A and -C is causing this? If not, maybe it is the MX480s
> sending something locally to EX3400-A and -C?
> >
> > The following PRs don't seem relevant--I'm not doing anywhere close
> to 60% utilization:
> >
> >
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1261567
> >
> > And I'm not seeing "runts":
> >
> >
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1469663
> >
> > I'm only seeing Framing errors (CRC/Align errors):
> >
> > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791
> > Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0,
> Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch
> timeouts: 0, FIFO errors: 0, Resource errors: 0
> > CRC/Align errors 227911 0
> >
> > A few seconds later, it increased to 227913:
> >
> > MAC statistics: Receive Transmit
> > Total octets 17953647117156 3221741316352
> > Total packets 13200126465 7010832956
> > Unicast packets 13194022205 7004785539
> > Broadcast packets 5272 0
> > Multicast packets 6098988 6047417
> > CRC/Align errors 227913 0
> > FIFO errors 0 0
> > MAC control frames 0 0
> > MAC pause frames 0 0
> > Oversized frames 0
> > Jabber frames 0
> > Fragment frames 0
> > VLAN tagged frames 13196813130
> > Code violations 0
> >
> > Rate is only 24 Mbps, 2200 pps:
> >
> > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match "bps|pps"
> > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps,
> BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error:
> None, MAC-REWRITE Error: None,
> > Input bytes : 17954037433802 24242136 bps
> > Output bytes : 3221749625049 498504 bps
> > Input packets: 13200416854 2190 pps
> > Output packets: 7010941872 830 pps
> >
> >
> >
> > On Sun, Apr 19, 2020 at 09:37:23AM +0200, james list wrote:
> > > Dear experts,
> > > I've an EX4300 (Junos 17.3R3-S3.3) which have a constant Framing
> error
> > > counter increase also if the traffic is very low.
> > > Interface is connected to a WAN link from a carrier and bw is 1
> Gbs but
> > > traffic max is actually 100 Mbs and on average 10 Mbs.
> > > On this interface I've enabled macsec, if I disable macsec the
> issue is not
> > > in place but unfortunately macsec is mandatory to be kept enabled.
> > >
> > > I cannot sniff since the packet is encrypted but to me it seems
> that
> > > traffic is not lost, if I have 100 Mbs inside from WAN I see 100
> Mbs
> > > outside to DataCenter.
> > >
> > > Due to the fact that monitoring system contantly raise an alert,
> I'd like
> > > to know how to fix it or at least let the EX4300 do not raise the
> counter
> > > increase.
> > >
> > > I've opened a JTAC case but they found a PR which is currently
> related to a
> > > Broadcom chipset raising framing errors during spikes (ie 70% of
> the
> > > interface bandwidth).
> > >
> > >
> https://kb.juniper.net/InfoCenter/index?page=content&id=KB32264&actp=METADATA
> > >
> > > Also enabling flow-control as described in the KB do not change the
> > > behaviour.
> > >
> > > I'm wondering if there could the option we're receiving some sort
> on
> > > "unknown protocol" from the carrier (I remeber Cisco has something
> like
> > > that) or could be an harware issue..
> > >
> > > On the other side of the link, the other EX4300 (side B) do not
> experience
> > > the same issue but the traffic is mostly from side B to side A.
> > >
> > > Here an example of the output, statistics cleared and after 1
> minute I get
> > > 12 framing errors with 2 Mbs running on the WAN link:
> > >
> > > @EX4300-A> show interfaces ae0 extensive
> > > Physical interface: ae0, Enabled, Physical link is Up
> > > Interface index: 220, SNMP ifIndex: 549, Generation: 131
> > > Description: xxx
> > > Link-level type: Ethernet, MTU: 9192, Speed: 1Gbps, BPDU Error:
> None,
> > > Ethernet-Switching Error: None, MAC-REWRITE Error: None,
> > > Loopback: Disabled, Source filtering: Disabled, Flow control:
> Enabled,
> > > Minimum links needed: 1, Minimum bandwidth needed: 1bps
> > > Device flags : Present Running
> > > Interface flags: SNMP-Traps Internal: 0x0
> > > Current address: cc:e5:94:11:43:23, Hardware address:
> cc:e5:94:11:43:23
> > > Last flapped : 2020-04-19 02:05:05 CEST (06:50:45 ago)
> > > Statistics last cleared: 2020-04-19 09:11:22 CEST (00:01:00 ago)
> > > Traffic statistics:
> > > Input bytes : 10014863 2205456 bps
> > > Output bytes : 4095720 582456 bps
> > > Input packets: 33292 624 pps
> > > Output packets: 33023 568 pps
> > > IPv6 transit statistics:
> > > Input bytes : 0
> > > Output bytes : 0
> > > Input packets: 0
> > > Output packets: 0
> > > Input errors:
> > > Errors: 12, Drops: 0, Framing errors: 12, Runts: 0, Giants: 0,
> Policed
> > > discards: 0, Resource errors: 0
> > > Output errors:
> > > Carrier transitions: 0, Errors: 0, Drops: 0, MTU errors: 0,
> Resource
> > > errors: 0
> > > Egress queues: 12 supported, 11 in use
> > >
> > >
> > > @EX4300-A> show interfaces ge-0/0/0 extensive
> > > Physical interface: ge-0/0/0, Enabled, Physical link is Up
> > > Interface index: 649, SNMP ifIndex: 509, Generation: 140
> > > Description: WAN link
> > > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Link-mode:
> > > Full-duplex, Speed: 1000mbps, BPDU Error: None, Loop Detect PDU
> Error: None,
> > > Ethernet-Switching Error: None, Source filtering: Disabled
> > > Ethernet-Switching Error: None, MAC-REWRITE Error: None,
> Loopback:
> > > Disabled, Flow control: Enabled, Auto-negotiation: Enabled,
> > > Remote fault: Online, Media type: Copper, IEEE 802.3az Energy
> Efficient
> > > Ethernet: Disabled, Auto-MDIX: Enabled
> > > Device flags : Present Running
> > > Interface flags: SNMP-Traps Internal: 0x0
> > > Link flags : None
> > > CoS queues : 12 supported, 12 maximum usable queues
> > > Hold-times : Up 0 ms, Down 0 ms
> > > Current address: cc:e5:94:11:43:23, Hardware address:
> cc:e5:94:11:43:23
> > > Last flapped : 2020-03-28 18:43:04 CET (3w0d 13:30 ago)
> > > Statistics last cleared: 2020-04-19 09:11:18 CEST (00:02:18 ago)
> > > Traffic statistics:
> > > Input bytes : 21782579 932296 bps
> > > Output bytes : 17898068 498704 bps
> > > Input packets: 76844 569 pps
> > > Output packets: 82594 590 pps
> > > IPv6 transit statistics:
> > > Input bytes : 0
> > > Output bytes : 0
> > > Input packets: 0
> > > Output packets: 0
> > > Input errors:
> > > Errors: 28, Drops: 0, Framing errors: 28, Runts: 0, Policed
> discards:
> > > 0, L3 incompletes: 0, L2 channel errors: 0,
> > > L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
> > >
> > >
> > > Here part of the config:
> > >
> > > @EX4300-A> show configuration interfaces ge-0/0/0 | display set
> > > set interfaces ge-0/0/0 ether-options auto-negotiation
> > > set interfaces ge-0/0/0 ether-options flow-control
> > > set interfaces ge-0/0/0 ether-options 802.3ad ae0
> > >
> > >
> > > @EX4300-A> show configuration interfaces ae0 | display set
> > > set interfaces ae0 mtu 9192
> > > set interfaces ae0 aggregated-ether-options flow-control
> > > set interfaces ae0 aggregated-ether-options lacp active
> > > set interfaces ae0 aggregated-ether-options lacp periodic fast
> > > set interfaces ae0 unit 0 family ethernet-switching interface-mode
> trunk
> > > set interfaces ae0 unit 0 family ethernet-switching vlan members
> 2228
> > > set interfaces ae0 unit 0 family ethernet-switching vlan members
> 2552-2553
> > > set interfaces ae0 unit 0 family ethernet-switching filter input
> QOS
> > >
> > >
> > > @EX4300-A> show configuration security macsec | display set
> > > set security macsec connectivity-association MAC security-mode
> static-cak
> > > set security macsec connectivity-association MAC pre-shared-key
> ckn xxxx
> > > set security macsec connectivity-association MAC pre-shared-key cak
> > > "tttttvvvv"
> > > set security macsec connectivity-association MAC exclude-protocol
> lldp
> > > set security macsec connectivity-association MAC exclude-protocol
> lacp
> > > set security macsec interfaces ge-0/0/0 connectivity-association
> MAC
> > >
> > >
> > > Dear all, an help is appreciated and welcomme, please let me thank
> in
> > > advance anyone will give an hint.
> > >
> > > Cheers
> > > James
>
>
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [EXT] EX4300: Framing error with macsec enabled [ In reply to ]
As I said, I'm not excluding any protocols from MACsec. With that configuration, LLDP apparently doesn't work "outside the tunnel"--I never see any directly attached neighbors. LLDP does work between the MACsec endpoints--they show as if they are directly connected neighbors. I'm fine with that result in my case.

The solution to eliminate the spurious framing errors appears to be: Ask your carrier to shut off LLDP/CDP/any other L2 protocols running on their interfaces directly attached to your MACsec endpoint devices.

On Tue, Apr 21, 2020 at 02:59:53PM +0000, Richard McGovern wrote:
> Based upon Chuck’s reply:
>
> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on the EX3400s connected directly to the MX480s.
>
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end between the EX3400s just fine.
>
> Don’t exclude LLDP from MACSEC and either stop or block LLDP from the Carrier/ISP. In Chuck’s case the Carrier (to the EX3400s) was his MX. Then EX3400s should see each other via LLDP, but not see the carrier. I am not sure if today, you have an LLDP neighbor with your Carrier/ISP or not?
>
> This is the way I now read his response.
>
> Yes?
>
>
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
>
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
>
> [signature_1140633420]
>
> From: james list <jameslist72@gmail.com>
> Date: Tuesday, April 21, 2020 at 10:53 AM
> To: Richard McGovern <rmcgovern@juniper.net>
> Cc: Chuck Anderson <cra@wpi.edu>, Juniper List <juniper-nsp@puck.nether.net>
> Subject: Re: [j-nsp] [EXT] EX4300: Framing error with macsec enabled
>
> [External Email. Be cautious of content]
>
> Hi Richard
> lldp and lacp are excluded:
>
> > > @EX4300-A> show configuration security macsec | display set
> > > set security macsec connectivity-association MAC security-mode static-cak
> > > set security macsec connectivity-association MAC pre-shared-key ckn xxxx
> > > set security macsec connectivity-association MAC pre-shared-key cak
> > > "tttttvvvv"
> > > set security macsec connectivity-association MAC exclude-protocol lldp
> > > set security macsec connectivity-association MAC exclude-protocol lacp
> > > set security macsec interfaces ge-0/0/0 connectivity-association MAC
>
> I did not catch the connection with Framing errors counter...
>
> Please detail if you can.
>
> Cheers
>
>
> Il giorno mar 21 apr 2020 alle ore 15:27 Richard McGovern <rmcgovern@juniper.net<mailto:rmcgovern@juniper.net>> ha scritto:
> Chuck, I thought you were running both LLDP and LACP outside the MACSEC tunnel, no?
>
> (Optional) Exclude a protocol from MACsec:
> [edit security macsec connectivity-association connectivity-association-name]
> user@switch# set exclude-protocol protocol-name
> For instance, if you did not want Link Level Discovery Protocol (LLDP) to be secured using MACsec:
>
> [edit security macsec connectivity-association ca-dynamic1]
> user@switch# set exclude-protocol lldp
> When this option is enabled, MACsec is disabled for all packets of the specified protocol—in this case, LLDP—that are sent or received on the link.
>
> BEST PRACTICEWe recommend that any protocol other than MACsec being used on the MACsec connection, such as LLDP, LACP, STP, or layer 3 routing protocols, should be excluded and moved outside of the MACsec tunnel.
>
> Is this not working properly for LLDP?
>
> Rich
>
> Richard McGovern
> Sr Sales Engineer, Juniper Networks
> 978-618-3342
>
> I’d rather be lucky than good, as I know I am not good
> I don’t make the news, I just report it
>
>
> On 4/19/20, 7:31 PM, "Chuck Anderson" <cra@WPI.EDU<mailto:cra@WPI.EDU>> wrote:
>
> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on the EX3400s connected directly to the MX480s.
>
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end between the EX3400s just fine.
>
> Check if the carrier is running LLDP or CDP or similar.
>
>
> On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote:
> > Yes, I see CRC errors on EX3400s with MACsec termination, but only on one side.
> >
> > Here is my topology:
> >
> > From A to B:
> >
> > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2 vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B]
> > MACsec L2 connection L2 xconnect MACsec
> >
> > From B to A:
> >
> > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2 vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B]
> > MACsec L2 connection L2 xconnect MACsec
> >
> > I also have a redundant path with EX3400-C (different local switch) and EX3400-B (same remote switch).
> >
> > I see the CRC errors increasing at a rate of about 2-3 per minute, but only on EX3400-A and EXX3400-C.
> >
> > All EX3400s were initially running 15.1X53-D57. Now A and C are running 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been consistent throughout all releases, no improvement with upgrades.
> >
> > I wonder if something the carrier's ASR9k is sending down the VLAN towards EX3400-A and -C is causing this? If not, maybe it is the MX480s sending something locally to EX3400-A and -C?
> >
> > The following PRs don't seem relevant--I'm not doing anywhere close to 60% utilization:
> >
> > https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1261567
> >
> > And I'm not seeing "runts":
> >
> > https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1469663
> >
> > I'm only seeing Framing errors (CRC/Align errors):
> >
> > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791
> > Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
> > CRC/Align errors 227911 0
> >
> > A few seconds later, it increased to 227913:
> >
> > MAC statistics: Receive Transmit
> > Total octets 17953647117156 3221741316352
> > Total packets 13200126465 7010832956
> > Unicast packets 13194022205 7004785539
> > Broadcast packets 5272 0
> > Multicast packets 6098988 6047417
> > CRC/Align errors 227913 0
> > FIFO errors 0 0
> > MAC control frames 0 0
> > MAC pause frames 0 0
> > Oversized frames 0
> > Jabber frames 0
> > Fragment frames 0
> > VLAN tagged frames 13196813130
> > Code violations 0
> >
> > Rate is only 24 Mbps, 2200 pps:
> >
> > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match "bps|pps"
> > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps, BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error: None, MAC-REWRITE Error: None,
> > Input bytes : 17954037433802 24242136 bps
> > Output bytes : 3221749625049 498504 bps
> > Input packets: 13200416854 2190 pps
> > Output packets: 7010941872 830 pps
> >
> >
> >
> > On Sun, Apr 19, 2020 at 09:37:23AM +0200, james list wrote:
> > > Dear experts,
> > > I've an EX4300 (Junos 17.3R3-S3.3) which have a constant Framing error
> > > counter increase also if the traffic is very low.
> > > Interface is connected to a WAN link from a carrier and bw is 1 Gbs but
> > > traffic max is actually 100 Mbs and on average 10 Mbs.
> > > On this interface I've enabled macsec, if I disable macsec the issue is not
> > > in place but unfortunately macsec is mandatory to be kept enabled.
> > >
> > > I cannot sniff since the packet is encrypted but to me it seems that
> > > traffic is not lost, if I have 100 Mbs inside from WAN I see 100 Mbs
> > > outside to DataCenter.
> > >
> > > Due to the fact that monitoring system contantly raise an alert, I'd like
> > > to know how to fix it or at least let the EX4300 do not raise the counter
> > > increase.
> > >
> > > I've opened a JTAC case but they found a PR which is currently related to a
> > > Broadcom chipset raising framing errors during spikes (ie 70% of the
> > > interface bandwidth).
> > >
> > > https://kb.juniper.net/InfoCenter/index?page=content&id=KB32264&actp=METADATA
> > >
> > > Also enabling flow-control as described in the KB do not change the
> > > behaviour.
> > >
> > > I'm wondering if there could the option we're receiving some sort on
> > > "unknown protocol" from the carrier (I remeber Cisco has something like
> > > that) or could be an harware issue..
> > >
> > > On the other side of the link, the other EX4300 (side B) do not experience
> > > the same issue but the traffic is mostly from side B to side A.
> > >
> > > Here an example of the output, statistics cleared and after 1 minute I get
> > > 12 framing errors with 2 Mbs running on the WAN link:
> > >
> > > @EX4300-A> show interfaces ae0 extensive
> > > Physical interface: ae0, Enabled, Physical link is Up
> > > Interface index: 220, SNMP ifIndex: 549, Generation: 131
> > > Description: xxx
> > > Link-level type: Ethernet, MTU: 9192, Speed: 1Gbps, BPDU Error: None,
> > > Ethernet-Switching Error: None, MAC-REWRITE Error: None,
> > > Loopback: Disabled, Source filtering: Disabled, Flow control: Enabled,
> > > Minimum links needed: 1, Minimum bandwidth needed: 1bps
> > > Device flags : Present Running
> > > Interface flags: SNMP-Traps Internal: 0x0
> > > Current address: cc:e5:94:11:43:23, Hardware address: cc:e5:94:11:43:23
> > > Last flapped : 2020-04-19 02:05:05 CEST (06:50:45 ago)
> > > Statistics last cleared: 2020-04-19 09:11:22 CEST (00:01:00 ago)
> > > Traffic statistics:
> > > Input bytes : 10014863 2205456 bps
> > > Output bytes : 4095720 582456 bps
> > > Input packets: 33292 624 pps
> > > Output packets: 33023 568 pps
> > > IPv6 transit statistics:
> > > Input bytes : 0
> > > Output bytes : 0
> > > Input packets: 0
> > > Output packets: 0
> > > Input errors:
> > > Errors: 12, Drops: 0, Framing errors: 12, Runts: 0, Giants: 0, Policed
> > > discards: 0, Resource errors: 0
> > > Output errors:
> > > Carrier transitions: 0, Errors: 0, Drops: 0, MTU errors: 0, Resource
> > > errors: 0
> > > Egress queues: 12 supported, 11 in use
> > >
> > >
> > > @EX4300-A> show interfaces ge-0/0/0 extensive
> > > Physical interface: ge-0/0/0, Enabled, Physical link is Up
> > > Interface index: 649, SNMP ifIndex: 509, Generation: 140
> > > Description: WAN link
> > > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Link-mode:
> > > Full-duplex, Speed: 1000mbps, BPDU Error: None, Loop Detect PDU Error: None,
> > > Ethernet-Switching Error: None, Source filtering: Disabled
> > > Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback:
> > > Disabled, Flow control: Enabled, Auto-negotiation: Enabled,
> > > Remote fault: Online, Media type: Copper, IEEE 802.3az Energy Efficient
> > > Ethernet: Disabled, Auto-MDIX: Enabled
> > > Device flags : Present Running
> > > Interface flags: SNMP-Traps Internal: 0x0
> > > Link flags : None
> > > CoS queues : 12 supported, 12 maximum usable queues
> > > Hold-times : Up 0 ms, Down 0 ms
> > > Current address: cc:e5:94:11:43:23, Hardware address: cc:e5:94:11:43:23
> > > Last flapped : 2020-03-28 18:43:04 CET (3w0d 13:30 ago)
> > > Statistics last cleared: 2020-04-19 09:11:18 CEST (00:02:18 ago)
> > > Traffic statistics:
> > > Input bytes : 21782579 932296 bps
> > > Output bytes : 17898068 498704 bps
> > > Input packets: 76844 569 pps
> > > Output packets: 82594 590 pps
> > > IPv6 transit statistics:
> > > Input bytes : 0
> > > Output bytes : 0
> > > Input packets: 0
> > > Output packets: 0
> > > Input errors:
> > > Errors: 28, Drops: 0, Framing errors: 28, Runts: 0, Policed discards:
> > > 0, L3 incompletes: 0, L2 channel errors: 0,
> > > L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
> > >
> > >
> > > Here part of the config:
> > >
> > > @EX4300-A> show configuration interfaces ge-0/0/0 | display set
> > > set interfaces ge-0/0/0 ether-options auto-negotiation
> > > set interfaces ge-0/0/0 ether-options flow-control
> > > set interfaces ge-0/0/0 ether-options 802.3ad ae0
> > >
> > >
> > > @EX4300-A> show configuration interfaces ae0 | display set
> > > set interfaces ae0 mtu 9192
> > > set interfaces ae0 aggregated-ether-options flow-control
> > > set interfaces ae0 aggregated-ether-options lacp active
> > > set interfaces ae0 aggregated-ether-options lacp periodic fast
> > > set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
> > > set interfaces ae0 unit 0 family ethernet-switching vlan members 2228
> > > set interfaces ae0 unit 0 family ethernet-switching vlan members 2552-2553
> > > set interfaces ae0 unit 0 family ethernet-switching filter input QOS
> > >
> > >
> > > @EX4300-A> show configuration security macsec | display set
> > > set security macsec connectivity-association MAC security-mode static-cak
> > > set security macsec connectivity-association MAC pre-shared-key ckn xxxx
> > > set security macsec connectivity-association MAC pre-shared-key cak
> > > "tttttvvvv"
> > > set security macsec connectivity-association MAC exclude-protocol lldp
> > > set security macsec connectivity-association MAC exclude-protocol lacp
> > > set security macsec interfaces ge-0/0/0 connectivity-association MAC
> > >
> > >
> > > Dear all, an help is appreciated and welcomme, please let me thank in
> > > advance anyone will give an hint.
> > >
> > > Cheers
> > > James
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [EXT] EX4300: Framing error with macsec enabled [ In reply to ]
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp
Re: [EXT] EX4300: Framing error with macsec enabled [ In reply to ]
For info, the issue was related to a carrier mediaconverter sending frames
with a private unknown ethertype not decoded by EX4300.

Thanks for your help all

Cheers
James

Il Lun 20 Apr 2020, 01:31 Chuck Anderson <cra@wpi.edu> ha scritto:

> Well, that was an easy fix on my MX480s:
>
> set protocols lldp interface xe-0/0/1 disable
>
> Now I'm not seeing CRC errors incrementing 2-3 times per minute on the
> EX3400s connected directly to the MX480s.
>
> I'm not excluding any protocols from MACsec--LLDP runs end-to-end between
> the EX3400s just fine.
>
> Check if the carrier is running LLDP or CDP or similar.
>
>
> On Sun, Apr 19, 2020 at 07:16:46PM -0400, Chuck Anderson wrote:
> > Yes, I see CRC errors on EX3400s with MACsec termination, but only on
> one side.
> >
> > Here is my topology:
> >
> > From A to B:
> >
> > [EX3400-A]-->--[push-vlan-tag-on-MX480]-->-L2
> vlan-->-[Carrier-ASR9k-pop-vlan-tag]-->--[EX3400-B]
> > MACsec L2 connection L2 xconnect
> MACsec
> >
> > From B to A:
> >
> > [EX3400-A]--<--[pop-vlan-tag-on-MX480]--<-L2
> vlan--<-[Carrier-ASR9k-push-vlan-tag]--<--[EX3400-B]
> > MACsec L2 connection L2 xconnect
> MACsec
> >
> > I also have a redundant path with EX3400-C (different local switch) and
> EX3400-B (same remote switch).
> >
> > I see the CRC errors increasing at a rate of about 2-3 per minute, but
> only on EX3400-A and EXX3400-C.
> >
> > All EX3400s were initially running 15.1X53-D57. Now A and C are running
> 18.2R3-S2 and B is running 15.1X53-D592. But the problem has been
> consistent throughout all releases, no improvement with upgrades.
> >
> > I wonder if something the carrier's ASR9k is sending down the VLAN
> towards EX3400-A and -C is causing this? If not, maybe it is the MX480s
> sending something locally to EX3400-A and -C?
> >
> > The following PRs don't seem relevant--I'm not doing anywhere close to
> 60% utilization:
> >
> >
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1261567
> >
> > And I'm not seeing "runts":
> >
> >
> https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1469663
> >
> > I'm only seeing Framing errors (CRC/Align errors):
> >
> > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match 22791
> > Errors: 227911, Drops: 0, Framing errors: 227911, Runts: 0, Policed
> discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts:
> 0, FIFO errors: 0, Resource errors: 0
> > CRC/Align errors 227911 0
> >
> > A few seconds later, it increased to 227913:
> >
> > MAC statistics: Receive Transmit
> > Total octets 17953647117156 3221741316352
> > Total packets 13200126465 7010832956
> > Unicast packets 13194022205 7004785539
> > Broadcast packets 5272 0
> > Multicast packets 6098988 6047417
> > CRC/Align errors 227913 0
> > FIFO errors 0 0
> > MAC control frames 0 0
> > MAC pause frames 0 0
> > Oversized frames 0
> > Jabber frames 0
> > Fragment frames 0
> > VLAN tagged frames 13196813130
> > Code violations 0
> >
> > Rate is only 24 Mbps, 2200 pps:
> >
> > admin@ex3400-a> show interfaces extensive xe-0/2/0 |match "bps|pps"
> > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Speed: 10Gbps,
> BPDU Error: None, Loop Detect PDU Error: None, Ethernet-Switching Error:
> None, MAC-REWRITE Error: None,
> > Input bytes : 17954037433802 24242136 bps
> > Output bytes : 3221749625049 498504 bps
> > Input packets: 13200416854 2190 pps
> > Output packets: 7010941872 830 pps
> >
> >
> >
> > On Sun, Apr 19, 2020 at 09:37:23AM +0200, james list wrote:
> > > Dear experts,
> > > I've an EX4300 (Junos 17.3R3-S3.3) which have a constant Framing error
> > > counter increase also if the traffic is very low.
> > > Interface is connected to a WAN link from a carrier and bw is 1 Gbs but
> > > traffic max is actually 100 Mbs and on average 10 Mbs.
> > > On this interface I've enabled macsec, if I disable macsec the issue
> is not
> > > in place but unfortunately macsec is mandatory to be kept enabled.
> > >
> > > I cannot sniff since the packet is encrypted but to me it seems that
> > > traffic is not lost, if I have 100 Mbs inside from WAN I see 100 Mbs
> > > outside to DataCenter.
> > >
> > > Due to the fact that monitoring system contantly raise an alert, I'd
> like
> > > to know how to fix it or at least let the EX4300 do not raise the
> counter
> > > increase.
> > >
> > > I've opened a JTAC case but they found a PR which is currently related
> to a
> > > Broadcom chipset raising framing errors during spikes (ie 70% of the
> > > interface bandwidth).
> > >
> > >
> https://kb.juniper.net/InfoCenter/index?page=content&id=KB32264&actp=METADATA
> > >
> > > Also enabling flow-control as described in the KB do not change the
> > > behaviour.
> > >
> > > I'm wondering if there could the option we're receiving some sort on
> > > "unknown protocol" from the carrier (I remeber Cisco has something like
> > > that) or could be an harware issue..
> > >
> > > On the other side of the link, the other EX4300 (side B) do not
> experience
> > > the same issue but the traffic is mostly from side B to side A.
> > >
> > > Here an example of the output, statistics cleared and after 1 minute I
> get
> > > 12 framing errors with 2 Mbs running on the WAN link:
> > >
> > > @EX4300-A> show interfaces ae0 extensive
> > > Physical interface: ae0, Enabled, Physical link is Up
> > > Interface index: 220, SNMP ifIndex: 549, Generation: 131
> > > Description: xxx
> > > Link-level type: Ethernet, MTU: 9192, Speed: 1Gbps, BPDU Error: None,
> > > Ethernet-Switching Error: None, MAC-REWRITE Error: None,
> > > Loopback: Disabled, Source filtering: Disabled, Flow control:
> Enabled,
> > > Minimum links needed: 1, Minimum bandwidth needed: 1bps
> > > Device flags : Present Running
> > > Interface flags: SNMP-Traps Internal: 0x0
> > > Current address: cc:e5:94:11:43:23, Hardware address:
> cc:e5:94:11:43:23
> > > Last flapped : 2020-04-19 02:05:05 CEST (06:50:45 ago)
> > > Statistics last cleared: 2020-04-19 09:11:22 CEST (00:01:00 ago)
> > > Traffic statistics:
> > > Input bytes : 10014863 2205456 bps
> > > Output bytes : 4095720 582456 bps
> > > Input packets: 33292 624 pps
> > > Output packets: 33023 568 pps
> > > IPv6 transit statistics:
> > > Input bytes : 0
> > > Output bytes : 0
> > > Input packets: 0
> > > Output packets: 0
> > > Input errors:
> > > Errors: 12, Drops: 0, Framing errors: 12, Runts: 0, Giants: 0,
> Policed
> > > discards: 0, Resource errors: 0
> > > Output errors:
> > > Carrier transitions: 0, Errors: 0, Drops: 0, MTU errors: 0,
> Resource
> > > errors: 0
> > > Egress queues: 12 supported, 11 in use
> > >
> > >
> > > @EX4300-A> show interfaces ge-0/0/0 extensive
> > > Physical interface: ge-0/0/0, Enabled, Physical link is Up
> > > Interface index: 649, SNMP ifIndex: 509, Generation: 140
> > > Description: WAN link
> > > Link-level type: Ethernet, MTU: 9192, LAN-PHY mode, Link-mode:
> > > Full-duplex, Speed: 1000mbps, BPDU Error: None, Loop Detect PDU Error:
> None,
> > > Ethernet-Switching Error: None, Source filtering: Disabled
> > > Ethernet-Switching Error: None, MAC-REWRITE Error: None, Loopback:
> > > Disabled, Flow control: Enabled, Auto-negotiation: Enabled,
> > > Remote fault: Online, Media type: Copper, IEEE 802.3az Energy
> Efficient
> > > Ethernet: Disabled, Auto-MDIX: Enabled
> > > Device flags : Present Running
> > > Interface flags: SNMP-Traps Internal: 0x0
> > > Link flags : None
> > > CoS queues : 12 supported, 12 maximum usable queues
> > > Hold-times : Up 0 ms, Down 0 ms
> > > Current address: cc:e5:94:11:43:23, Hardware address:
> cc:e5:94:11:43:23
> > > Last flapped : 2020-03-28 18:43:04 CET (3w0d 13:30 ago)
> > > Statistics last cleared: 2020-04-19 09:11:18 CEST (00:02:18 ago)
> > > Traffic statistics:
> > > Input bytes : 21782579 932296 bps
> > > Output bytes : 17898068 498704 bps
> > > Input packets: 76844 569 pps
> > > Output packets: 82594 590 pps
> > > IPv6 transit statistics:
> > > Input bytes : 0
> > > Output bytes : 0
> > > Input packets: 0
> > > Output packets: 0
> > > Input errors:
> > > Errors: 28, Drops: 0, Framing errors: 28, Runts: 0, Policed
> discards:
> > > 0, L3 incompletes: 0, L2 channel errors: 0,
> > > L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
> > >
> > >
> > > Here part of the config:
> > >
> > > @EX4300-A> show configuration interfaces ge-0/0/0 | display set
> > > set interfaces ge-0/0/0 ether-options auto-negotiation
> > > set interfaces ge-0/0/0 ether-options flow-control
> > > set interfaces ge-0/0/0 ether-options 802.3ad ae0
> > >
> > >
> > > @EX4300-A> show configuration interfaces ae0 | display set
> > > set interfaces ae0 mtu 9192
> > > set interfaces ae0 aggregated-ether-options flow-control
> > > set interfaces ae0 aggregated-ether-options lacp active
> > > set interfaces ae0 aggregated-ether-options lacp periodic fast
> > > set interfaces ae0 unit 0 family ethernet-switching interface-mode
> trunk
> > > set interfaces ae0 unit 0 family ethernet-switching vlan members 2228
> > > set interfaces ae0 unit 0 family ethernet-switching vlan members
> 2552-2553
> > > set interfaces ae0 unit 0 family ethernet-switching filter input QOS
> > >
> > >
> > > @EX4300-A> show configuration security macsec | display set
> > > set security macsec connectivity-association MAC security-mode
> static-cak
> > > set security macsec connectivity-association MAC pre-shared-key ckn
> xxxx
> > > set security macsec connectivity-association MAC pre-shared-key cak
> > > "tttttvvvv"
> > > set security macsec connectivity-association MAC exclude-protocol lldp
> > > set security macsec connectivity-association MAC exclude-protocol lacp
> > > set security macsec interfaces ge-0/0/0 connectivity-association MAC
> > >
> > >
> > > Dear all, an help is appreciated and welcomme, please let me thank in
> > > advance anyone will give an hint.
> > >
> > > Cheers
> > > James
>
_______________________________________________
juniper-nsp mailing list juniper-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/juniper-nsp