Mailing List Archive: Sup2T multicast hardware forwarding missing

Sup2T multicast hardware forwarding missing

Oct 17, 2019, 10:55 AM

Post #1 of 4 (820 views)

Hi,

I just spent two days debugging a multicast issue with a quad sup2t 6509 running 151-2.SY3

I am at my wits end and while I wait for the customers cisco partner to finally escalate the case
we opened to cisco I would like to try my luck here.

It seems that the box no longer does HW Forwarding for ip mfib.

Router#show ip mfib
Entry Flags: C - Directly Connected, S - Signal, IA - Inherit A flag,
ET - Data Rate Exceeds Threshold, K - Keepalive
DDE - Data Driven Event, HW - Hardware Installed
I/O Item Flags: IC - Internal Copy, NP - Not platform switched,
NS - Negate Signalling, SP - Signal Present,
A - Accept, F - Forward, RA - MRIB Accept, RF - MRIB Forward,
MA - MFIB Accept
Forwarding Counts: Pkt Count/Pkts per second/Avg Pkt Size/Kbits per second
Other counts: Total/RPF failed/Other drops
I/O Item Counts: FS Pkt Count/PS Pkt Count
Default
(*,224.0.0.0/4) Flags: C
SW Forwarding: 0/0/0/0, Other: 0/0/0
Null0 Flags: NS
(*,224.0.1.40) Flags: C
SW Forwarding: 0/0/0/0, Other: 0/0/0
Tunnel2 Flags: A
Vlan50 Flags: F IC NS
Pkts: 0/0
(*,239.0.0.30) Flags: C
SW Forwarding: 0/0/0/0, Other: 0/0/0
Tunnel2 Flags: A
Vlan50 Flags: F NS
Pkts: 0/0
Tunnel4, MDT Decap Flags: F NS
Pkts: 0/0

The box started dropping so many mcast packets that iptv streams where not showing anymore.

A full power cycle [1] in the service window resolved the issue for a couple of hours.

The next morning HW Forwrding was gone again.

I have not found a way to renable HW Forwarding and am unsure about the real cause of this as the
config had been working fine for 4 years and then broke over night.

[1] We initially wanted to upgrade to 151-2.SY12 in the service window but the issu upgrade failed
miserably and we had to had to hard reboot sups from the cmp. But that is another story.

Any ideas ?

Greetings
Christian

--
Christian Kratzer CK Software GmbH
Email: ck@cksoft.de Wildberger Weg 24/2
Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden
Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart
Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer
Web: http://www.cksoft.de/
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Sup2T multicast hardware forwarding missing [ In reply to ]

lukasz at bromirski

Oct 18, 2019, 2:28 AM

Post #2 of 4 (817 views)

Permalink

Christian,

What does the 'show mls cef exception status’ say?

If you want, send the SR # to my @cisco.com address (lukasz.bromirski) and I’ll
look into what’s going on at the TAC end.

Thanks!
--
?ukasz Bromirski
CCIE R&S/SP #15929, CCDE #2012::17, PGP Key ID: 0xFD077F6A

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Sup2T multicast hardware forwarding missing [ In reply to ]

ck-lists at cksoft

Oct 21, 2019, 12:52 AM

Post #3 of 4 (805 views)

Permalink

Hi,

On Fri, 18 Oct 2019, ?ukasz Bromirski wrote:
> Christian,
>
> What does the 'show mls cef exception status’ say?
>
> If you want, send the SR # to my @cisco.com address (lukasz.bromirski) and I’ll
> look into what’s going on at the TAC end.

summary to the list.

This turned out to be plain old run out of resources

#show platform hardware cef resource-level
Global watermarks: apply to Fib shared area only.
Protocol watermarks: apply to protocols with non-default max-routes

Fib-size: 256k (262144), shared-size: 248k (253952), shared-usage: 234k(239753)

Global watermarks:
Red_WM: 95%, Greem_WM: 80%, Current usage: 94%

Protocol watermarks:

Protocol Red_WM(%) Green_WM(%) Current(%)
-------- --------- ---------- ----------
IPV4 -- -- 30% (of shared)
IPV4-MCAST -- -- 0 % (of shared)
IPV6 -- -- 64% (of shared)
IPV6-MCAST -- -- 0 % (of shared)
MPLS -- -- 0 % (of shared)
EoMPLS -- -- 0 % (of shared)
VPLS-IPV4-MCAST -- -- 0 % (of shared)
VPLS-IPV6-MCAST -- -- 0 % (of shared)

We just needed a pointer to the right place to look, filterd bgp routes to a minimum and hw forwarding was instantly back.

Neither the gold partner with the support contract nor tac has yet replied with the simple details.

We also searched for ages because we were expecting the router to log something when it runs low on tcam or whatnot hardware resources.

Once again we see that community support seems to run circles around vendor or partner support contracts.

Greetings
Christian

--
Christian Kratzer CK Software GmbH
Email: ck@cksoft.de Wildberger Weg 24/2
Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden
Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart
Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer
Web: http://www.cksoft.de/
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Re: Sup2T multicast hardware forwarding missing [ In reply to ]

hf0002 at uah

Oct 21, 2019, 3:00 PM

Post #4 of 4 (802 views)

Permalink

The router should log something like this, actually:

DFC4-4-FIB_EXCEPTION: FIB TCAM exception for IPv4 multicast, Some
routes will be software switched.

--
Hunter Fuller
Router Jockey
VBH Annex B-5
+1 256 824 5331

Office of Information Technology
The University of Alabama in Huntsville
Network Engineering

On Mon, Oct 21, 2019 at 2:53 AM Christian Kratzer <ck-lists@cksoft.de> wrote:
>
> Hi,
>
> On Fri, 18 Oct 2019, ?ukasz Bromirski wrote:
> > Christian,
> >
> > What does the 'show mls cef exception status’ say?
> >
> > If you want, send the SR # to my @cisco.com address (lukasz.bromirski) and I’ll
> > look into what’s going on at the TAC end.
>
> summary to the list.
>
> This turned out to be plain old run out of resources
>
> #show platform hardware cef resource-level
> Global watermarks: apply to Fib shared area only.
> Protocol watermarks: apply to protocols with non-default max-routes
>
> Fib-size: 256k (262144), shared-size: 248k (253952), shared-usage: 234k(239753)
>
> Global watermarks:
> Red_WM: 95%, Greem_WM: 80%, Current usage: 94%
>
> Protocol watermarks:
>
> Protocol Red_WM(%) Green_WM(%) Current(%)
> -------- --------- ---------- ----------
> IPV4 -- -- 30% (of shared)
> IPV4-MCAST -- -- 0 % (of shared)
> IPV6 -- -- 64% (of shared)
> IPV6-MCAST -- -- 0 % (of shared)
> MPLS -- -- 0 % (of shared)
> EoMPLS -- -- 0 % (of shared)
> VPLS-IPV4-MCAST -- -- 0 % (of shared)
> VPLS-IPV6-MCAST -- -- 0 % (of shared)
>
> We just needed a pointer to the right place to look, filterd bgp routes to a minimum and hw forwarding was instantly back.
>
> Neither the gold partner with the support contract nor tac has yet replied with the simple details.
>
> We also searched for ages because we were expecting the router to log something when it runs low on tcam or whatnot hardware resources.
>
> Once again we see that community support seems to run circles around vendor or partner support contracts.
>
> Greetings
> Christian
>
> --
> Christian Kratzer CK Software GmbH
> Email: ck@cksoft.de Wildberger Weg 24/2
> Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden
> Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart
> Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer
> Web: http://www.cksoft.de/
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/