Mailing List Archive: [nsp] 7200, 12.0(21)S4 and "show int" counters?

[nsp] 7200, 12.0(21)S4 and "show int" counters?

Aug 20, 2002, 3:51 PM

Post #1 of 6 (3217 views)

Hi,

is it just me, or has Cisco engineering reached a new all-time high in
"show int" counter brokenness?

I just upgraded one of our 7200s to 12.0(21)S4 to get some of the S2/S3
bugfixes plus the "E3/T3 adapters don't work" bugfix S3->S4.

The good news: the T3 PA works fine (PA-2T3).

The bad news: the "show int" output counters are finally and completely
broken - they don't hang eventually (as in earlier releases), but they
don't count *at all* anymore. Or to be precise, they count "something"
but that has no relation to reality.

Look at this:

Cisco-M-XII#sh int p4/0
POS4/0 is up, line protocol is up
Hardware is Packet over Sonet
[..]
Last input 00:00:00, output 00:00:00, output hang never
Last clearing of "show interface" counters never
[..]
5 minute input rate 8321000 bits/sec, 1341 packets/sec
5 minute output rate 2838000 bits/sec, 638 packets/sec
513382 packets input, 396467799 bytes, 0 no buffer
Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
0 parity
18 input errors, 18 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
28 packets output, 2658 bytes, 0 underruns
0 output errors, 0 applique, 2 interface resets
0 output buffer failures, 0 output buffers swapped out
0 carrier transitions
Cisco-M-XII#sh int p4/0 acc
Protocol Pkts In Chars In Pkts Out Chars Out
IP 518490 400148237 237198 136223697
CDP 7 2457 6 2130

I'm amazed. How difficult can it be to implement interface counters
that actually count what's being sent out of the interface, and do so
*correctly* and *all the time*? This is broken since 11.2P, and I
tend to give up hope that it will *EVER* be fixed.

(Yes, some sort of bug fix went into 12.0(22)S, but I'm not going to
touch *that* can of worms)

It would be so easy - just return the SNMP counters minus a correction
value that is saved when you type "clear counter int ..." - as the SNMP
counters are (mostly) correct.

Did I mention that the SNMP values for "5 minute ... rate" differ up to
20% from the "show int" values for the same rate? I have no idea how
they managed *that*.

thoroughly annoyed,

gert

--
USENET is *not* the non-clickable part of WWW!
//www.muc.de/~gert/
Gert Doering - Munich, Germany gert@greenie.muc.de
fax: +49-89-35655025 gert.doering@physik.tu-muenchen.de

Re: [nsp] 7200, 12.0(21)S4 and "show int" counters? [ In reply to ]

dek at hades

Aug 20, 2002, 4:45 PM

Post #2 of 6 (3053 views)

Permalink

12.0(21)S2 FastEth dot1q subintarfaces on 7500's don't count "In" packets,
that were dCEF-switched. Nor CAR "input" counters on the same sbifs are
working. We've been told it's not fixed in 12.0(22)S. Oh, yes, forgot
- Netflow on the same dot1q subifs aren't working, either. *sigh*.

On Wed, Aug 21, 2002 at 12:51:24AM +0200, Gert Doering wrote:
> Hi,
>
> is it just me, or has Cisco engineering reached a new all-time high in
> "show int" counter brokenness?
>
> I just upgraded one of our 7200s to 12.0(21)S4 to get some of the S2/S3
> bugfixes plus the "E3/T3 adapters don't work" bugfix S3->S4.
>
> The good news: the T3 PA works fine (PA-2T3).
>
> The bad news: the "show int" output counters are finally and completely
> broken - they don't hang eventually (as in earlier releases), but they
> don't count *at all* anymore. Or to be precise, they count "something"
> but that has no relation to reality.
>
> Look at this:
>
> Cisco-M-XII#sh int p4/0
> POS4/0 is up, line protocol is up
> Hardware is Packet over Sonet
> [..]
> Last input 00:00:00, output 00:00:00, output hang never
> Last clearing of "show interface" counters never
> [..]
> 5 minute input rate 8321000 bits/sec, 1341 packets/sec
> 5 minute output rate 2838000 bits/sec, 638 packets/sec
> 513382 packets input, 396467799 bytes, 0 no buffer
> Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
> 0 parity
> 18 input errors, 18 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
> 28 packets output, 2658 bytes, 0 underruns
> 0 output errors, 0 applique, 2 interface resets
> 0 output buffer failures, 0 output buffers swapped out
> 0 carrier transitions
> Cisco-M-XII#sh int p4/0 acc
> Protocol Pkts In Chars In Pkts Out Chars Out
> IP 518490 400148237 237198 136223697
> CDP 7 2457 6 2130
>
> I'm amazed. How difficult can it be to implement interface counters
> that actually count what's being sent out of the interface, and do so
> *correctly* and *all the time*? This is broken since 11.2P, and I
> tend to give up hope that it will *EVER* be fixed.
>
> (Yes, some sort of bug fix went into 12.0(22)S, but I'm not going to
> touch *that* can of worms)
>
> It would be so easy - just return the SNMP counters minus a correction
> value that is saved when you type "clear counter int ..." - as the SNMP
> counters are (mostly) correct.
>
> Did I mention that the SNMP values for "5 minute ... rate" differ up to
> 20% from the "show int" values for the same rate? I have no idea how
> they managed *that*.
>
> thoroughly annoyed,
>
> gert
>
> --
> USENET is *not* the non-clickable part of WWW!
> //www.muc.de/~gert/
> Gert Doering - Munich, Germany gert@greenie.muc.de
> fax: +49-89-35655025 gert.doering@physik.tu-muenchen.de
> _______________________________________________
> cisco-nsp mailing list real_name)s@puck.nether.net
> http://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/
---end quoted text---

--
CCNP, CCDP (R&S) Dmitri E. Kalintsev
CDPlayer@irc Network Architect @ connect.com.au
dek @ connect.com.au phone: +61 3 9674 3913 fax: 9251 3666
http://-UNAVAIL- UIN:7150410 cell: +61 414 821 382

Re: [nsp] 7200, 12.0(21)S4 and "show int" counters? [ In reply to ]

ler at lerctr

Aug 20, 2002, 4:59 PM

Post #3 of 6 (3068 views)

Permalink

On Tue, 2002-08-20 at 17:51, Gert Doering wrote:
> Hi,
>
> is it just me, or has Cisco engineering reached a new all-time high in
> "show int" counter brokenness?
>
> I just upgraded one of our 7200s to 12.0(21)S4 to get some of the S2/S3
> bugfixes plus the "E3/T3 adapters don't work" bugfix S3->S4.
>
> The good news: the T3 PA works fine (PA-2T3).
>
> The bad news: the "show int" output counters are finally and completely
> broken - they don't hang eventually (as in earlier releases), but they
> don't count *at all* anymore. Or to be precise, they count "something"
> but that has no relation to reality.
>
> Look at this:
At least it's showing SOMETHING. on 12.0(22)S it shows 0's on ALL VIP
based interfaces, at least on a 7513 of mine.

Still waiting for a real fix.

:-(

[snip]
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 972-414-9812 E-Mail: ler@lerctr.org
US Mail: 1905 Steamboat Springs Drive, Garland, TX 75044-6749

Re: [nsp] 7200, 12.0(21)S4 and "show int" counters? [ In reply to ]

pekkas at netcore

Aug 20, 2002, 10:35 PM

Post #4 of 6 (3069 views)

Permalink

On Wed, 21 Aug 2002, Gert Doering wrote:
> is it just me, or has Cisco engineering reached a new all-time high in
> "show int" counter brokenness?

Don't count on it.

This almost wins my favoourite:

ATM0/0/0 is up, line protocol is up
Hardware is cyBus ENHANCED ATM PA, address is 0010.2fcc.6800 (bia 0010.2fcc.6
[...]
5 minute output rate 1014029704 bits/sec, 1682 packets/sec
[...]

It's nice to know STM1 ATM managed to accelerate to 1 Gbit/s.. and big
packets those are, too! :-)

(This was fixed, at least for currently, somewhere between 12.0(15) and
12.0(17)).

> I just upgraded one of our 7200s to 12.0(21)S4 to get some of the S2/S3
> bugfixes plus the "E3/T3 adapters don't work" bugfix S3->S4.
>
> The good news: the T3 PA works fine (PA-2T3).
>
> The bad news: the "show int" output counters are finally and completely
> broken - they don't hang eventually (as in earlier releases), but they
> don't count *at all* anymore. Or to be precise, they count "something"
> but that has no relation to reality.
>
> Look at this:
>
> Cisco-M-XII#sh int p4/0
> POS4/0 is up, line protocol is up
> Hardware is Packet over Sonet
> [..]
> Last input 00:00:00, output 00:00:00, output hang never
> Last clearing of "show interface" counters never
> [..]
> 5 minute input rate 8321000 bits/sec, 1341 packets/sec
> 5 minute output rate 2838000 bits/sec, 638 packets/sec
> 513382 packets input, 396467799 bytes, 0 no buffer
> Received 0 broadcasts, 0 runts, 0 giants, 0 throttles
> 0 parity
> 18 input errors, 18 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
> 28 packets output, 2658 bytes, 0 underruns
> 0 output errors, 0 applique, 2 interface resets
> 0 output buffer failures, 0 output buffers swapped out
> 0 carrier transitions
> Cisco-M-XII#sh int p4/0 acc
> Protocol Pkts In Chars In Pkts Out Chars Out
> IP 518490 400148237 237198 136223697
> CDP 7 2457 6 2130
>
> I'm amazed. How difficult can it be to implement interface counters
> that actually count what's being sent out of the interface, and do so
> *correctly* and *all the time*? This is broken since 11.2P, and I
> tend to give up hope that it will *EVER* be fixed.
>
> (Yes, some sort of bug fix went into 12.0(22)S, but I'm not going to
> touch *that* can of worms)
>
> It would be so easy - just return the SNMP counters minus a correction
> value that is saved when you type "clear counter int ..." - as the SNMP
> counters are (mostly) correct.
>
> Did I mention that the SNMP values for "5 minute ... rate" differ up to
> 20% from the "show int" values for the same rate? I have no idea how
> they managed *that*.
>
> thoroughly annoyed,
>
> gert
>
>

--
Pekka Savola "Tell me of difficulties surmounted,
Netcore Oy not those you stumble over and fall"
Systems. Networks. Security. -- Robert Jordan: A Crown of Swords

Re: [nsp] 7200, 12.0(21)S4 and "show int" counters? [ In reply to ]

gert at greenie

Aug 21, 2002, 1:14 AM

Post #5 of 6 (3066 views)

Permalink

Hi,

On Tue, Aug 20, 2002 at 06:59:21PM -0500, Larry Rosenman wrote:
> > The bad news: the "show int" output counters are finally and completely
> > broken - they don't hang eventually (as in earlier releases), but they
> > don't count *at all* anymore. Or to be precise, they count "something"
> > but that has no relation to reality.
> >
> > Look at this:
> At least it's showing SOMETHING. on 12.0(22)S it shows 0's on ALL VIP
> based interfaces, at least on a 7513 of mine.

As the "SOMETHING" is just "28 packets out of many 1000", this isn't
really something to qualify for "at least"... - 0 is more honest.

But thanks for your replies - it means it's not only me :-)

For the records, because Jared asked me about it: we're doing CEF
switching, and classical IP accounting with netflow feature-acceleration.
No other special features (subifs, MPLS, ...), and it affects all sorts
of interfaces (POS-OC3, T3, FastE).

gert

--
USENET is *not* the non-clickable part of WWW!
//www.muc.de/~gert/
Gert Doering - Munich, Germany gert@greenie.muc.de
fax: +49-89-35655025 gert.doering@physik.tu-muenchen.de

Re: [nsp] 7200, 12.0(21)S4 and "show int" counters? [ In reply to ]

gert at greenie

Aug 21, 2002, 1:18 AM

Post #6 of 6 (3080 views)

Permalink

hi,

On Wed, Aug 21, 2002 at 08:35:16AM +0300, Pekka Savola wrote:
> On Wed, 21 Aug 2002, Gert Doering wrote:
> > is it just me, or has Cisco engineering reached a new all-time high in
> > "show int" counter brokenness?
>
> Don't count on it.
>
> This almost wins my favoourite:
>
> ATM0/0/0 is up, line protocol is up
> Hardware is cyBus ENHANCED ATM PA, address is 0010.2fcc.6800 (bia 0010.2fcc.6
> [...]
> 5 minute output rate 1014029704 bits/sec, 1682 packets/sec
> [...]
>
> It's nice to know STM1 ATM managed to accelerate to 1 Gbit/s.. and big
> packets those are, too! :-)

That's an old one :-) - I've seen that on our FastE's as well, bit rates
up to a Gbit. I'm actually pretty sure what happens here - when the 32bit
byte counter wraps, it seems to overflow the bit rate calculation and
insert a huuuuuuge sample into the values used for this "5 minute decay"
calculation.

I've had a script save away interface counters and output rate, and it
was reproduceable - every time the output counter wrapped, there was a
Gbit-"Burst" on the output rate.

> (This was fixed, at least for currently, somewhere between 12.0(15) and
> 12.0(17)).

Yep.

I think that this one is actually excuseable in a way - there is a
calculation going on, with averaging and so, and a 32bit overflow is
happening which is something all programmers do in a while.

The other thing - counters not counting at all - I just can't understand.

gert
--
USENET is *not* the non-clickable part of WWW!
//www.muc.de/~gert/
Gert Doering - Munich, Germany gert@greenie.muc.de
fax: +49-89-35655025 gert.doering@physik.tu-muenchen.de