Mailing List Archive

Cisco CPU goes crazy
Hello,

Rancid version is 3.7 on Linux.

When Rancid logs on to the Cisco switches listed bellow, the CPU goes crazy for a few seconds which causes some processes like OSPF to converge.

Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 316 ports
Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 124 ports
Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 200 ports
Stack of 2 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 56 ports
Stack of 2 x Cisco WS-C3650-24PS, cat3k_caa-universalk9.16.03.05b.SPA.bin, 56 ports
Stack of 3 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.07.05.E.152-3.E5.bin, 80 ports
Stack of 2 x Cisco WS-C3850-48P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 104 ports

Regards,
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Cisco CPU goes crazy [ In reply to ]
Fri, May 04, 2018 at 09:51:54AM -0400, Gerhard Mourani:
> Hello,
>
> Rancid version is 3.7 on Linux.
>
> When Rancid logs on to the Cisco switches listed bellow, the CPU goes crazy for a few seconds which causes some processes like OSPF to converge.

suppose you're hitting a IOS bug. figure-out which command causes the
high cpu usage and open a ticket with cisco.

> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 316 ports
> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 124 ports
> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 200 ports
> Stack of 2 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 56 ports
> Stack of 2 x Cisco WS-C3650-24PS, cat3k_caa-universalk9.16.03.05b.SPA.bin, 56 ports
> Stack of 3 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.07.05.E.152-3.E5.bin, 80 ports
> Stack of 2 x Cisco WS-C3850-48P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 104 ports
>
> Regards,
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@shrubbery.net
> http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Cisco CPU goes crazy [ In reply to ]
Hello,

Thanks for the reply.
How to know which command causes it with rancid ?

Regards,

> On May 4, 2018, at 10:51 AM, heasley <heas@shrubbery.net> wrote:
>
> Fri, May 04, 2018 at 09:51:54AM -0400, Gerhard Mourani:
>> Hello,
>>
>> Rancid version is 3.7 on Linux.
>>
>> When Rancid logs on to the Cisco switches listed bellow, the CPU goes crazy for a few seconds which causes some processes like OSPF to converge.
>
> suppose you're hitting a IOS bug. figure-out which command causes the
> high cpu usage and open a ticket with cisco.
>
>> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 316 ports
>> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 124 ports
>> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 200 ports
>> Stack of 2 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 56 ports
>> Stack of 2 x Cisco WS-C3650-24PS, cat3k_caa-universalk9.16.03.05b.SPA.bin, 56 ports
>> Stack of 3 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.07.05.E.152-3.E5.bin, 80 ports
>> Stack of 2 x Cisco WS-C3850-48P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 104 ports
>>
>> Regards,
>> _______________________________________________
>> Rancid-discuss mailing list
>> Rancid-discuss@shrubbery.net
>> http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Cisco CPU goes crazy [ In reply to ]
Fri, May 04, 2018 at 11:00:02AM -0400, Gerhard Mourani:
> How to know which command causes it with rancid ?

try each of the commands rancid uses.

rancid -t cisco -C

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Cisco CPU goes crazy [ In reply to ]
Cisco can help you with this. You'd have another window open looking at
the process table information at the time that rancid is running. They
have a continuous mode.


On 5/4/2018 11:00 AM, Gerhard Mourani wrote:
> Hello,
>
> Thanks for the reply.
> How to know which command causes it with rancid ?
>
> Regards,
>
>> On May 4, 2018, at 10:51 AM, heasley <heas@shrubbery.net> wrote:
>>
>> Fri, May 04, 2018 at 09:51:54AM -0400, Gerhard Mourani:
>>> Hello,
>>>
>>> Rancid version is 3.7 on Linux.
>>>
>>> When Rancid logs on to the Cisco switches listed bellow, the CPU goes crazy for a few seconds which causes some processes like OSPF to converge.
>> suppose you're hitting a IOS bug. figure-out which command causes the
>> high cpu usage and open a ticket with cisco.
>>
>>> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 316 ports
>>> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 124 ports
>>> Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 200 ports
>>> Stack of 2 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 56 ports
>>> Stack of 2 x Cisco WS-C3650-24PS, cat3k_caa-universalk9.16.03.05b.SPA.bin, 56 ports
>>> Stack of 3 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.07.05.E.152-3.E5.bin, 80 ports
>>> Stack of 2 x Cisco WS-C3850-48P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 104 ports
>>>
>>> Regards,
>>> _______________________________________________
>>> Rancid-discuss mailing list
>>> Rancid-discuss@shrubbery.net
>>> http://www.shrubbery.net/mailman/listinfo/rancid-discuss
> _______________________________________________
> Rancid-discuss mailing list
> Rancid-discuss@shrubbery.net
> http://www.shrubbery.net/mailman/listinfo/rancid-discuss

--
Doug Hughes
Keystone NAP
Fairless Hills, PA
1.844.KEYBLOCK (539.2562)
Re: Cisco CPU goes crazy [ In reply to ]
If you have AAA accounting configured, you can see the commands in the accounting logs.

See also the command “show history all | begin rancid” (my rancid username in AAA is “rancid” and my rancid version is 3.4, so you might see different output).

weylin


cumm111-b17des01#sh ver | inc ^\*

* 1 56 WS-C3850-48P 03.07.00E cat3k_caa-universalk9 INSTALL

cumm111-b17des01#sh history all | beg rancid

May 4 16:24:45.760 EDT: %PARSER-5-CFGLOG_LOGGEDCMD: User:rancid logged command:!exec: enable

CMD: 'terminal length 0' 16:24:45 EDT Fri May 4 2018

CMD: 'terminal width 132' 16:24:46 EDT Fri May 4 2018

CMD: 'show version' 16:24:46 EDT Fri May 4 2018

CMD: 'show redundancy secondary' 16:24:46 EDT Fri May 4 2018

CMD: 'show idprom backplane' 16:24:46 EDT Fri May 4 2018

CMD: 'show install active' 16:24:46 EDT Fri May 4 2018

CMD: 'show env all' 16:24:46 EDT Fri May 4 2018

CMD: 'show rsp chassis-info' 16:24:47 EDT Fri May 4 2018

CMD: 'show gsr chassis' 16:24:47 EDT Fri May 4 2018

CMD: 'show diag chassis-info' 16:24:47 EDT Fri May 4 2018

CMD: 'show boot' 16:24:47 EDT Fri May 4 2018

CMD: 'show bootvar' 16:24:47 EDT Fri May 4 2018

CMD: 'show variables boot' 16:24:47 EDT Fri May 4 2018

CMD: 'show license udi' 16:24:47 EDT Fri May 4 2018

CMD: 'show license feature' 16:24:47 EDT Fri May 4 2018

CMD: 'show license' 16:24:47 EDT Fri May 4 2018

CMD: 'show activation-key' 16:24:47 EDT Fri May 4 2018

CMD: 'show flash' 16:24:47 EDT Fri May 4 2018

CMD: 'dir /all nvram:' 16:24:47 EDT Fri May 4 2018

CMD: 'dir /all bootflash:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slot0:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all disk0:' 16:24:48 EDT Fri May 4 2018



CMD: 'dir /all slot1:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all disk1:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slot2:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all disk2:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all harddisk:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all harddiska:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all harddiskb:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sup-bootdisk:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sup-bootflash:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sup-microcode:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slavenvram:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slavebootflash:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slaveslot0:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slavedisk0:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slaveslot1:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slavedisk1:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slaveslot2:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slavedisk2:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all slavesup-bootflash:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sec-nvram:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sec-bootflash:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sec-slot0:' 16:24:48 EDT Fri May 4 2018



CMD: 'dir /all sec-disk0:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sec-slot1:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sec-disk1:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sec-slot2:' 16:24:48 EDT Fri May 4 2018

CMD: 'dir /all sec-disk2:' 16:24:48 EDT Fri May 4 2018

CMD: 'show controllers' 16:24:48 EDT Fri May 4 2018

CMD: 'show controllers cbus' 16:24:49 EDT Fri May 4 2018

CMD: 'show diagbus' 16:24:49 EDT Fri May 4 2018

CMD: 'show diag' 16:24:49 EDT Fri May 4 2018

CMD: 'show capture' 16:24:49 EDT Fri May 4 2018

CMD: 'show module' 16:24:49 EDT Fri May 4 2018

CMD: 'show spe version' 16:24:49 EDT Fri May 4 2018

CMD: 'show c7200' 16:24:49 EDT Fri May 4 2018

CMD: 'show cellular 0 profile' 16:24:49 EDT Fri May 4 2018

CMD: 'show inventory raw' 16:24:49 EDT Fri May 4 2018

CMD: 'show vtp status' 16:24:50 EDT Fri May 4 2018

CMD: 'show vlan' 16:24:50 EDT Fri May 4 2018

CMD: 'show vlan-switch' 16:24:50 EDT Fri May 4 2018

CMD: 'show switch detail' 16:24:50 EDT Fri May 4 2018

CMD: 'show sdm prefer' 16:24:50 EDT Fri May 4 2018

CMD: 'show system mtu' 16:24:51 EDT Fri May 4 2018

CMD: 'show debug' 16:24:51 EDT Fri May 4 2018



CMD: 'show shun' 16:24:51 EDT Fri May 4 2018

CMD: 'more system:running-config' 16:25:00 EDT Fri May 4 2018

CMD: 'show running-config view full' 16:25:09 EDT Fri May 4 2018

CMD: 'show running-config' 16:25:18 EDT Fri May 4 2018

CMD: 'write term' 16:25:27 EDT Fri May 4 2018

CMD: 'exit' 16:25:36 EDT Fri May 4 2018

CMD: 'en' 16:53:32 EDT Fri May 4 2018

May 4 16:53:34.519 EDT: %PARSER-5-CFGLOG_LOGGEDCMD: User:weylin logged command:!exec: enable failed

CMD: 'en' 16:53:36 EDT Fri May 4 2018

May 4 16:53:37.971 EDT: %PARSER-5-CFGLOG_LOGGEDCMD: User:weylin logged command:!exec: enable

CMD: 'sh ver' 16:53:40 EDT Fri May 4 2018

CMD: 'sh ver | inc ^\*' 16:53:52 EDT Fri May 4 2018

CMD: 'sh run' 16:54:02 EDT Fri May 4 2018

CMD: 'sh ver | inc ^\*' 16:54:15 EDT Fri May 4 2018

CMD: 'sh run | sec ^archive' 16:54:21 EDT Fri May 4 2018

CMD: 'sh archive log config all' 16:54:41 EDT Fri May 4 2018

CMD: 'sh archive log config all | exclude exec enable' 16:54:56 EDT Fri May 4 2018

CMD: 'sh archive log config all | exclude exec: enable' 16:55:05 EDT Fri May 4 2018

CMD: 'sh archive log config all' 16:55:17 EDT Fri May 4 2018

CMD: 'sh logg' 16:55:30 EDT Fri May 4 2018

CMD: 'sh history ' 16:55:49 EDT Fri May 4 2018

CMD: 'sh history all' 16:55:53 EDT Fri May 4 2018



CMD: 'sh history all' 16:57:02 EDT Fri May 4 2018

CMD: 'sh history all | beg rancid' 16:57:34 EDT Fri May 4 2018

cumm111-b17des01#


weylin

From: Doug Hughes <doug.hughes@keystonenap.com>
Date: Friday, May 4, 2018 at 12:11 PM
To: <rancid-discuss@shrubbery.net>
Subject: Re: [rancid] Cisco CPU goes crazy


Cisco can help you with this. You'd have another window open looking at the process table information at the time that rancid is running. They have a continuous mode.

On 5/4/2018 11:00 AM, Gerhard Mourani wrote:

Hello,



Thanks for the reply.

How to know which command causes it with rancid ?



Regards,



On May 4, 2018, at 10:51 AM, heasley <heas@shrubbery.net><mailto:heas@shrubbery.net> wrote:



Fri, May 04, 2018 at 09:51:54AM -0400, Gerhard Mourani:

Hello,



Rancid version is 3.7 on Linux.



When Rancid logs on to the Cisco switches listed bellow, the CPU goes crazy for a few seconds which causes some processes like OSPF to converge.

suppose you're hitting a IOS bug. figure-out which command causes the

high cpu usage and open a ticket with cisco.



Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 316 ports

Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 124 ports

Cisco WS-C4510R+E, cat4500e-universalk9.SPA.03.07.03.E.152-3.E3.bin, 200 ports

Stack of 2 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 56 ports

Stack of 2 x Cisco WS-C3650-24PS, cat3k_caa-universalk9.16.03.05b.SPA.bin, 56 ports

Stack of 3 x Cisco WS-C3850-24P, cat3k_caa-universalk9.SPA.03.07.05.E.152-3.E5.bin, 80 ports

Stack of 2 x Cisco WS-C3850-48P, cat3k_caa-universalk9.SPA.03.06.03.E.152-2.E3.bin, 104 ports



Regards,

_______________________________________________

Rancid-discuss mailing list

Rancid-discuss@shrubbery.net<mailto:Rancid-discuss@shrubbery.net>

http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________

Rancid-discuss mailing list

Rancid-discuss@shrubbery.net<mailto:Rancid-discuss@shrubbery.net>

http://www.shrubbery.net/mailman/listinfo/rancid-discuss

--
Doug Hughes
Keystone NAP
Fairless Hills, PA
1.844.KEYBLOCK (539.2562)

[cid:part1.F6EFCE0B.7B0068D5@keystonenap.com]