Mailing List Archive

Cisco 3650 IOS-XE active VLAN port state changes
Hello,

We are seeing an issue on several WS-C3650-48PS (IOS-XE) switches whereby almost every rancid check shows changes that are solely the port showing active or inactive on a particular vlan. Is there some way to ignore that? This appears to happen when a port state changes. An example of what we see each time is below.


- !VLAN: 10 VLAN0010 active Gi1/0/3, Gi1/0/4, Gi1/0/5, Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9, Gi1/0/10, Gi1/0/11
- !VLAN: Gi1/0/13, Gi1/0/14, Gi1/0/15, Gi1/0/16, Gi1/0/17, Gi1/0/18, Gi1/0/19, Gi1/0/20
- !VLAN: Gi1/0/21, Gi1/0/22, Gi1/0/23, Gi1/0/24, Gi1/0/25, Gi1/0/26, Gi1/0/27, Gi1/0/28
- !VLAN: Gi1/0/29, Gi1/0/30, Gi1/0/31, Gi1/0/32, Gi1/0/33, Gi1/0/34, Gi1/0/35, Gi1/0/36
- !VLAN: Gi1/0/37, Gi1/0/38, Gi1/0/39, Gi1/0/40, Gi1/0/41, Gi1/0/42, Gi1/0/43, Gi1/0/44
- !VLAN: Gi1/0/45, Gi1/0/46, Gi1/0/47, Gi2/0/1, Gi2/0/2, Gi2/0/3, Gi2/0/4, Gi2/0/5, Gi2/0/6
- !VLAN: Gi2/0/7, Gi2/0/8, Gi2/0/9, Gi2/0/10, Gi2/0/11, Gi2/0/12, Gi2/0/13, Gi2/0/14
+ !VLAN: 10 VLAN0010 active Gi1/0/2, Gi1/0/3, Gi1/0/4, Gi1/0/5, Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9, Gi1/0/10
+ !VLAN: Gi1/0/11, Gi1/0/13, Gi1/0/14, Gi1/0/15, Gi1/0/16, Gi1/0/17, Gi1/0/18, Gi1/0/19
+ !VLAN: Gi1/0/20, Gi1/0/21, Gi1/0/22, Gi1/0/23, Gi1/0/24, Gi1/0/25, Gi1/0/26, Gi1/0/27
+ !VLAN: Gi1/0/28, Gi1/0/29, Gi1/0/30, Gi1/0/31, Gi1/0/32, Gi1/0/33, Gi1/0/34, Gi1/0/35
+ !VLAN: Gi1/0/36, Gi1/0/37, Gi1/0/38, Gi1/0/39, Gi1/0/40, Gi1/0/41, Gi1/0/42, Gi1/0/43
+ !VLAN: Gi1/0/44, Gi1/0/45, Gi1/0/46, Gi1/0/47, Gi2/0/1, Gi2/0/2, Gi2/0/3, Gi2/0/4, Gi2/0/5
+ !VLAN:

Best,

-Troy
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Cisco 3650 IOS-XE active VLAN port state changes [ In reply to ]
Hello,

We are seeing an issue on several WS-C3650-48PS (IOS-XE) switches whereby almost every rancid check shows changes that are solely the port showing active or inactive on a particular vlan. Is there some way to ignore that? This appears to happen when a port state changes. An example of what we see each time is below.


- !VLAN: 10 VLAN0010 active Gi1/0/3, Gi1/0/4, Gi1/0/5, Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9, Gi1/0/10, Gi1/0/11
- !VLAN: Gi1/0/13, Gi1/0/14, Gi1/0/15, Gi1/0/16, Gi1/0/17, Gi1/0/18, Gi1/0/19, Gi1/0/20
- !VLAN: Gi1/0/21, Gi1/0/22, Gi1/0/23, Gi1/0/24, Gi1/0/25, Gi1/0/26, Gi1/0/27, Gi1/0/28
- !VLAN: Gi1/0/29, Gi1/0/30, Gi1/0/31, Gi1/0/32, Gi1/0/33, Gi1/0/34, Gi1/0/35, Gi1/0/36
- !VLAN: Gi1/0/37, Gi1/0/38, Gi1/0/39, Gi1/0/40, Gi1/0/41, Gi1/0/42, Gi1/0/43, Gi1/0/44
- !VLAN: Gi1/0/45, Gi1/0/46, Gi1/0/47, Gi2/0/1, Gi2/0/2, Gi2/0/3, Gi2/0/4, Gi2/0/5, Gi2/0/6
- !VLAN: Gi2/0/7, Gi2/0/8, Gi2/0/9, Gi2/0/10, Gi2/0/11, Gi2/0/12, Gi2/0/13, Gi2/0/14
+ !VLAN: 10 VLAN0010 active Gi1/0/2, Gi1/0/3, Gi1/0/4, Gi1/0/5, Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9, Gi1/0/10
+ !VLAN: Gi1/0/11, Gi1/0/13, Gi1/0/14, Gi1/0/15, Gi1/0/16, Gi1/0/17, Gi1/0/18, Gi1/0/19
+ !VLAN: Gi1/0/20, Gi1/0/21, Gi1/0/22, Gi1/0/23, Gi1/0/24, Gi1/0/25, Gi1/0/26, Gi1/0/27
+ !VLAN: Gi1/0/28, Gi1/0/29, Gi1/0/30, Gi1/0/31, Gi1/0/32, Gi1/0/33, Gi1/0/34, Gi1/0/35
+ !VLAN: Gi1/0/36, Gi1/0/37, Gi1/0/38, Gi1/0/39, Gi1/0/40, Gi1/0/41, Gi1/0/42, Gi1/0/43
+ !VLAN: Gi1/0/44, Gi1/0/45, Gi1/0/46, Gi1/0/47, Gi2/0/1, Gi2/0/2, Gi2/0/3, Gi2/0/4, Gi2/0/5
+ !VLAN:

Best,

-Troy
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Cisco 3650 IOS-XE active VLAN port state changes [ In reply to ]
Yeah, this is something I keep meaning to change as well. I haven't done it, but I think you can just put something like

next if (^!VLAN:);

in showVLAN in ios.pm. You'll see similar checks. But I also just want to put a plug in that this is a good candidate for a rancid option. I would think that people fall into two camps -- those who declare VLAN's by interface (and so are not likely to want to see the changes as interfaces change) and those who do not (who might).

Just a thought...

Linwood



-----Original Message-----
From: Rancid-discuss [mailto:rancid-discuss-bounces@shrubbery.net] On Behalf Of Troy Beisigl
Sent: Friday, November 22, 2019 7:58 PM
To: rancid-discuss@shrubbery.net
Subject: [rancid] Cisco 3650 IOS-XE active VLAN port state changes

Hello,

We are seeing an issue on several WS-C3650-48PS (IOS-XE) switches whereby almost every rancid check shows changes that are solely the port showing active or inactive on a particular vlan. Is there some way to ignore that? This appears to happen when a port state changes. An example of what we see each time is below.


- !VLAN: 10 VLAN0010 active Gi1/0/3, Gi1/0/4, Gi1/0/5, Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9, Gi1/0/10, Gi1/0/11
- !VLAN: Gi1/0/13, Gi1/0/14, Gi1/0/15, Gi1/0/16, Gi1/0/17, Gi1/0/18, Gi1/0/19, Gi1/0/20
- !VLAN: Gi1/0/21, Gi1/0/22, Gi1/0/23, Gi1/0/24, Gi1/0/25, Gi1/0/26, Gi1/0/27, Gi1/0/28
- !VLAN: Gi1/0/29, Gi1/0/30, Gi1/0/31, Gi1/0/32, Gi1/0/33, Gi1/0/34, Gi1/0/35, Gi1/0/36
- !VLAN: Gi1/0/37, Gi1/0/38, Gi1/0/39, Gi1/0/40, Gi1/0/41, Gi1/0/42, Gi1/0/43, Gi1/0/44
- !VLAN: Gi1/0/45, Gi1/0/46, Gi1/0/47, Gi2/0/1, Gi2/0/2, Gi2/0/3, Gi2/0/4, Gi2/0/5, Gi2/0/6
- !VLAN: Gi2/0/7, Gi2/0/8, Gi2/0/9, Gi2/0/10, Gi2/0/11, Gi2/0/12, Gi2/0/13, Gi2/0/14
+ !VLAN: 10 VLAN0010 active Gi1/0/2, Gi1/0/3, Gi1/0/4, Gi1/0/5, Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9, Gi1/0/10
+ !VLAN: Gi1/0/11, Gi1/0/13, Gi1/0/14, Gi1/0/15, Gi1/0/16, Gi1/0/17, Gi1/0/18, Gi1/0/19
+ !VLAN: Gi1/0/20, Gi1/0/21, Gi1/0/22, Gi1/0/23, Gi1/0/24, Gi1/0/25, Gi1/0/26, Gi1/0/27
+ !VLAN: Gi1/0/28, Gi1/0/29, Gi1/0/30, Gi1/0/31, Gi1/0/32, Gi1/0/33, Gi1/0/34, Gi1/0/35
+ !VLAN: Gi1/0/36, Gi1/0/37, Gi1/0/38, Gi1/0/39, Gi1/0/40, Gi1/0/41, Gi1/0/42, Gi1/0/43
+ !VLAN: Gi1/0/44, Gi1/0/45, Gi1/0/46, Gi1/0/47, Gi2/0/1, Gi2/0/2, Gi2/0/3, Gi2/0/4, Gi2/0/5
+ !VLAN:

Best,

-Troy
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Cisco 3650 IOS-XE active VLAN port state changes [ In reply to ]
You can also develop a custom type that doesn't call "show vlan".

Also, I've had this occur twice in the past.
- One time was happening campus-wide. I dug into it hard, and after a good amount of effort found out there was something actually happening based on a misunderstanding I had about native VLANs work in IOS. In other words: (a) I learned something, and (b) I found I had an actual misconfiguration.
- The other time it turned out that there was a hardware fault on the ASIC (we're actually still using that particular Catalyst 3508).

weylin

?On 11/22/19, 8:08 PM, "on@LEFerguson.com" <on@LEFerguson.com> wrote:

Yeah, this is something I keep meaning to change as well. I haven't done it, but I think you can just put something like

next if (^!VLAN:);

in showVLAN in ios.pm. You'll see similar checks. But I also just want to put a plug in that this is a good candidate for a rancid option. I would think that people fall into two camps -- those who declare VLAN's by interface (and so are not likely to want to see the changes as interfaces change) and those who do not (who might).

Just a thought...

Linwood



-----Original Message-----
From: Rancid-discuss [mailto:rancid-discuss-bounces@shrubbery.net] On Behalf Of Troy Beisigl
Sent: Friday, November 22, 2019 7:58 PM
To: rancid-discuss@shrubbery.net
Subject: [rancid] Cisco 3650 IOS-XE active VLAN port state changes

Hello,

We are seeing an issue on several WS-C3650-48PS (IOS-XE) switches whereby almost every rancid check shows changes that are solely the port showing active or inactive on a particular vlan. Is there some way to ignore that? This appears to happen when a port state changes. An example of what we see each time is below.


- !VLAN: 10 VLAN0010 active Gi1/0/3, Gi1/0/4, Gi1/0/5, Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9, Gi1/0/10, Gi1/0/11
- !VLAN: Gi1/0/13, Gi1/0/14, Gi1/0/15, Gi1/0/16, Gi1/0/17, Gi1/0/18, Gi1/0/19, Gi1/0/20
- !VLAN: Gi1/0/21, Gi1/0/22, Gi1/0/23, Gi1/0/24, Gi1/0/25, Gi1/0/26, Gi1/0/27, Gi1/0/28
- !VLAN: Gi1/0/29, Gi1/0/30, Gi1/0/31, Gi1/0/32, Gi1/0/33, Gi1/0/34, Gi1/0/35, Gi1/0/36
- !VLAN: Gi1/0/37, Gi1/0/38, Gi1/0/39, Gi1/0/40, Gi1/0/41, Gi1/0/42, Gi1/0/43, Gi1/0/44
- !VLAN: Gi1/0/45, Gi1/0/46, Gi1/0/47, Gi2/0/1, Gi2/0/2, Gi2/0/3, Gi2/0/4, Gi2/0/5, Gi2/0/6
- !VLAN: Gi2/0/7, Gi2/0/8, Gi2/0/9, Gi2/0/10, Gi2/0/11, Gi2/0/12, Gi2/0/13, Gi2/0/14
+ !VLAN: 10 VLAN0010 active Gi1/0/2, Gi1/0/3, Gi1/0/4, Gi1/0/5, Gi1/0/6, Gi1/0/7, Gi1/0/8, Gi1/0/9, Gi1/0/10
+ !VLAN: Gi1/0/11, Gi1/0/13, Gi1/0/14, Gi1/0/15, Gi1/0/16, Gi1/0/17, Gi1/0/18, Gi1/0/19
+ !VLAN: Gi1/0/20, Gi1/0/21, Gi1/0/22, Gi1/0/23, Gi1/0/24, Gi1/0/25, Gi1/0/26, Gi1/0/27
+ !VLAN: Gi1/0/28, Gi1/0/29, Gi1/0/30, Gi1/0/31, Gi1/0/32, Gi1/0/33, Gi1/0/34, Gi1/0/35
+ !VLAN: Gi1/0/36, Gi1/0/37, Gi1/0/38, Gi1/0/39, Gi1/0/40, Gi1/0/41, Gi1/0/42, Gi1/0/43
+ !VLAN: Gi1/0/44, Gi1/0/45, Gi1/0/46, Gi1/0/47, Gi2/0/1, Gi2/0/2, Gi2/0/3, Gi2/0/4, Gi2/0/5
+ !VLAN:

Best,

-Troy
_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss




_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Cisco 3650 IOS-XE active VLAN port state changes [ In reply to ]
Sat, Nov 23, 2019 at 09:50:06PM +0000, Piegorsch, Weylin William:
> You can also develop a custom type that doesn't call "show vlan".

please do this, rather than change ios.pm. This makes it easier for you
to upgrade rancid, both of which i prefer because it is easier to support
you.

> Also, I've had this occur twice in the past.
> - One time was happening campus-wide. I dug into it hard, and after a good amount of effort found out there was something actually happening based on a misunderstanding I had about native VLANs work in IOS. In other words: (a) I learned something, and (b) I found I had an actual misconfiguration.
> - The other time it turned out that there was a hardware fault on the ASIC (we're actually still using that particular Catalyst 3508).

I would like to understand why this occurs for some folks and change
the code to automatically ignore show vlan output when the switch is
configured in a manner that would lead to it. I know that VTP does
this and sometimes 802.1x and the current code tries to recognize
both of these. tia for any help here.

_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss
Re: Cisco 3650 IOS-XE active VLAN port state changes [ In reply to ]
> I would like to understand why this occurs for some folks

The hardware fault, I still can’t explain what's happening.

The other issue that I encountered was with native VLAN tagging. Um... I'm not sure this is something that RANCiD is geared to tackle, but here goes anyway.

Depending on where you look for guidance, if you tag the native VLAN:
- global config on most switches: "vlan dot1q tag native"
- interface config on Cat6k (and possibly N7k, untested): "switchport trunk native vlan tag"

and also specify a native VLAN ("switchport trunk native vlan <x>"), different guidance tells you sometimes to or not-to add the native VLAN to the trunk. What I found is that if I tagged the native VLAN, used a non-default native VLAN, and didn’t include it, I got spurious behavior, but largely if I either had different native VLANs across the various different trunk ports or connected to a remote device whose adjacent interface was configured differently than the local device. If I tag the native VLAN ewverywhere and use a non-default native VLAN everywhere (the same native VLAN on both ends of a link), and also always include the native VLAN on the trunk, then it all worked ok.

What was the spurious behavior?

In the past, I've done some extensive independent research on native VLAN tagging, and found that VLAN 1 is always on a trunk - for some specific protocols - whether it's explicitly allowed or not (see posting I wrote on Cisco's community support forum, reference 1 below). TAC confirmed that if you use a native VLAN besides VLAN 1, then you should allow the native VLAN on the trunk regardless of the tagged state. I found across a lot of regression testing that results varied by HW model/SW version, but if you allow the native VLAN and configure both ends the same way, then things are stable.

Personally, I suspect STP Loopguard, but that's just a guess plus some interesting log messages. UDLD also had interesting log messages. I also saw some weird messages from CDP, LLDP, and (most-weirdly) even LACP on non-bundled interfaces, but I'm most suspect of STP and somewhat suspect of UDLD. Basically, I never got to the root issue of what protocol was causing RANCiD's issues, but I found that doing things "correctly" then RANCiD's problem went away.

Also, caution - if you’re trying for logic to determine if a VLAN is on a trunk or not, consider this scenario. All 5 commands on a single interface, what's the "native vlan"? (hint: 22, and an egressed frame is either untagged or in some cases tagged with VLAN 0)
switchport mode access
switchport access vlan 22
switchport trunk native vlan tag
switchport trunk native vlan 117
switchport trunk allowed vlan 102-128

I've had to redo VLAN definitions across entire datacenters, but I had a devil of a time finding the extent that VLANs were applied (shutdown ports; VLAN ranges; admin-up ports that had no SFP or were unpatched; etc). I had to write a script to help me out, see reference 2 below.

As for support in RANCiD - RANCiD's not built to organically be a configuration compliance validation and/or enforcement tool (though, it's got fantastic baseline to be useful as a launching point). The problem I had was a problem of configuration compliance, and a problem of fabric inter-relationships. Compliance can be built around RANCiD, but I'm concerned that if you build that in, you're really taking a big bite, especially if you're trying to be multi-vendor about it. Doing that would lead to the world of SDN. PRIME INFRASTRUCTURE and DNA-C are better tools for this (at least for Cisco kit), and if those are too expensive for a given network shop then developing and maintaining compliance scripts might be a better approach than building it directly into RANCiD.

Though, I would be thrilled if shrubbery.net had a compliance tool built around RANCiD.

Reference 1: https://community.cisco.com/t5/switching/does-the-native-vlan-need-to-be-allowed-on-the-trunk-port/td-p/1648181/page/2
Reference 2: https://community.cisco.com/t5/switching/identify-if-vlan-is-applied-to-switchports/td-p/3693599

weylin

?On 11/25/19, 11:58 AM, "john heasley" <heas@shrubbery.net> wrote:

Sat, Nov 23, 2019 at 09:50:06PM +0000, Piegorsch, Weylin William:
> You can also develop a custom type that doesn't call "show vlan".

please do this, rather than change ios.pm. This makes it easier for you
to upgrade rancid, both of which i prefer because it is easier to support
you.

> Also, I've had this occur twice in the past.
> - One time was happening campus-wide. I dug into it hard, and after a good amount of effort found out there was something actually happening based on a misunderstanding I had about native VLANs work in IOS. In other words: (a) I learned something, and (b) I found I had an actual misconfiguration.
> - The other time it turned out that there was a hardware fault on the ASIC (we're actually still using that particular Catalyst 3508).

I would like to understand why this occurs for some folks and change
the code to automatically ignore show vlan output when the switch is
configured in a manner that would lead to it. I know that VTP does
this and sometimes 802.1x and the current code tries to recognize
both of these. tia for any help here.


_______________________________________________
Rancid-discuss mailing list
Rancid-discuss@shrubbery.net
http://www.shrubbery.net/mailman/listinfo/rancid-discuss