Mailing List Archive

OSPF flapping between Nexus 7000 and ASR 1001x
Hi All,

I am running OSPF across a point to point link between a Nexus 7000
and an ASR1000x.

The OSPF works fine but every couple of hours, it breaks and neighbor
state goes into EXCHANGE and remains there for another 40 minutes or
so. It then goes into full, OSPF comes up and all is well for another
couple of hours or so and then the cycle starts again. All the time,
IP connectivity between the two point to point IPs is up.

Configuration as below..

ASR1001x#sh run int g0/0/1
interface GigabitEthernet0/0/1
description ** MPLS London - SO031732 **
mtu 9216
ip address 10.32.250.17 255.255.255.252
ip ospf network point-to-point
ip ospf mtu-ignore
negotiation auto
cdp enable
end

ASR1000x#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
40.1.1.1 0 EXCHANGE/ - 00:00:33 10.32.250.18
GigabitEtherne
t0/0/1
10.32.8.253 1 FULL/DR 00:00:32 10.32.250.1
GigabitEtherne
t0/0/0


On the Nexus
N7K1-LON-AGG# sh run int vlan 3960

interface Vlan3960
no shutdown
mtu 9216
ip address 10.32.250.18/30
ip ospf cost 1
ip ospf network point-to-point
no ip ospf passive-interface
ip ospf mtu-ignore
ip router ospf 100 area 0.0.0.32

N7K1-LON-AGG# sh ip ospf neighbors 10.32.250.2
OSPF Process ID 100 VRF default
Total number of neighbors: 15
Neighbor ID Pri State Up Time Address Interface
10.32.250.2 1 EXCHANGE/ - 00:05:06 10.32.250.17 Vlan3960
N7K1-LON-AGG#

And the logs on the ASR

Jul 9 14:33:00.682: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from FULL to DOWN, Neighbor Down: Too many
retransmissions
Jul 9 14:34:00.683: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from DOWN to DOWN, Neighbor Down: Ignore timer
expired
Jul 9 14:34:46.962: %CLNS-5-ADJCHANGE: ISIS (Overlay1): Adjacency to
N7K1-LON-OTV (Overlay1) Down, hold time expired
Jul 9 14:39:56.543: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Interface
down or detached
Jul 9 14:55:20.865: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Interface
down or detached
.Jul 9 15:06:25.955: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Too many
retransmissions
.Jul 9 15:07:25.955: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from DOWN to DOWN, Neighbor Down: Ignore timer
expired
.Jul 9 15:37:57.514: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from LOADING to FULL, Loading Done
.Jul 9 15:38:03.296: %CLNS-5-ADJCHANGE: ISIS (Overlay1): Adjacency to
8478.AC22.FEC3 (Overlay1) Up, new adjacency
.Jul 9 15:42:25.399: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user:
richard.mikisa] [Source: 10.0.1.2] [localport: 22] at 15:42:25 UTC Tue
Jul 9 2019


Can someone advise on where to look?
--
cheers
Richard
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: OSPF flapping between Nexus 7000 and ASR 1001x [ In reply to ]
Multicast.

On 7/9/19 6:58 PM, Richard Mikisa wrote:
> Hi All,
>
> I am running OSPF across a point to point link between a Nexus 7000
> and an ASR1000x.
>
> The OSPF works fine but every couple of hours, it breaks and neighbor
> state goes into EXCHANGE and remains there for another 40 minutes or
> so. It then goes into full, OSPF comes up and all is well for another
> couple of hours or so and then the cycle starts again. All the time,
> IP connectivity between the two point to point IPs is up.
>
> Configuration as below..
>
> ASR1001x#sh run int g0/0/1
> interface GigabitEthernet0/0/1
> description ** MPLS London - SO031732 **
> mtu 9216
> ip address 10.32.250.17 255.255.255.252
> ip ospf network point-to-point
> ip ospf mtu-ignore
> negotiation auto
> cdp enable
> end
>
> ASR1000x#sh ip ospf neighbor
>
> Neighbor ID Pri State Dead Time Address Interface
> 40.1.1.1 0 EXCHANGE/ - 00:00:33 10.32.250.18
> GigabitEtherne
> t0/0/1
> 10.32.8.253 1 FULL/DR 00:00:32 10.32.250.1
> GigabitEtherne
> t0/0/0
>
>
> On the Nexus
> N7K1-LON-AGG# sh run int vlan 3960
>
> interface Vlan3960
> no shutdown
> mtu 9216
> ip address 10.32.250.18/30
> ip ospf cost 1
> ip ospf network point-to-point
> no ip ospf passive-interface
> ip ospf mtu-ignore
> ip router ospf 100 area 0.0.0.32
>
> N7K1-LON-AGG# sh ip ospf neighbors 10.32.250.2
> OSPF Process ID 100 VRF default
> Total number of neighbors: 15
> Neighbor ID Pri State Up Time Address Interface
> 10.32.250.2 1 EXCHANGE/ - 00:05:06 10.32.250.17 Vlan3960
> N7K1-LON-AGG#
>
> And the logs on the ASR
>
> Jul 9 14:33:00.682: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from FULL to DOWN, Neighbor Down: Too many
> retransmissions
> Jul 9 14:34:00.683: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from DOWN to DOWN, Neighbor Down: Ignore timer
> expired
> Jul 9 14:34:46.962: %CLNS-5-ADJCHANGE: ISIS (Overlay1): Adjacency to
> N7K1-LON-OTV (Overlay1) Down, hold time expired
> Jul 9 14:39:56.543: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Interface
> down or detached
> Jul 9 14:55:20.865: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Interface
> down or detached
> .Jul 9 15:06:25.955: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Too many
> retransmissions
> .Jul 9 15:07:25.955: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from DOWN to DOWN, Neighbor Down: Ignore timer
> expired
> .Jul 9 15:37:57.514: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from LOADING to FULL, Loading Done
> .Jul 9 15:38:03.296: %CLNS-5-ADJCHANGE: ISIS (Overlay1): Adjacency to
> 8478.AC22.FEC3 (Overlay1) Up, new adjacency
> .Jul 9 15:42:25.399: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user:
> richard.mikisa] [Source: 10.0.1.2] [localport: 22] at 15:42:25 UTC Tue
> Jul 9 2019
>
>
> Can someone advise on where to look?

--
Best regards,
Adrian Minta


_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: OSPF flapping between Nexus 7000 and ASR 1001x [ In reply to ]
Is that a dark fiber P2P or a an ISP provided "L2 over L3 P2P"?
If it's the latter like an eLine link - could they be re-routing traffic on their underlay where the mtu changes maybe?

-----Original Message-----
From: cisco-nsp <cisco-nsp-bounces@puck.nether.net> On Behalf Of Richard Mikisa
Sent: Tuesday, July 9, 2019 11:58 AM
To: cisco-nsp@puck.nether.net
Subject: [c-nsp] OSPF flapping between Nexus 7000 and ASR 1001x

This message originates from outside of your organisation.

Hi All,

I am running OSPF across a point to point link between a Nexus 7000 and an ASR1000x.

The OSPF works fine but every couple of hours, it breaks and neighbor state goes into EXCHANGE and remains there for another 40 minutes or so. It then goes into full, OSPF comes up and all is well for another couple of hours or so and then the cycle starts again. All the time, IP connectivity between the two point to point IPs is up.

Configuration as below..

ASR1001x#sh run int g0/0/1
interface GigabitEthernet0/0/1
description ** MPLS London - SO031732 ** mtu 9216 ip address 10.32.250.17 255.255.255.252 ip ospf network point-to-point ip ospf mtu-ignore negotiation auto cdp enable end

ASR1000x#sh ip ospf neighbor

Neighbor ID Pri State Dead Time Address Interface
40.1.1.1 0 EXCHANGE/ - 00:00:33 10.32.250.18
GigabitEtherne
t0/0/1
10.32.8.253 1 FULL/DR 00:00:32 10.32.250.1
GigabitEtherne
t0/0/0


On the Nexus
N7K1-LON-AGG# sh run int vlan 3960

interface Vlan3960
no shutdown
mtu 9216
ip address 10.32.250.18/30
ip ospf cost 1
ip ospf network point-to-point
no ip ospf passive-interface
ip ospf mtu-ignore
ip router ospf 100 area 0.0.0.32

N7K1-LON-AGG# sh ip ospf neighbors 10.32.250.2 OSPF Process ID 100 VRF default Total number of neighbors: 15
Neighbor ID Pri State Up Time Address Interface
10.32.250.2 1 EXCHANGE/ - 00:05:06 10.32.250.17 Vlan3960
N7K1-LON-AGG#

And the logs on the ASR

Jul 9 14:33:00.682: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from FULL to DOWN, Neighbor Down: Too many retransmissions Jul 9 14:34:00.683: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from DOWN to DOWN, Neighbor Down: Ignore timer expired Jul 9 14:34:46.962: %CLNS-5-ADJCHANGE: ISIS (Overlay1): Adjacency to N7K1-LON-OTV (Overlay1) Down, hold time expired Jul 9 14:39:56.543: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Interface down or detached Jul 9 14:55:20.865: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Interface down or detached .Jul 9 15:06:25.955: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Too many retransmissions .Jul 9 15:07:25.955: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from DOWN to DOWN, Neighbor Down: Ignore timer expired .Jul 9 15:37:57.514: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
GigabitEthernet0/0/1 from LOADING to FULL, Loading Done .Jul 9 15:38:03.296: %CLNS-5-ADJCHANGE: ISIS (Overlay1): Adjacency to
8478.AC22.FEC3 (Overlay1) Up, new adjacency .Jul 9 15:42:25.399: %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user:
richard.mikisa] [Source: 10.0.1.2] [localport: 22] at 15:42:25 UTC Tue Jul 9 2019


Can someone advise on where to look?
--
cheers
Richard
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: OSPF flapping between Nexus 7000 and ASR 1001x [ In reply to ]
Try removing the "ip ospf mtu-ignore" from both ends -see if the adjacency
comes up (past exchange).
Try enabling debug to see what exactly is going on -you should see straight
away what the crux of the problem is.

adam
> -----Original Message-----
> From: cisco-nsp <cisco-nsp-bounces@puck.nether.net> On Behalf Of Richard
> Mikisa
> Sent: Tuesday, July 9, 2019 4:58 PM
> To: cisco-nsp@puck.nether.net
> Subject: [c-nsp] OSPF flapping between Nexus 7000 and ASR 1001x
>
> Hi All,
>
> I am running OSPF across a point to point link between a Nexus 7000 and an
> ASR1000x.
>
> The OSPF works fine but every couple of hours, it breaks and neighbor
state
> goes into EXCHANGE and remains there for another 40 minutes or so. It then
> goes into full, OSPF comes up and all is well for another couple of hours
or so
> and then the cycle starts again. All the time, IP connectivity between the
two
> point to point IPs is up.
>
> Configuration as below..
>
> ASR1001x#sh run int g0/0/1
> interface GigabitEthernet0/0/1
> description ** MPLS London - SO031732 ** mtu 9216 ip address
> 10.32.250.17 255.255.255.252 ip ospf network point-to-point ip ospf mtu-
> ignore negotiation auto cdp enable end
>
> ASR1000x#sh ip ospf neighbor
>
> Neighbor ID Pri State Dead Time Address
Interface
> 40.1.1.1 0 EXCHANGE/ - 00:00:33 10.32.250.18
> GigabitEtherne
> t0/0/1
> 10.32.8.253 1 FULL/DR 00:00:32 10.32.250.1
> GigabitEtherne
> t0/0/0
>
>
> On the Nexus
> N7K1-LON-AGG# sh run int vlan 3960
>
> interface Vlan3960
> no shutdown
> mtu 9216
> ip address 10.32.250.18/30
> ip ospf cost 1
> ip ospf network point-to-point
> no ip ospf passive-interface
> ip ospf mtu-ignore
> ip router ospf 100 area 0.0.0.32
>
> N7K1-LON-AGG# sh ip ospf neighbors 10.32.250.2 OSPF Process ID 100 VRF
> default Total number of neighbors: 15
> Neighbor ID Pri State Up Time Address Interface
> 10.32.250.2 1 EXCHANGE/ - 00:05:06 10.32.250.17 Vlan3960
> N7K1-LON-AGG#
>
> And the logs on the ASR
>
> Jul 9 14:33:00.682: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from FULL to DOWN, Neighbor Down: Too many
> retransmissions Jul 9 14:34:00.683: %OSPF-5-ADJCHG: Process 100, Nbr
> 40.1.1.1 on
> GigabitEthernet0/0/1 from DOWN to DOWN, Neighbor Down: Ignore timer
> expired Jul 9 14:34:46.962: %CLNS-5-ADJCHANGE: ISIS (Overlay1): Adjacency
> to N7K1-LON-OTV (Overlay1) Down, hold time expired Jul 9 14:39:56.543:
> %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Interface
> down or detached Jul 9 14:55:20.865: %OSPF-5-ADJCHG: Process 100, Nbr
> 40.1.1.1 on
> GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Interface
> down or detached .Jul 9 15:06:25.955: %OSPF-5-ADJCHG: Process 100, Nbr
> 40.1.1.1 on
> GigabitEthernet0/0/1 from EXCHANGE to DOWN, Neighbor Down: Too many
> retransmissions .Jul 9 15:07:25.955: %OSPF-5-ADJCHG: Process 100, Nbr
> 40.1.1.1 on
> GigabitEthernet0/0/1 from DOWN to DOWN, Neighbor Down: Ignore timer
> expired .Jul 9 15:37:57.514: %OSPF-5-ADJCHG: Process 100, Nbr 40.1.1.1 on
> GigabitEthernet0/0/1 from LOADING to FULL, Loading Done .Jul 9
> 15:38:03.296: %CLNS-5-ADJCHANGE: ISIS (Overlay1): Adjacency to
> 8478.AC22.FEC3 (Overlay1) Up, new adjacency .Jul 9 15:42:25.399:
> %SEC_LOGIN-5-LOGIN_SUCCESS: Login Success [user:
> richard.mikisa] [Source: 10.0.1.2] [localport: 22] at 15:42:25 UTC Tue Jul
9 2019
>
>
> Can someone advise on where to look?
> --
> cheers
> Richard
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: OSPF flapping between Nexus 7000 and ASR 1001x [ In reply to ]
On Tue, 9 Jul 2019 at 21:01, <adamv0025@netconsultings.com> wrote:
>
> Try removing the "ip ospf mtu-ignore" from both ends -see if the adjacency
> comes up (past exchange).
> Try enabling debug to see what exactly is going on -you should see straight
> away what the crux of the problem is.
>
> adam

^ This. Never use "ip ospf mtu-ignore" - you'll just mask problems
that will bite you later (like now, possibly).

Cheers,
James.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: OSPF flapping between Nexus 7000 and ASR 1001x [ In reply to ]
> On 9 Jul 2019, at 17:58, Richard Mikisa <rmikisa@gmail.com> wrote:
>
> Hi All,
>
> I am running OSPF across a point to point link between a Nexus 7000
> and an ASR1000x.
>
> The OSPF works fine but every couple of hours, it breaks and neighbor
> state goes into EXCHANGE and remains there for another 40 minutes or
> so. It then goes into full, OSPF comes up and all is well for another
> couple of hours or so and then the cycle starts again. All the time,
> IP connectivity between the two point to point IPs is up.

Apart from OSPF config changes already mentioned, check what’s going
on in your CoPP policies on both ends. This doesn’t look like a directly
config-related problem, rather environmental (like circuit switchover and
problems with MTU) but it may also be something related to amount
of traffic hitting CoPP policies from time to time and OSPF traffic overflowing
defined queues.


./
Re: OSPF flapping between Nexus 7000 and ASR 1001x [ In reply to ]
On 7/17/19 6:54 AM, ?ukasz Bromirski wrote:
>> On 9 Jul 2019, at 17:58, Richard Mikisa <rmikisa@gmail.com> wrote:
>>
>> Hi All,
>>
>> I am running OSPF across a point to point link between a Nexus 7000
>> and an ASR1000x.
>>
>> The OSPF works fine but every couple of hours, it breaks and neighbor
>> state goes into EXCHANGE and remains there for another 40 minutes or
>> so. It then goes into full, OSPF comes up and all is well for another
>> couple of hours or so and then the cycle starts again. All the time,
>> IP connectivity between the two point to point IPs is up.
> Apart from OSPF config changes already mentioned, check what’s going
> on in your CoPP policies on both ends. This doesn’t look like a directly
> config-related problem, rather environmental (like circuit switchover and
> problems with MTU) but it may also be something related to amount
> of traffic hitting CoPP policies from time to time and OSPF traffic overflowing
> defined queues.
>

You can do some debugging on ip ospf adjacency and such. I have seen
cases like this where there have been duplicate loopback ip's, duplicate
ptp link id's, and such, which cause similar results. The adjacency
debugging should give you more of a reason WHY the adjacency is going down.


Post your results if you find the cause.


Mike-

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: OSPF flapping between Nexus 7000 and ASR 1001x [ In reply to ]
Hi all

Issue was mismatched MTU. I reduced them down to 2000 after a ping
sweep and its been steady since.

On Thu, Jul 18, 2019 at 3:18 AM Mike
<mike-cisconsplist@tiedyenetworks.com> wrote:
>
> On 7/17/19 6:54 AM, ?ukasz Bromirski wrote:
> >> On 9 Jul 2019, at 17:58, Richard Mikisa <rmikisa@gmail.com> wrote:
> >>
> >> Hi All,
> >>
> >> I am running OSPF across a point to point link between a Nexus 7000
> >> and an ASR1000x.
> >>
> >> The OSPF works fine but every couple of hours, it breaks and neighbor
> >> state goes into EXCHANGE and remains there for another 40 minutes or
> >> so. It then goes into full, OSPF comes up and all is well for another
> >> couple of hours or so and then the cycle starts again. All the time,
> >> IP connectivity between the two point to point IPs is up.
> > Apart from OSPF config changes already mentioned, check what’s going
> > on in your CoPP policies on both ends. This doesn’t look like a directly
> > config-related problem, rather environmental (like circuit switchover and
> > problems with MTU) but it may also be something related to amount
> > of traffic hitting CoPP policies from time to time and OSPF traffic overflowing
> > defined queues.
> >
>
> You can do some debugging on ip ospf adjacency and such. I have seen
> cases like this where there have been duplicate loopback ip's, duplicate
> ptp link id's, and such, which cause similar results. The adjacency
> debugging should give you more of a reason WHY the adjacency is going down.
>
>
> Post your results if you find the cause.
>
>
> Mike-
>
> _______________________________________________
> cisco-nsp mailing list cisco-nsp@puck.nether.net
> https://puck.nether.net/mailman/listinfo/cisco-nsp
> archive at http://puck.nether.net/pipermail/cisco-nsp/



--
cheers
Richard
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: OSPF flapping between Nexus 7000 and ASR 1001x [ In reply to ]
On 7/18/19 1:09 AM, Richard Mikisa wrote:
> Issue was mismatched MTU. I reduced them down to 2000 after a ping
> sweep and its been steady since.


I would ping across the link with DF bit set and see what the maximum
size packet you can successfully transmit over the point-to-point and
set your MTU accordingly. Once you have them matching on both sides, as
others have suggested, remove mtu-ignore.

--
inoc.net!rblayzor
XMPP: rblayzor.AT.inoc.net
PGP: https://pgp.inoc.net/rblayzor/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/