Mailing List Archive

On 11/11/21 10:26, Gert Doering wrote:

> I've been extremely underwhelmed with the quality of IOS XR documentation
> in many respects, ROV being one of them.
>
> (And ROV on IOS XE is full of different eels... as if a totally different
> company has implemented it, without ever speaking to someone who has
> done this before, or understand what it's supposed to do)

I and the usual friends you know of have been fighting with Cisco to fix
ROV's default behaviour in IOS XE since 2015. They refuse to listen.
Personally, I gave up.

We don't run RPKI on our CSR1000v route reflectors, so I'm not bothered.

We don't run RPKI on the few ASR1000's that are in the network, and
those are also being replaced by the MX204.

We are also working on replacing our ASR920's with something better. We
ran RPKI on those for all of 3 days until we turned it off. The box has
bigger problems, like insufficient FIB, that we are already dealing with.


> We're moving towards Arista 7280R, which is not without its own set of
> surprises...

We use this box for customer Layer 2 aggregation in the data centre.
Very happy with it.

As far as its maturity goes as an IP/MPLS box goes, I cannot tell you.
But, knowing you, I'll keep my eyes on arista-nsp open wide open :-).
It's an awfully quiet list - haven't got anything new on there since
February of this year.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Nov 2021 at 10:19, Mark Tinka <mark@tinka.africa> wrote:

> Thanks for the clue, Saku. Hopefully someone here has the energy to ask
> Cisco to update their documentation, to make this a recommendation. I
> can't be asked :-).

I think it should just be a config error. You're not just cucking
yourself, but your peers and customers. So it shouldn't be a choice
you can make.

We can also imagine improvements
1) by default keep all RPKI rejects, and have 'soft-inbound never'
optionally to turn that off
2) have 1 bit per neighbor indicating policy had rpki rejects and 2
bits for validation database update iindicating database become
less/more permissive
IFF database became more permissive and neighbor has rpki
rejects and we have soft-inbound never, then refresh





--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/11/21 11:22, Saku Ytti wrote:

> I think it should just be a config error. You're not just cucking
> yourself, but your peers and customers. So it shouldn't be a choice
> you can make.

I don't disagree, especially as there are likely several other operators
working this way, and not knowing it because the neighbor either hasn't
complained, or isn't detecting for Route Refresh noise.

However, the documentation should still be updated for folk running old
code earlier than the new code which would have this improvement.


>
> We can also imagine improvements
> 1) by default keep all RPKI rejects, and have 'soft-inbound never'
> optionally to turn that off

Similar to how Junos does it, but specifically for RPKI. That would make
sense.

Of course, if someone already uses 'soft-reconfiguration inbound' for
historical reasons, then keeping it as they enable ROV works out for
them anyway.


> 2) have 1 bit per neighbor indicating policy had rpki rejects and 2
> bits for validation database update iindicating database become
> less/more permissive
> IFF database became more permissive and neighbor has rpki
> rejects and we have soft-inbound never, then refresh

Reasonable.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hello,

On Thu, 11 Nov 2021 at 10:22, Saku Ytti <saku@ytti.fi> wrote:
>
> On Thu, 11 Nov 2021 at 10:19, Mark Tinka <mark@tinka.africa> wrote:
>
> > Thanks for the clue, Saku. Hopefully someone here has the energy to ask
> > Cisco to update their documentation, to make this a recommendation. I
> > can't be asked :-).
>
> I think it should just be a config error. You're not just cucking
> yourself, but your peers and customers. So it shouldn't be a choice
> you can make.
>
> We can also imagine improvements
> 1) by default keep all RPKI rejects, and have 'soft-inbound never'
> optionally to turn that off

When enabling ROV on the ASR9001, I set everything to
"soft-reconfiguration inbound always", because I couldn't imagine how
ROV would work in a reliable way otherwise. I didn't think people
would complain about periodic route-refreshes, I just didn't trust
that it would work reliably on huge internet exchanges with many peers
and questionable BGP implementations on the other side. I wanted my
EBGP behaviour to remain *exactly* the same, just as before-ROV.

For ROV to work reliably it needs to be able to reconsider previously
rejected invalids, so I would not recommend disabling
soft-reconfiguartion inbound, instead I'd suggest to set it to always.

Of course it would be better to store ROV-rejects separately, instead
of rejecting them. Better yet: add a new drop statement in RPL, which
makes the route not eligible for path selection, but doesn't remove it
from memory - this way the operator has full flexibility. I can't
believe I'm saying this, but a famous CPE vendor from Latvia actually
supports this (routing filter action: reject vs discard) [1], maybe
the XR BGP team could steal some ideas from them ;)

I don't like to depend on optional or not commonly used BGP features
and code paths in other people's networks for changes of any kind on
my end.

Memory consumption was negligible for my use-case, iirc it was 200 -
300 MB for 30 - 40 peers, one of which was transit (so full table -
this was about a year ago). I believe we had this conversation before,
and you mentioned that this could be a DoS vector, which is true but
all the solutions that we can possibly suggest simply implement a
"selective" soft-reconfig inbound always" anyway, so the DoS vector
needs to be addressed in a different way, in my opinion.


cheers,
lukas

[1] https://wiki.mikrotik.com/wiki/Manual:Routing/Routing_filters
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/11/21 15:43, Lukas Tribus wrote:

> For ROV to work reliably it needs to be able to reconsider previously
> rejected invalids, so I would not recommend disabling
> soft-reconfiguartion inbound, instead I'd suggest to set it to always.

If my router vendor is not automatically applying BGP policy based on
RPKI state, it shouldn't matter what ended up in RIB if I have not set
an explicit policy to act on RPKI state.

So if a previously-Invalid route now becomes a VRP, and is then good to
go for export toward a neighbor based on existing policy that matches,
why can we not leverage Route Refresh to only cater for that change?

I'm wondering if we can carry a loose relationship between the VRP
database, and RIB state.


> Of course it would be better to store ROV-rejects separately, instead
> of rejecting them. Better yet: add a new drop statement in RPL, which
> makes the route not eligible for path selection, but doesn't remove it
> from memory - this way the operator has full flexibility.

I still don't get why we need to worry about Invalid routes, if the
operator is doing ROV to drop them. As soon as they become Valid, the
RTR update will tell us that and we can allow for incremental changes
for what we ask our neighbors to accept.


> I can't
> believe I'm saying this, but a famous CPE vendor from Latvia actually
> supports this (routing filter action: reject vs discard) [1], maybe
> the XR BGP team could steal some ideas from them ;)

:-).


> I don't like to depend on optional or not commonly used BGP features
> and code paths in other people's networks for changes of any kind on
> my end.

In Juniper's case, their default to keep a copy of Adj-RIB-In had the
unintended side-effect of making ROV deployment less exciting. However,
I still feel not being able to leverage Route Refresh and avoid this
clumsiness with ROV is below par for what we can design. After all, that
was the whole point of Route Refresh...


> Memory consumption was negligible for my use-case, iirc it was 200 -
> 300 MB for 30 - 40 peers, one of which was transit (so full table -
> this was about a year ago). I believe we had this conversation before,
> and you mentioned that this could be a DoS vector, which is true but
> all the solutions that we can possibly suggest simply implement a
> "selective" soft-reconfig inbound always" anyway, so the DoS vector
> needs to be addressed in a different way, in my opinion.

Well, we had several peers disable sessions with us because their CPU
was being impacted by all the refresh messages we were sending. So we
have seen it become a DoS vector toward others, and then by extension,
toward us when we have to lose shorter paths to those peers due to the
session tear-down.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hello Gert,

On Thu, 11 Nov 2021 at 08:18, Gert Doering <gert@greenie.muc.de> wrote:
>
> Hi,
>
> On Thu, Nov 11, 2021 at 08:27:44AM +0200, Mark Tinka wrote:
> > We have nothing against the forwarding performance of the ASR9001. It's
> > the control/management plane that seems to be slowing down (at least for
> > us, anyway) with newer code and a growing Internet DFZ.
>
> "newer code" might be the key issue here - what version are you on?
>
> Our 9001 have been extremely well-behaved in regards to BGP performance
> and robustness. Not as fast as the RSP440, but still well up to the
> job.
>
> We're on 6.5.3 ("nothing interesting in newer versions, and still
> supported").

When I tested RTR on IOS-XR I hit some strange bugs in the RTR client,
specifically the RTR session would hang in certain scenarios (router
restart, RTR server reboot, etc), requiring manual intervention with a
"clear bgp rpki-server 1.2.3.4". It was *a lot* better in more recent
releases, but 6.5.3 was not part of those better releases. I still did
not have a lot of trust in XR for this reason, so I wrote a NETCONF
script that would make some sanity checks on the RTR session state on
XR boxes (nagios friendly):

https://gist.github.com/lukastribus/695c9e780d118755271519d4d3cb54f3

(minimum number of absolute v4 and v6 VRPs on each RTR server, maximum
v4/v6 VRP deviation between the RTR servers, RTR servers not in
"connected" state).


cheers,
lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/11/21 16:02, Lukas Tribus wrote:

> When I tested RTR on IOS-XR I hit some strange bugs in the RTR client,
> specifically the RTR session would hang in certain scenarios (router
> restart, RTR server reboot, etc), requiring manual intervention with a
> "clear bgp rpki-server 1.2.3.4". It was *a lot* better in more recent
> releases, but 6.5.3 was not part of those better releases. I still did
> not have a lot of trust in XR for this reason, so I wrote a NETCONF
> script that would make some sanity checks on the RTR session state on
> XR boxes (nagios friendly):
>
> https://gist.github.com/lukastribus/695c9e780d118755271519d4d3cb54f3
>
> (minimum number of absolute v4 and v6 VRPs on each RTR server, maximum
> v4/v6 VRP deviation between the RTR servers, RTR servers not in
> "connected" state).

And now, with the code you are currently running for IOS XR?

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Nov 2021 at 15:12, Mark Tinka <mark@tinka.africa> wrote:
>
>
>
> On 11/11/21 16:02, Lukas Tribus wrote:
>
> > When I tested RTR on IOS-XR I hit some strange bugs in the RTR client,
> > specifically the RTR session would hang in certain scenarios (router
> > restart, RTR server reboot, etc), requiring manual intervention with a
> > "clear bgp rpki-server 1.2.3.4". It was *a lot* better in more recent
> > releases, but 6.5.3 was not part of those better releases. I still did
> > not have a lot of trust in XR for this reason, so I wrote a NETCONF
> > script that would make some sanity checks on the RTR session state on
> > XR boxes (nagios friendly):
> >
> > https://gist.github.com/lukastribus/695c9e780d118755271519d4d3cb54f3
> >
> > (minimum number of absolute v4 and v6 VRPs on each RTR server, maximum
> > v4/v6 VRP deviation between the RTR servers, RTR servers not in
> > "connected" state).
>
> And now, with the code you are currently running for IOS XR?

I haven't been able to reproduce the hung RTR session issue due to
reboot/reloads in newer code. 6.7.3 (32-bit)/7.1.3 (64-bit) should be
fine. I don't recall if 6.6.X was good or bad, but 6.5 certainly was.
Still, the monitoring script makes sense.

Lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/11/21 16:26, Lukas Tribus wrote:

> Still, the monitoring script makes sense.

Yep - lots of monitoring is needed.

I'm monitoring some odd behaviour where Fort will, after a few days,
grow its delta in the number of VRP's vs. rpki-client.

Probably one of the best reasons to run different validators.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Thu, 11 Nov 2021 at 15:01, Mark Tinka <mark@tinka.africa> wrote:
>
>
>
> On 11/11/21 15:43, Lukas Tribus wrote:
>
> > For ROV to work reliably it needs to be able to reconsider previously
> > rejected invalids, so I would not recommend disabling
> > soft-reconfiguartion inbound, instead I'd suggest to set it to always.
>
> If my router vendor is not automatically applying BGP policy based on
> RPKI state, it shouldn't matter what ended up in RIB if I have not set
> an explicit policy to act on RPKI state.
>
> So if a previously-Invalid route now becomes a VRP, and is then good to
> go for export toward a neighbor based on existing policy that matches,
> why can we not leverage Route Refresh to only cater for that change?

I believe with the amount of RAM we have on those boxes nowadays,
keeping a copy of everything should be a non-issue.

On the other hand, leveraging route-refresh changes your EBGP
behaviour, which can trigger remote and local bugs, or, as in your
case, trigger humans with most likely a little over-dramatic
monitoring. I won't trust other peoples BGP routers and
implementations more than I absolutely have to and I don't think my
time is well spent arguing with other people about their
underdimensioned control plane CPU, oversensitive CPU load monitoring
or troubleshooting corner cases in their BGP implementation that
trigger bugs in route refresh code. And then the need to explain in a
RFO why your network heavily uses route-refresh which triggered that
remote bug in the first place, while your competitor didn't change
anything in their BGP configuration in the last decade, so "they
didn't have any issue with this, only your network has issues''.

Those are all rabbit holes that I will gladly trade for a little bit
of RAM usage in a heartbeat.


Lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

Hi,

On Thu, Nov 11, 2021 at 03:02:35PM +0100, Lukas Tribus wrote:
> > We're on 6.5.3 ("nothing interesting in newer versions, and still
> > supported").
>
> When I tested RTR on IOS-XR I hit some strange bugs in the RTR client,
> specifically the RTR session would hang in certain scenarios (router
> restart, RTR server reboot, etc), requiring manual intervention with a
> "clear bgp rpki-server 1.2.3.4". It was *a lot* better in more recent
> releases, but 6.5.3 was not part of those better releases.

Yeah. That bug hits here as well, every now and then one of the two
sessions (or both) get stuck. We have a script that walks all our XR
boxes and verifies that they all see the same amount of prefixes on
both sessions, across all boxes... and then alerts, and we go "clear".

Not exactly displaying best of software quality, but more of a minor
nuisance (for us).

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de

On 11/11/21 16:33, Lukas Tribus wrote:

> I believe with the amount of RAM we have on those boxes nowadays,
> keeping a copy of everything should be a non-issue.
>
> On the other hand, leveraging route-refresh changes your EBGP
> behaviour, which can trigger remote and local bugs, or, as in your
> case, trigger humans with most likely a little over-dramatic
> monitoring. I won't trust other peoples BGP routers and
> implementations more than I absolutely have to and I don't think my
> time is well spent arguing with other people about their
> underdimensioned control plane CPU, oversensitive CPU load monitoring
> or troubleshooting corner cases in their BGP implementation that
> trigger bugs in route refresh code. And then the need to explain in a
> RFO why your network heavily uses route-refresh which triggered that
> remote bug in the first place, while your competitor didn't change
> anything in their BGP configuration in the last decade, so "they
> didn't have any issue with this, only your network has issues''.
>
> Those are all rabbit holes that I will gladly trade for a little bit
> of RAM usage in a heartbeat.

So some friends and I are working on an RFC draft to fix this:

https://datatracker.ietf.org/doc/html/draft-ymbk-sidrops-rov-no-rr

Comments and contributions are most welcome.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Sat, 13 Nov 2021 at 13:48, Mark Tinka <mark@tinka.africa> wrote:
>

> So some friends and I are working on an RFC draft to fix this:
>
> https://datatracker.ietf.org/doc/html/draft-ymbk-sidrops-rov-no-rr
>
> Comments and contributions are most welcome.

I chose my words carefully when I said 'RPKI rejects', instead of 'invalid'.

The problem only cursorily relates to a specific RPKI validation
state. We may reject RPKI 'unknown', we may even imagine policies
which reject based on some criteria AND RPKI 'valid' (maybe I have my
own motivations for how I use VRP and want to capitalise on all three
states arbitrarily, maybe I'm rejecting valids, because I'm collecting
invalids to some separate RIB for research purposes).

That is:
soft-reconfiguration inbound never # don't keep anything
soft-reconfiguration inbound rpki ## default, keep if policy
rejected route while using validation database state (may have used
something else, but as long as reject policy used validation state,
regardless of state, we need to keep it).



--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/11/21 16:49, Gert Doering wrote:
> Not exactly displaying best of software quality, but more of a minor
> nuisance (for us).

Fundamentally, we just need to provide a lot more feedback to router
vendors as well as validator coders, about real world experiences.

Especially now since, really, there are effectively just 3 - 4
validators in the wild:

https://docs.google.com/spreadsheets/d/1uuDlO6g1DLATV5OVCa20kU9OOiX9XWBFoZT2OkOezi8/edit#gid=0

Mark.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/13/21 17:20, Saku Ytti wrote:

> I chose my words carefully when I said 'RPKI rejects', instead of 'invalid'.

Well, this only really happens on IOS XE since Cisco apply policy by
default.

On IOS XR, you'll need 'bgp bestpath origin-as allow invalid' for
Invalids not to be automatically dropped.


> The problem only cursorily relates to a specific RPKI validation
> state. We may reject RPKI 'unknown', we may even imagine policies
> which reject based on some criteria AND RPKI 'valid' (maybe I have my
> own motivations for how I use VRP and want to capitalise on all three
> states arbitrarily, maybe I'm rejecting valids, because I'm collecting
> invalids to some separate RIB for research purposes).

And that is all fine, provided YOU, as the operator, are deciding policy.

The problem is that Cisco seem to want to automatically apply policy,
particularly on IOS XE. We've hounded them about this since 2015, and
nothing has changed.

IOS XR is a little better in this specific regard, but not by much when
compared against Junos.


> soft-reconfiguration inbound rpki ## default, keep if policy
> rejected route while using validation database state (may have used
> something else, but as long as reject policy used validation state,
> regardless of state, we need to keep it).

This is what we are trying to write the RFC for - to decouple the
historical need to keep or drop Adj-RIB-In from the operational
requirements of RTR dynamics, i.e., leverage the value of Route Refresh
to its fullest extent.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Sat, 13 Nov 2021 at 21:23, Mark Tinka <mark@tinka.africa> wrote:

> And that is all fine, provided YOU, as the operator, are deciding policy.

I don't think IETF is making standards for specific implementation.
The implementation agnostic solution is to keep all routes which we
rejected due to consulting validation database, regardless of
validation state.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/14/21 08:58, Saku Ytti wrote:

> I don't think IETF is making standards for specific implementation.
> The implementation agnostic solution is to keep all routes which we
> rejected due to consulting validation database, regardless of
> validation state.

Agreed, and covered in the draft:

https://datatracker.ietf.org/doc/html/draft-ymbk-sidrops-rov-no-rr-02

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Mon, 15 Nov 2021 at 16:12, Mark Tinka <mark@tinka.africa> wrote:
>
>
>
> On 11/14/21 08:58, Saku Ytti wrote:
>
> > I don't think IETF is making standards for specific implementation.
> > The implementation agnostic solution is to keep all routes which we
> > rejected due to consulting validation database, regardless of
> > validation state.
>
> Agreed, and covered in the draft:

You better clarify:

- when this is implemented, STOP triggering route-refreshes from RTR
updates (otherwise there is no point to it in the first place)
- implement the above without BREAKING existing route-refresh
functionality for non-RTR related code paths (like other RPL changes)


Lukas
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On Wed, Nov 10, 2021 at 8:15 AM Mark Tinka <mark@tinka.africa> wrote:
> Not really that interested in Cisco anymore.

We will keep our ASR 9001 until support will expire, but for small Edge nodes.

Well it is hard to trust Cisco currently.

I can recall our CSR 1000V story (permament licenses). CSR 1000V
permament licenses are EoL/EoS. But subscription CSR 1000V are not, so
you need to pay again for something you already paid for.
Next move was Catalyst 8000V which is CSR 1000V but with just new name,

How to deceive a customer who bought licenses?
1. Release a newer version under a newer name (Catalyst 8000V).
2. Retiring the previous product/version - EoS/EoL - CSR 1000V.
3. By that you simply stop discussion regarding perpetual ->
subscrption model change
4. Done. Dear customers - pay again for same piece of software!

Rob
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

On 11/22/21 14:10, Robert Hass wrote:

> We will keep our ASR 9001 until support will expire, but for small Edge nodes.
>
> Well it is hard to trust Cisco currently.
>
> I can recall our CSR 1000V story (permament licenses). CSR 1000V
> permament licenses are EoL/EoS. But subscription CSR 1000V are not, so
> you need to pay again for something you already paid for.
> Next move was Catalyst 8000V which is CSR 1000V but with just new name,
>
> How to deceive a customer who bought licenses?
> 1. Release a newer version under a newer name (Catalyst 8000V).
> 2. Retiring the previous product/version - EoS/EoL - CSR 1000V.
> 3. By that you simply stop discussion regarding perpetual ->
> subscrption model change
> 4. Done. Dear customers - pay again for same piece of software!

Oh dear, sorry to hear that. But that's exactly the kind of nonsense we
are running away from Cisco for.

We love the CSR1000v, but we are not running the latest & greatest. We
use them primarily for iBGP route reflection, and have settled on
15.5(3)S9 - a.k.a. 3.16(09)S - for some time now, even though it shipped
in March of 2019. It has all the relevant BGP features we need now, and
into the future, so we aren't worried about running old code.

We did attempt upgrading them to Amsterdam - 17.3(4a) - but that was as
cluster-f* of note. First, due to some issue between later versions of
CSR1000v (16.x and higher) and how VMware identifies interface drivers,
you lose the ability to set Jumbo frames, even if the host supports it.
This totally broke iBGP sessions because IS-IS was confused, without
looking confused.

Secondly, the license drama went to hell, same way you describe. But
what I didn't know is that Cisco make you pay again for it; what horror!
I just assumed they'd honour your previous permanent license, and port
you over. Thanks for that!

Anyway, we downgraded back to what we are running today, and are happy.
Our perpetual licenses are still valid, for the code we run.

If you need the newer code, I'd recommend taking a hard look at vMX or
vSR, just for a better account management situation, if nothing else.

Mark.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/