Mailing List Archive

MTU and PMTUD
Hi all,

Assuming typical MPLS network running L2VPNs and L3VPNs, how do you
configure MTU values for core interfaces? Do you set interface (L2) MTU,
MPLS MTU and IP MTU separately?

Do you enable PMTUD for TCP based control plane protocols like BGP or LDP?


Thanks!

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
> Assuming typical MPLS network running L2VPNs and L3VPNs, how do you
> configure MTU values for core interfaces? Do you set interface (L2) MTU,
> MPLS MTU and IP MTU separately?

We set the 'physical' MTU (In IOS-XR+Junos L2 but no CRC is in this
humber) as high as it went when the decision was made. Today you can
do I think 10k in Cisco8k and 16k in Juniper. We do not seet MPLS or
IP MTUs separately in Core. On the edge you should always set L3 MTU,
because you want to have the ability to add subinterfaces with large
MTU, so physical MTU must be large, as change will affect all
subinterfaces. This way you can later add big MTU subint, without
affecting other subints.

> Do you enable PMTUD for TCP based control plane protocols like BGP or LDP?

No, not at this time, our BGP transfer performance is limited by TCP
window-size, so larger packets would not do anything for us. And LDP
has a trivial amount of stable data so it doesn't matter.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
> We set the 'physical' MTU (In IOS-XR+Junos L2 but no CRC is in this
> humber) as high as it went when the decision was made. Today you can
> do I think 10k in Cisco8k and 16k in Juniper. We do not seet MPLS or
> IP MTUs separately in Core. On the edge you should always set L3 MTU,
> because you want to have the ability to add subinterfaces with large
> MTU, so physical MTU must be large, as change will affect all
> subinterfaces. This way you can later add big MTU subint, without
> affecting other subints.

Thanks. Actually, I should have said "core-facing" interfaces in the edge :)

XR without enabled PMTUD (default setting) means ~1240 bytes available
for TCP payload.

That seems to be a bit small, did you perform any kind of performance
testing to see the difference between defaults and let's say 9000 for BGP?

I'm thinking about RRs in particular, higher MTU (9000 vs 1200) should
result in some performance benefit, at least from CPU point of view. I
haven't tested this though.


> No, not at this time, our BGP transfer performance is limited by TCP
> window-size, so larger packets would not do anything for us. And LDP
> has a trivial amount of stable data so it doesn't matter.

Agree. Thing is, PMTUD on XR is a global setting, so it does affect all
TCP based protocols.

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
Hey Marcin,

> XR without enabled PMTUD (default setting) means ~1240 bytes available
> for TCP payload.
>
> That seems to be a bit small, did you perform any kind of performance
> testing to see the difference between defaults and let's say 9000 for BGP?

I am embarrassed to say, but I _just_ found out, like literally weeks
ago, that Junos BGP TCP window is 16kB, I did also find hidden command
(https://github.com/ytti/seeker-junos) to bump it up to 64kB. I am
working with JNPR to have public documentation for the hidden command
to improve supportability and optics. I am separately working on hopes
of getting TCP window scaling.
I know that we are limited by TCP window, as the BGP transfer speed is
within 0.5% of theoretical max, and increasing 16kB to 64kB increases
BGP transfer speed exactly 4 times, being still capped by TCP window.
I think Cisco can go to 130k, but won't by default.
Maybe after that issue is remedied I will review packet size.

> I'm thinking about RRs in particular, higher MTU (9000 vs 1200) should
> result in some performance benefit, at least from CPU point of view. I
> haven't tested this though.

I've seen Cisco presentations in the 90s and early 00s showing
significant benefit from it. I have no idea how accurate it is
today,nor why it would have made a difference in the past, like was
the CPU interrupt rate constrained?

> Agree. Thing is, PMTUD on XR is a global setting, so it does affect all
> TCP based protocols.

You can do 'tcp mss X' under neighbor stanza.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
Hi Saku,

> I am embarrassed to say, but I _just_ found out, like literally weeks
> ago, that Junos BGP TCP window is 16kB, I did also find hidden command
> (https://github.com/ytti/seeker-junos) to bump it up to 64kB. I am
> working with JNPR to have public documentation for the hidden command
> to improve supportability and optics. I am separately working on hopes
> of getting TCP window scaling.
> I know that we are limited by TCP window, as the BGP transfer speed is
> within 0.5% of theoretical max, and increasing 16kB to 64kB increases
> BGP transfer speed exactly 4 times, being still capped by TCP window.
> I think Cisco can go to 130k, but won't by default.
> Maybe after that issue is remedied I will review packet size.

Thanks for sharing, that's some great info here. Probably I'll have to
take a closer look at the TCP window setting as well.

> I've seen Cisco presentations in the 90s and early 00s showing
> significant benefit from it. I have no idea how accurate it is
> today,nor why it would have made a difference in the past, like was
> the CPU interrupt rate constrained?

I'm sorry, I didn't get that part about constrained CPU interrupt rate?
My simple way of looking into that is that if we bump up the MTU, we end
up with fewer packets on the wire, so less processing on both sides.

> You can do 'tcp mss X' under neighbor stanza.

Thanks, gave it a try today, but I'm getting some strange results.

Testing using XR 7.5.2 and older IOS XE, resulting MSS depends on who is
passive/active.


Kind regards,
Marcin

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
On Wed, 7 Dec 2022 at 22:20, Marcin Kurek <md.kurek@gmail.com> wrote:

> > I've seen Cisco presentations in the 90s and early 00s showing
> > significant benefit from it. I have no idea how accurate it is
> > today,nor why it would have made a difference in the past, like was
> > the CPU interrupt rate constrained?
>
> I'm sorry, I didn't get that part about constrained CPU interrupt rate?
> My simple way of looking into that is that if we bump up the MTU, we end
> up with fewer packets on the wire, so less processing on both sides.

To handle NIC received packets you can do two things

a) CPU can get interrupt, and handle the interrupt
b) Interrupts can be disabled, and CPU can poll to see if there are
packets to process

The mechanism a) is the norm and the mechanism b) is modernish. To
improve PPS performance under heavy rate, at cost of increasing jitter
and latency because it takes variable time to pick up packet. In
software based routers, like VXR, if you had precise enough (thanks
Creanord!) measurements of network performance, you could observe
jitter during rancid (Thanks Heas!) collections, because 'show run'
and 'write' raises interrupts, which stops packet forwarding.

So less PPS, less interrupt, might be one contributing factor. I don't
know what the overhead cost of processing packets is, but intuitively
I don't expect much improvement with large MTU BGP packets. And at any
rate, going above 4k would mean newish features you don't have. But I
don't have high confidence in being right.

> Testing using XR 7.5.2 and older IOS XE, resulting MSS depends on who is
> passive/active.

MSS is 'negotiated' to the smallest. Much like BGP timers are
'negotiated' to the smallest (so your customer controls your BGP
timers, not you). Does this help to explain what you saw?



--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
Hi Saku,

> To handle NIC received packets you can do two things
>
> a) CPU can get interrupt, and handle the interrupt
> b) Interrupts can be disabled, and CPU can poll to see if there are
> packets to process
>
> The mechanism a) is the norm and the mechanism b) is modernish. To
> improve PPS performance under heavy rate, at cost of increasing jitter
> and latency because it takes variable time to pick up packet. In
> software based routers, like VXR, if you had precise enough (thanks
> Creanord!) measurements of network performance, you could observe
> jitter during rancid (Thanks Heas!) collections, because 'show run'
> and 'write' raises interrupts, which stops packet forwarding.

Interesting, but why would 'sh run' or 'write' raise an interrupt?
Isn't this a branch in code that handles the CLI?
I'm not sure if I'm reading it right - on the one hand, the interrupts are
disabled, but on the other hand, some CLI commands actually raise them?

> So less PPS, less interrupt, might be one contributing factor. I don't
> know what the overhead cost of processing packets is, but intuitively
> I don't expect much improvement with large MTU BGP packets. And at any
> rate, going above 4k would mean newish features you don't have. But I
> don't have high confidence in being right.

Would you mind elaborating on why going above 4k would mean "newish
features" and what are they?

>MSS is 'negotiated' to the smallest. Much like BGP timers are
>'negotiated' to the smallest (so your customer controls your BGP
>timers, not you). Does this help to explain what you saw?

Right, MSS should be 'negotiated' to the smallest. But what I'm referring
to is a situation where negotiated MSS value depends on who is initiating
the BGP session.

Scenario:

CSR1kv (12.0.0.13) ----------(iBGP)----------------- ASR9006 (12.0.0.7)

If I clear the BGP session on CSR1kv, resulting MSS is 1240.

Logs from CSR1kv:

*Dec 8 11:17:15.453: TCB7FFB9A6D64C0 bound to 12.0.0.13.20794
*Dec 8 11:17:15.453: Reserved port 20794 in Transport Port Agent for TCP
IP type 1
*Dec 8 11:17:15.453: TCP: pmtu enabled,mss is now set to 8936
*Dec 8 11:17:15.453: TCP: sending SYN, seq 1638888268, ack 0
*Dec 8 11:17:15.453: TCP0: Connection to 12.0.0.7:179, advertising MSS 8936
*Dec 8 11:17:15.453: TCP0: state was CLOSED -> SYNSENT [20794 ->
12.0.0.7(179)]
*Dec 8 11:17:15.456: TCP0: state was SYNSENT -> ESTAB [20794 ->
12.0.0.7(179)]
*Dec 8 11:17:15.456: TCP: tcb 7FFB9A6D64C0 connection to 12.0.0.7:179,
peer MSS 1240, MSS is 1240
*Dec 8 11:17:15.456: TCB7FFB9A6D64C0 connected to 12.0.0.7.179
*Dec 8 11:17:15.456: TCB7FFB9A6D64C0 setting property TCP_NO_DELAY (0)
7FFBF2CCED7C
*Dec 8 11:17:15.456: TCB7FFB9A6D64C0 setting property TCP_RTRANSTMO (36)
7FFBF2CCED7C
*Dec 8 11:17:18.081: TCP0: RETRANS timeout timer expired
*Dec 8 11:17:18.081: 12.0.0.13:20794 <---> 12.0.0.7:179 congestion
window changes
*Dec 8 11:17:18.081: cwnd from 1240 to 1240, ssthresh from 65535 to 2480
*Dec 8 11:17:18.081: TCP0: timeout #1 - timeout is 5250 ms, seq 1638888269
*Dec 8 11:17:18.081: TCP: (20794) -> 12.0.0.7(179)
*Dec 8 11:17:18.084: %BGP_SESSION-5-ADJCHANGE: neighbor 12.0.0.7 IPv4
Unicast topology base removed from session Capability changed
*Dec 8 11:17:18.084: %BGP-5-ADJCHANGE: neighbor 12.0.0.7 Up
CSR1000v#show bgp vpnv4 unicast all neighbors 12.0.0.7 | i segment
Maximum output segment queue size: 50
Datagrams (max data segment is 1240 bytes):

So here CSR1kv is initiating the connection to XR box advertising MSS 8936
(as expected).
However, peer MSS is 1240, which is not quite expected, considering XR
config:

neighbor 12.0.0.13
remote-as 12
tcp mss 8936
update-source Loopback0
address-family vpnv4 unicast
!
address-family ipv4 rt-filter
!
address-family l2vpn evpn
!

If I clear the BGP session on XR, resulting MSS is 8936.

Again, logs from CSR1kv:

*Dec 8 11:20:22.918: TCB7FFB9A6CF7A8 created
*Dec 8 11:20:22.918: TCP0: state was LISTEN -> SYNRCVD [179 ->
12.0.0.7(38087)]
*Dec 8 11:20:22.918: TCP: tcb 7FFB9A6CF7A8 connection to 12.0.0.7:38087,
peer MSS 8936, MSS is 516
*Dec 8 11:20:22.918: TCP: pmtu enabled,mss is now set to 8936
*Dec 8 11:20:22.918: TCP: sending SYN, seq 1440899137, ack 3532544296
*Dec 8 11:20:22.918: TCP0: Connection to 12.0.0.7:38087, advertising MSS
8936
*Dec 8 11:20:22.921: TCP0: state was SYNRCVD -> ESTAB [179 ->
12.0.0.7(38087)]
*Dec 8 11:20:22.921: TCB7FFBF25C07E0 accepting 7FFB9A6CF7A8 from
12.0.0.7.38087
*Dec 8 11:20:22.921: TCB7FFB9A6CF7A8 setting property TCP_VRFTABLEID (20)
7FFB9A0B9820
*Dec 8 11:20:22.921: TCB7FFB9A6CF7A8 setting property TCP_PMTU (45)
7FFBF2CCED00
*Dec 8 11:20:22.921: TCB7FFB9A6CF7A8 setting property TCP_NO_DELAY (0)
7FFBF2CCED60
*Dec 8 11:20:22.921: TCB7FFB9A6CF7A8 setting property TCP_ACK_RATE (37)
7FFBF2CCED5C
*Dec 8 11:20:22.921: TCB7FFB9A6CF7A8 setting property TCP_RTRANSTMO (36)
7FFBF2CCED60
*Dec 8 11:20:22.922: %BGP_SESSION-5-ADJCHANGE: neighbor 12.0.0.7 IPv4
Unicast topology base removed from session Capability changed
*Dec 8 11:20:22.926: %BGP-5-ADJCHANGE: neighbor 12.0.0.7 Up

CSR1000v#show bgp vpnv4 unicast all neighbors 12.0.0.7 | i segment
Maximum output segment queue size: 50
Datagrams (max data segment is 8936 bytes):

Here it's the other way around, XR is the client, and XE is the server and
apparently XR has advertised MSS=8936.

But maybe I don't understand how this command 'tcp mss' is supposed to work
:)


Kind regards,
Marcin
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
On Thu, 8 Dec 2022 at 11:25, Marcin Kurek <md.kurek@gmail.com> wrote:

> Interesting, but why would 'sh run' or 'write' raise an interrupt?
> Isn't this a branch in code that handles the CLI?

Maybe to access NVRAM? I don't know for sure, I didn't pursue it, I
just know I could observe it.

> I'm not sure if I'm reading it right - on the one hand, the interrupts are disabled, but on the other hand, some CLI commands actually raise them?

I don't know if IOS does polling or interrupt for NIC packets, but
there are tons of reasons to raise interrupt, not just NIC.

> Would you mind elaborating on why going above 4k would mean "newish features" and what are they?

https://www.rfc-editor.org/rfc/rfc8654.html

> So here CSR1kv is initiating the connection to XR box advertising MSS 8936 (as expected).
> However, peer MSS is 1240, which is not quite expected, considering XR config:

But it was [8936, 1240].min - so it was 'negotiated' here to the
smallest? If you change the 8936 end to 1239, then that will be used,
regardless who starts it.

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
> https://www.rfc-editor.org/rfc/rfc8654.html

Thanks!

> But it was [8936, 1240].min - so it was 'negotiated' here to the
> smallest? If you change the 8936 end to 1239, then that will be used,
> regardless who starts it.

Yes, but why would XR advertise 1240 if I'm using 'tcp mss 8936' for that
neighbor?
Initially I thought that the whole point of this command is to set MSS to a
fixed value of our choice.

Kind regards,
Marcin


czw., 8 gru 2022 o 10:31 Saku Ytti <saku@ytti.fi> napisa?(a):

> On Thu, 8 Dec 2022 at 11:25, Marcin Kurek <md.kurek@gmail.com> wrote:
>
> > Interesting, but why would 'sh run' or 'write' raise an interrupt?
> > Isn't this a branch in code that handles the CLI?
>
> Maybe to access NVRAM? I don't know for sure, I didn't pursue it, I
> just know I could observe it.
>
> > I'm not sure if I'm reading it right - on the one hand, the interrupts
> are disabled, but on the other hand, some CLI commands actually raise them?
>
> I don't know if IOS does polling or interrupt for NIC packets, but
> there are tons of reasons to raise interrupt, not just NIC.
>
> > Would you mind elaborating on why going above 4k would mean "newish
> features" and what are they?
>
> https://www.rfc-editor.org/rfc/rfc8654.html
>
> > So here CSR1kv is initiating the connection to XR box advertising MSS
> 8936 (as expected).
> > However, peer MSS is 1240, which is not quite expected, considering XR
> config:
>
> But it was [8936, 1240].min - so it was 'negotiated' here to the
> smallest? If you change the 8936 end to 1239, then that will be used,
> regardless who starts it.
>
> --
> ++ytti
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
On Thu, 8 Dec 2022 at 11:40, Marcin Kurek <md.kurek@gmail.com> wrote:
>
> > https://www.rfc-editor.org/rfc/rfc8654.html
>
> Thanks!
>
> > But it was [8936, 1240].min - so it was 'negotiated' here to the
> > smallest? If you change the 8936 end to 1239, then that will be used,
> > regardless who starts it.
>
> Yes, but why would XR advertise 1240 if I'm using 'tcp mss 8936' for that neighbor?
> Initially I thought that the whole point of this command is to set MSS to a fixed value of our choice.

I may have misunderstood you. Ddi you have 8936 configured on both
ends? I thought you had only 8936 on the CSR.

How I understood it:

*Dec 8 11:17:15.453: TCP0: Connection to 12.0.0.7:179, advertising MSS 8936
*Dec 8 11:17:15.456: TCP: tcb 7FFB9A6D64C0 connection to
12.0.0.7:179, peer MSS 1240, MSS is 1240

Local 12.0.0.13 CSR is advertising 8936 to remote 12.0.0.7
Remote 12.0.0.7 is advertising 1240
We settle to 1240

I guess you are saying the remote 12.0.0.7 is as well configured to
8936? Then I agree, I wouldn't expect that, and can't explain it.
Almost sounds like a bug where the configuration command is only read
when IOS-XR establishes the connection, but not when it receives it?

--
++ytti
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
Marcin,

I did some benchmarking back in ~2008-2009 (with 12.2.31SB and 12.2SR,
who still remembers that?), and we (Cisco) fixed numerous non optimized
paths in the routing code. But that was over a decade ago, so I’ll
limit bragging about it ;) There are two presentations, quoted from
about that time I can find some references to, which show in general
terms what improvements you can expect. XR code was however way
more optimized right from the start, and specially over last 5 years
there was a lot additional work done to optimize it.

Anyway - Cisco BGP stack is still limited to 4kB message for BGP (as
far as I know), so we still don’t implement RFC 8654 neither in XR or XE.
I can check that with developers, but last time I’ve heard it’s only on
roadmap.

In the end you’ll get about 2.5-3x improvement using MTU 9k, but the actual
BGP messages inside will still be limited to 4k. Back in the day, in my
testbed, that made 7200VXR with NPE-G1 converge in about 21 seconds versus
a minute with ~600k prefixes and 100 sessions.

Most of the benefits are coming BTW from the fact its lesser amount of
packets in BGP communication (to Ytti point) as for real-world BGP table,
the packing of attributes and updates with 4k vs 1.3-1.5k is better,
but only by so much. Again, the old numbers I’m quoting are back from the
day when IOS was monolithic, now with CSR you’re using for testing, IOS-XE
should handle that way more smoothly even under stress.

Running 9k MTU is anyway recommended for BGP RR scenarios, as many core SP
networks (and some Enterprises) run 9k MTU in the core anyway. You’ll get
less bursts for initial convergence and then ongoing session maintenance
thanks to less packets. Some very old recommendations (again, remember
ISP Best Practices guide v2.9?) about interface hold buffers and SPD tuning
will no longer apply obviously.

--
./
Re: MTU and PMTUD [ In reply to ]
> I may have misunderstood you. Ddi you have 8936 configured on both
> ends? I thought you had only 8936 on the CSR.
>
> How I understood it:
>
> *Dec 8 11:17:15.453: TCP0: Connection to 12.0.0.7:179, advertising MSS
8936
> *Dec 8 11:17:15.456: TCP: tcb 7FFB9A6D64C0 connection to
> 12.0.0.7:179, peer MSS 1240, MSS is 1240
>
> Local 12.0.0.13 CSR is advertising 8936 to remote 12.0.0.7
> Remote 12.0.0.7 is advertising 1240
> We settle to 1240

> I guess you are saying the remote 12.0.0.7 is as well configured to
> 8936?

Exactly. This is the configuration I have on 12.0.0.7:

neighbor 12.0.0.13
remote-as 12
tcp mss 8936
update-source Loopback0
address-family vpnv4 unicast
!
address-family ipv4 rt-filter
!
address-family l2vpn evpn

So my understanding *was* that this command is enforcing MSS=8936, no
matter if it's SYN or SYN ACK.

> Then I agree, I wouldn't expect that, and can't explain it.
> Almost sounds like a bug where the configuration command is only read
> when IOS-XR establishes the connection, but not when it receives it?

That's my thought as well. But this is strange as I have 7.5.2 and 6.7.3 in
the lab and the behavior is the same.

I guess it's time to ask TAC friends :)

Thanks!

Kind regards,
Marcin

czw., 8 gru 2022 o 10:50 Saku Ytti <saku@ytti.fi> napisa?(a):

> On Thu, 8 Dec 2022 at 11:40, Marcin Kurek <md.kurek@gmail.com> wrote:
> >
> > > https://www.rfc-editor.org/rfc/rfc8654.html
> >
> > Thanks!
> >
> > > But it was [8936, 1240].min - so it was 'negotiated' here to the
> > > smallest? If you change the 8936 end to 1239, then that will be used,
> > > regardless who starts it.
> >
> > Yes, but why would XR advertise 1240 if I'm using 'tcp mss 8936' for
> that neighbor?
> > Initially I thought that the whole point of this command is to set MSS
> to a fixed value of our choice.
>
> I may have misunderstood you. Ddi you have 8936 configured on both
> ends? I thought you had only 8936 on the CSR.
>
> How I understood it:
>
> *Dec 8 11:17:15.453: TCP0: Connection to 12.0.0.7:179, advertising MSS
> 8936
> *Dec 8 11:17:15.456: TCP: tcb 7FFB9A6D64C0 connection to
> 12.0.0.7:179, peer MSS 1240, MSS is 1240
>
> Local 12.0.0.13 CSR is advertising 8936 to remote 12.0.0.7
> Remote 12.0.0.7 is advertising 1240
> We settle to 1240
>
> I guess you are saying the remote 12.0.0.7 is as well configured to
> 8936? Then I agree, I wouldn't expect that, and can't explain it.
> Almost sounds like a bug where the configuration command is only read
> when IOS-XR establishes the connection, but not when it receives it?
>
> --
> ++ytti
>
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
Marcin Kurek via cisco-nsp wrote on 08/12/2022 09:25:
> Interesting, but why would 'sh run' or 'write' raise an interrupt?
> Isn't this a branch in code that handles the CLI?

this was monolithic IOS running on older platforms. On a non-preemptive
multitasking operating system like this, interrupts will interrupt. So
once you have an interrupt, you need to wait until the interrupt handler
is finished what it's doing before the kernel can continue with anything
else. If you have interrupt-driven flash i/o writes, with a CPU based
forwarding plane - which is what you had on everything from AGS to 7200
- then packet forwarding is going to be impacted.

The effect was very obvious if you issued a flash write on an older
device (e.g. C2500), while pinging the device. The ping response times
would increase up to 200ms. Obviously packet forwarding is handled with
much higher priority than ping, but it will be affected all the same.

Nick
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: MTU and PMTUD [ In reply to ]
> I did some benchmarking back in ~2008-2009 (with 12.2.31SB and 12.2SR,
> who still remembers that?), and we (Cisco) fixed numerous non optimized
> paths in the routing code. But that was over a decade ago, so I’ll
> limit bragging about it ;) There are two presentations, quoted from
> about that time I can find some references to, which show in general
> terms what improvements you can expect. XR code was however way
> more optimized right from the start, and specially over last 5 years
> there was a lot additional work done to optimize it.
>
> <SNIP>

Hi Lukasz,

Thanks for sharing this interesting information.

I've done a bit of testing and there is some performance gain, but not
as much as I initially expected.

Still, 9k MTU is the right way to go, because as you pointed out, most
networks run 9k in the core anyway.


Just to update this thread - I've consulted with TAC about the MSS
"negotiation" where XR is passive.
It is a known issue, and currently, there is no way to fix it, because
it would require complex changes to how TCP listening sockets are
programmed.

However, as a workaround, we can configure the remote peer as passive to
make sure that XR is always initiating the TCP connection.


Kind regards,
Marcin





_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/