Mailing List Archive

IOS-XR has duplicate update suppression logic for EBGP sessions,
not for IBGP sessions.

If you are using EBGP and seeing a fault in the duplicate update
suppression logic in IOS-XR, please let me know configs and details
of the experiment.

Regards,
Jakob.

-----Original Message-----
Date: Thu, 15 Oct 2020 18:35:58 -0700
From: Thomas Krenc <tkrenc@nps.edu>

Dear NANOG,

As a team of researchers from NPS and TU Berlin, we are investigating
the impact of BGP community attributes on the update behavior between ASes.

We find that when a route is associated with multiple distinct community
attributes it does not only lead to multiple announcement at the tagging
AS, but also at neighboring ASes, if communities are not filtered
properly. This behavior is wide-spread.

In order to better understand our observations, we have performed a
series of laboratory experiments using Cisco IOS, Junos OS, as well as
the BIRD daemon.

We find that - by default - all tested routers generate announcements
with changing community attributes, even when other attributes do not
change. In addition, when communities are filtered at egress, Cisco und
BIRD send duplicate announcements (Juniper does not).

Since our findings are limited to observations in public data as well as
few router implementations, we would like to share our research and
kindly ask you to have a look at:

??? https://www.cmand.org/communityexploration/

There, we provide some resources documenting our research, as well as
open questions. We greatly appreciate any feedback and insights you can
offer. Also, please don't hesitate to contact us directly:

??? communityexploration AT cmand DOT org

best regards

Thomas Krenc
Postdoctoral Researcher
Naval Postgraduate School

> IOS-XR has duplicate update suppression logic for EBGP sessions

as, i believe, do most all implementations, to protect best path
computation costs.

randy

On 18.10.20 00:59, Jakob Heitz (jheitz) via NANOG wrote:
> IOS-XR has duplicate update suppression logic for EBGP sessions,
> not for IBGP sessions.

Does this feature hinder incoming duplicates from triggering best path
selection or does it stop the export of duplicates?

Cheers,
Clemens

This feature suppresses outgoing duplicates. Another feature ignores incoming duplicates from any BGP session.

Regards,
Jakob.


> On Oct 18, 2020, at 1:46 AM, Clemens Mosig <clemens.mosig@fu-berlin.de> wrote:
>
> ?On 18.10.20 00:59, Jakob Heitz (jheitz) via NANOG wrote:
>> IOS-XR has duplicate update suppression logic for EBGP sessions,
>> not for IBGP sessions.
>
> Does this feature hinder incoming duplicates from triggering best path selection or does it stop the export of duplicates?
>
> Cheers,
> Clemens

Hi Jakob.

The simple configuration below allows communities to be forwarded
(send-community-ebgp), but are cleaned at egress (using route-policy and
community-set).

In the experiment, the router receives announcements with altering
community attributes only, from the internal peer. After the filter is
applied, the router sends duplicates to the external peer.

Also, In a slightly different setup, the router sends duplicates due to
changes in the next-hop only.

best regards
Thomas

---

RP/0/0/CPU0:ios(config)#show running-config
Tue Oct 20 02:56:24.230 UTC
Building configuration...
!! IOS XR Configuration 6.0.1
!! Last configuration change at Tue Oct 20 02:56:02 2020 by cisco
!
interface MgmtEth0/0/CPU0/0
 shutdown
!
interface GigabitEthernet0/0/0/0
 ipv4 address 10.12.0.2 255.255.255.252
!
interface GigabitEthernet0/0/0/1
 ipv4 address 10.20.0.1 255.255.255.252
!
community-set all
  *:*
end-set
!
route-policy nofilter
  pass
end-policy
!
route-policy egressfilter
  delete community in all
  pass
end-policy
!
router bgp 65002
 bgp router-id 10.12.0.2
 address-family ipv4 unicast
!
 neighbor 10.12.0.1
  remote-as 65001
  address-family ipv4 unicast
   send-community-ebgp
   route-policy egressfilter out
!
 neighbor 10.20.0.2
  remote-as 65002
  address-family ipv4 unicast
!
end

On 10/17/20 3:59 PM, Jakob Heitz (jheitz) via NANOG wrote:
> IOS-XR has duplicate update suppression logic for EBGP sessions,
> not for IBGP sessions.
>
> If you are using EBGP and seeing a fault in the duplicate update
> suppression logic in IOS-XR, please let me know configs and details
> of the experiment.
>
> Regards,
> Jakob.
>
> -----Original Message-----
> Date: Thu, 15 Oct 2020 18:35:58 -0700
> From: Thomas Krenc <tkrenc@nps.edu>
>
> Dear NANOG,
>
> As a team of researchers from NPS and TU Berlin, we are investigating
> the impact of BGP community attributes on the update behavior between ASes.
>
> We find that when a route is associated with multiple distinct community
> attributes it does not only lead to multiple announcement at the tagging
> AS, but also at neighboring ASes, if communities are not filtered
> properly. This behavior is wide-spread.
>
> In order to better understand our observations, we have performed a
> series of laboratory experiments using Cisco IOS, Junos OS, as well as
> the BIRD daemon.
>
> We find that - by default - all tested routers generate announcements
> with changing community attributes, even when other attributes do not
> change. In addition, when communities are filtered at egress, Cisco und
> BIRD send duplicate announcements (Juniper does not).
>
> Since our findings are limited to observations in public data as well as
> few router implementations, we would like to share our research and
> kindly ask you to have a look at:
>
> ??? https://www.cmand.org/communityexploration/
>
> There, we provide some resources documenting our research, as well as
> open questions. We greatly appreciate any feedback and insights you can
> offer. Also, please don't hesitate to contact us directly:
>
> ??? communityexploration AT cmand DOT org
>
> best regards
>
> Thomas Krenc
> Postdoctoral Researcher
> Naval Postgraduate School

Thomas,

I confirmed your case and took a look at the code.
The outbound duplicate suppression function tries to detect
duplicates without actually storing or recreating the
previously sent update, so it misses some cases.

Your use case is a good one. We will check to see if we can
detect it without compromising significantly on resource usage.
Thank you for raising the issue.

Regards,
Jakob.

-----Original Message-----
Date: Tue, 20 Oct 2020 04:48:37 -0700
From: Thomas Krenc <tkrenc@nps.edu>

Hi Jakob.

The simple configuration below allows communities to be forwarded
(send-community-ebgp), but are cleaned at egress (using route-policy and
community-set).

In the experiment, the router receives announcements with altering
community attributes only, from the internal peer. After the filter is
applied, the router sends duplicates to the external peer.

Also, In a slightly different setup, the router sends duplicates due to
changes in the next-hop only.

best regards
Thomas

---

RP/0/0/CPU0:ios(config)#show running-config
Tue Oct 20 02:56:24.230 UTC
Building configuration...
!! IOS XR Configuration 6.0.1
!! Last configuration change at Tue Oct 20 02:56:02 2020 by cisco
!
interface MgmtEth0/0/CPU0/0
?shutdown
!
interface GigabitEthernet0/0/0/0
?ipv4 address 10.12.0.2 255.255.255.252
!
interface GigabitEthernet0/0/0/1
?ipv4 address 10.20.0.1 255.255.255.252
!
community-set all
? *:*
end-set
!
route-policy nofilter
? pass
end-policy
!
route-policy egressfilter
? delete community in all
? pass
end-policy
!
router bgp 65002
?bgp router-id 10.12.0.2
?address-family ipv4 unicast
!
?neighbor 10.12.0.1
? remote-as 65001
? address-family ipv4 unicast
?? send-community-ebgp
?? route-policy egressfilter out
!
?neighbor 10.20.0.2
? remote-as 65002
? address-family ipv4 unicast
!
end

On 10/17/20 3:59 PM, Jakob Heitz (jheitz) via NANOG wrote:
> IOS-XR has duplicate update suppression logic for EBGP sessions,
> not for IBGP sessions.
>
> If you are using EBGP and seeing a fault in the duplicate update
> suppression logic in IOS-XR, please let me know configs and details
> of the experiment.
>
> Regards,
> Jakob.
>
> -----Original Message-----
> Date: Thu, 15 Oct 2020 18:35:58 -0700
> From: Thomas Krenc <tkrenc@nps.edu>
>
> Dear NANOG,
>
> As a team of researchers from NPS and TU Berlin, we are investigating
> the impact of BGP community attributes on the update behavior between ASes.
>
> We find that when a route is associated with multiple distinct community
> attributes it does not only lead to multiple announcement at the tagging
> AS, but also at neighboring ASes, if communities are not filtered
> properly. This behavior is wide-spread.
>
> In order to better understand our observations, we have performed a
> series of laboratory experiments using Cisco IOS, Junos OS, as well as
> the BIRD daemon.
>
> We find that - by default - all tested routers generate announcements
> with changing community attributes, even when other attributes do not
> change. In addition, when communities are filtered at egress, Cisco und
> BIRD send duplicate announcements (Juniper does not).
>
> Since our findings are limited to observations in public data as well as
> few router implementations, we would like to share our research and
> kindly ask you to have a look at:
>
> ??? https://www.cmand.org/communityexploration/
>
> There, we provide some resources documenting our research, as well as
> open questions. We greatly appreciate any feedback and insights you can
> offer. Also, please don't hesitate to contact us directly:
>
> ??? communityexploration AT cmand DOT org
>
> best regards
>
> Thomas Krenc
> Postdoctoral Researcher
> Naval Postgraduate School

Hi Thomas,

We had a similar discussion on FRR slack, there are some duplicates indeed.
Are you planing to test FRR at some point in time?

Cheers,
Jeff
On Oct 21, 2020, 3:58 PM -0700, Jakob Heitz (jheitz) via NANOG <nanog@nanog.org>, wrote:
> Thomas,
>
> I confirmed your case and took a look at the code.
> The outbound duplicate suppression function tries to detect
> duplicates without actually storing or recreating the
> previously sent update, so it misses some cases.
>
> Your use case is a good one. We will check to see if we can
> detect it without compromising significantly on resource usage.
> Thank you for raising the issue.
>
> Regards,
> Jakob.
>
> -----Original Message-----
> Date: Tue, 20 Oct 2020 04:48:37 -0700
> From: Thomas Krenc <tkrenc@nps.edu>
>
> Hi Jakob.
>
> The simple configuration below allows communities to be forwarded
> (send-community-ebgp), but are cleaned at egress (using route-policy and
> community-set).
>
> In the experiment, the router receives announcements with altering
> community attributes only, from the internal peer. After the filter is
> applied, the router sends duplicates to the external peer.
>
> Also, In a slightly different setup, the router sends duplicates due to
> changes in the next-hop only.
>
> best regards
> Thomas
>
> ---
>
> RP/0/0/CPU0:ios(config)#show running-config
> Tue Oct 20 02:56:24.230 UTC
> Building configuration...
> !! IOS XR Configuration 6.0.1
> !! Last configuration change at Tue Oct 20 02:56:02 2020 by cisco
> !
> interface MgmtEth0/0/CPU0/0
> ?shutdown
> !
> interface GigabitEthernet0/0/0/0
> ?ipv4 address 10.12.0.2 255.255.255.252
> !
> interface GigabitEthernet0/0/0/1
> ?ipv4 address 10.20.0.1 255.255.255.252
> !
> community-set all
> ? *:*
> end-set
> !
> route-policy nofilter
> ? pass
> end-policy
> !
> route-policy egressfilter
> ? delete community in all
> ? pass
> end-policy
> !
> router bgp 65002
> ?bgp router-id 10.12.0.2
> ?address-family ipv4 unicast
> !
> ?neighbor 10.12.0.1
> ? remote-as 65001
> ? address-family ipv4 unicast
> ?? send-community-ebgp
> ?? route-policy egressfilter out
> !
> ?neighbor 10.20.0.2
> ? remote-as 65002
> ? address-family ipv4 unicast
> !
> end
>
> On 10/17/20 3:59 PM, Jakob Heitz (jheitz) via NANOG wrote:
> > IOS-XR has duplicate update suppression logic for EBGP sessions,
> > not for IBGP sessions.
> >
> > If you are using EBGP and seeing a fault in the duplicate update
> > suppression logic in IOS-XR, please let me know configs and details
> > of the experiment.
> >
> > Regards,
> > Jakob.
> >
> > -----Original Message-----
> > Date: Thu, 15 Oct 2020 18:35:58 -0700
> > From: Thomas Krenc <tkrenc@nps.edu>
> >
> > Dear NANOG,
> >
> > As a team of researchers from NPS and TU Berlin, we are investigating
> > the impact of BGP community attributes on the update behavior between ASes.
> >
> > We find that when a route is associated with multiple distinct community
> > attributes it does not only lead to multiple announcement at the tagging
> > AS, but also at neighboring ASes, if communities are not filtered
> > properly. This behavior is wide-spread.
> >
> > In order to better understand our observations, we have performed a
> > series of laboratory experiments using Cisco IOS, Junos OS, as well as
> > the BIRD daemon.
> >
> > We find that - by default - all tested routers generate announcements
> > with changing community attributes, even when other attributes do not
> > change. In addition, when communities are filtered at egress, Cisco und
> > BIRD send duplicate announcements (Juniper does not).
> >
> > Since our findings are limited to observations in public data as well as
> > few router implementations, we would like to share our research and
> > kindly ask you to have a look at:
> >
> > ??? https://www.cmand.org/communityexploration/
> >
> > There, we provide some resources documenting our research, as well as
> > open questions. We greatly appreciate any feedback and insights you can
> > offer. Also, please don't hesitate to contact us directly:
> >
> > ??? communityexploration AT cmand DOT org
> >
> > best regards
> >
> > Thomas Krenc
> > Postdoctoral Researcher
> > Naval Postgraduate School

Hi Jeff,

We have tested FRR (v6.0.2) indeed and found that duplicates are not
suppressed by default.

We will publish more detailed results and configurations on the website
soon.

Thomas

On 10/21/20 4:35 PM, Jeff Tantsura wrote:
>
> Hi Thomas,
>
> We had a similar discussion on FRR slack, there are some duplicates
> indeed.
> Are you planing to test FRR at some point in time?
>
> Cheers,
> Jeff
> On Oct 21, 2020, 3:58 PM -0700, Jakob Heitz (jheitz) via NANOG
> <nanog@nanog.org>, wrote:
>> Thomas,
>>
>> I confirmed your case and took a look at the code.
>> The outbound duplicate suppression function tries to detect
>> duplicates without actually storing or recreating the
>> previously sent update, so it misses some cases.
>>
>> Your use case is a good one. We will check to see if we can
>> detect it without compromising significantly on resource usage.
>> Thank you for raising the issue.
>>
>> Regards,
>> Jakob.
>>
>> -----Original Message-----
>> Date: Tue, 20 Oct 2020 04:48:37 -0700
>> From: Thomas Krenc <tkrenc@nps.edu>
>>
>> Hi Jakob.
>>
>> The simple configuration below allows communities to be forwarded
>> (send-community-ebgp), but are cleaned at egress (using route-policy and
>> community-set).
>>
>> In the experiment, the router receives announcements with altering
>> community attributes only, from the internal peer. After the filter is
>> applied, the router sends duplicates to the external peer.
>>
>> Also, In a slightly different setup, the router sends duplicates due to
>> changes in the next-hop only.
>>
>> best regards
>> Thomas
>>
>> ---
>>
>> RP/0/0/CPU0:ios(config)#show running-config
>> Tue Oct 20 02:56:24.230 UTC
>> Building configuration...
>> !! IOS XR Configuration 6.0.1
>> !! Last configuration change at Tue Oct 20 02:56:02 2020 by cisco
>> !
>> interface MgmtEth0/0/CPU0/0
>> ?shutdown
>> !
>> interface GigabitEthernet0/0/0/0
>> ?ipv4 address 10.12.0.2 255.255.255.252
>> !
>> interface GigabitEthernet0/0/0/1
>> ?ipv4 address 10.20.0.1 255.255.255.252
>> !
>> community-set all
>> ? *:*
>> end-set
>> !
>> route-policy nofilter
>> ? pass
>> end-policy
>> !
>> route-policy egressfilter
>> ? delete community in all
>> ? pass
>> end-policy
>> !
>> router bgp 65002
>> ?bgp router-id 10.12.0.2
>> ?address-family ipv4 unicast
>> !
>> ?neighbor 10.12.0.1
>> ? remote-as 65001
>> ? address-family ipv4 unicast
>> ?? send-community-ebgp
>> ?? route-policy egressfilter out
>> !
>> ?neighbor 10.20.0.2
>> ? remote-as 65002
>> ? address-family ipv4 unicast
>> !
>> end
>>
>> On 10/17/20 3:59 PM, Jakob Heitz (jheitz) via NANOG wrote:
>>> IOS-XR has duplicate update suppression logic for EBGP sessions,
>>> not for IBGP sessions.
>>>
>>> If you are using EBGP and seeing a fault in the duplicate update
>>> suppression logic in IOS-XR, please let me know configs and details
>>> of the experiment.
>>>
>>> Regards,
>>> Jakob.
>>>
>>> -----Original Message-----
>>> Date: Thu, 15 Oct 2020 18:35:58 -0700
>>> From: Thomas Krenc <tkrenc@nps.edu>
>>>
>>> Dear NANOG,
>>>
>>> As a team of researchers from NPS and TU Berlin, we are investigating
>>> the impact of BGP community attributes on the update behavior
>>> between ASes.
>>>
>>> We find that when a route is associated with multiple distinct community
>>> attributes it does not only lead to multiple announcement at the tagging
>>> AS, but also at neighboring ASes, if communities are not filtered
>>> properly. This behavior is wide-spread.
>>>
>>> In order to better understand our observations, we have performed a
>>> series of laboratory experiments using Cisco IOS, Junos OS, as well as
>>> the BIRD daemon.
>>>
>>> We find that - by default - all tested routers generate announcements
>>> with changing community attributes, even when other attributes do not
>>> change. In addition, when communities are filtered at egress, Cisco und
>>> BIRD send duplicate announcements (Juniper does not).
>>>
>>> Since our findings are limited to observations in public data as well as
>>> few router implementations, we would like to share our research and
>>> kindly ask you to have a look at:
>>>
>>> ??? https://www.cmand.org/communityexploration/
>>>
>>> There, we provide some resources documenting our research, as well as
>>> open questions. We greatly appreciate any feedback and insights you can
>>> offer. Also, please don't hesitate to contact us directly:
>>>
>>> ??? communityexploration AT cmand DOT org
>>>
>>> best regards
>>>
>>> Thomas Krenc
>>> Postdoctoral Researcher
>>> Naval Postgraduate School