Mailing List Archive

How to disable ILMI/SNMP CSCvs33325
Recently Shodan has been showing how it probes all our IOS-XE routers
via SNMP even though we have an ACL on all our SNMP.  We then found that
there is a bugid on the issue (ILMI can't be blocked by ACL):
CSCvs33325
As well as an internal TAC bugid:
CSCdp11863

Basically, none of the commands offered by these bugids or via the TAC
case we opened have worked to block ILMI.  So we tried to use
control-plane blocking as we do on our IOS-XR routers, but we have not
managed to get that to work.

Does anyone have an actual tried and working solution to blocking ILMI
on IOS-XE?  control-plane or other command?

Thanks,
Hank

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: How to disable ILMI/SNMP CSCvs33325 [ In reply to ]
HI,

On Mon, Sep 19, 2022 at 02:29:06PM +0300, Hank Nussbacher via cisco-nsp wrote:
> Recently Shodan has been showing how it probes all our IOS-XE routers
> via SNMP even though we have an ACL on all our SNMP.? We then found that
> there is a bugid on the issue (ILMI can't be blocked by ACL):
> CSCvs33325

Is that still a thing? Insane.

It used to be an issue on IOS 15+ years ago... (on IOS, the issue was
"ILMI is a predefined community which cannot be deleted" - but you
*could* expose it, make it explicit, and then put an ACL on it).


That bug is amazing anyway. My suggestion would have been "escalate via
PSIRT", but the bug says "The Cisco PSIRT has evaluated this issue and
determined it does not meet the criteria for PSIRT ownership or involvement.
This issue will be addressed via normal resolution channels."

WAT?!


That said, I tried to reproduce it on our boxes, and neither the ASR920
nor the lone ASR1000 reponds to SNMP v1 or v2c queries with community
"ILMI", with nothing in the config to block it (same source host can
query with one of the configured SNMP communities). This is on IOS XE
16.6.10 and 15.5(3)S10 respectively. Seems you need something extra.

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: How to disable ILMI/SNMP CSCvs33325 [ In reply to ]
On 19/09/2022 15:40, Gert Doering wrote:
> HI,
>
> On Mon, Sep 19, 2022 at 02:29:06PM +0300, Hank Nussbacher via cisco-nsp wrote:
>> Recently Shodan has been showing how it probes all our IOS-XE routers
>> via SNMP even though we have an ACL on all our SNMP.  We then found that
>> there is a bugid on the issue (ILMI can't be blocked by ACL):
>> CSCvs33325
>
> Is that still a thing? Insane.

Indeed.

>
> It used to be an issue on IOS 15+ years ago... (on IOS, the issue was
> "ILMI is a predefined community which cannot be deleted" - but you
> *could* expose it, make it explicit, and then put an ACL on it).
>
>
> That bug is amazing anyway. My suggestion would have been "escalate via
> PSIRT", but the bug says "The Cisco PSIRT has evaluated this issue and
> determined it does not meet the criteria for PSIRT ownership or involvement.
> This issue will be addressed via normal resolution channels."
>
> WAT?!
>
>
> That said, I tried to reproduce it on our boxes, and neither the ASR920
> nor the lone ASR1000 reponds to SNMP v1 or v2c queries with community
> "ILMI", with nothing in the config to block it (same source host can
> query with one of the configured SNMP communities). This is on IOS XE
> 16.6.10 and 15.5(3)S10 respectively. Seems you need something extra.

It is V3. Here is a Shodan snippet from one of dozens of alerts we get
per day:

Banner (snmp_v3)
Snmp:
Versions:
3
Engineid Format: mac
Engine Boots: 20
Engineid Data: 70:ca:9b:a9:2f:40
Enterprise: 9
Engine Time: 189 days, 9:15:11


-Hank

>
> gert

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: How to disable ILMI/SNMP CSCvs33325 [ In reply to ]
Hi,

On Mon, Sep 19, 2022 at 03:47:09PM +0300, Hank Nussbacher via cisco-nsp wrote:
> On 19/09/2022 15:40, Gert Doering wrote:
> > On Mon, Sep 19, 2022 at 02:29:06PM +0300, Hank Nussbacher via cisco-nsp wrote:
> >> Recently Shodan has been showing how it probes all our IOS-XE routers
> >> via SNMP even though we have an ACL on all our SNMP.? We then found that
> >> there is a bugid on the issue (ILMI can't be blocked by ACL):
> >> CSCvs33325
> >
> > Is that still a thing? Insane.
> Indeed.

Just for reference, here's the 2001 bug. With full PSIRT "get free
software upgrade" parts...

https://www.cisco.com/c/dam/en/us/support/docs/csa/cisco-sa-20010227-ios-snmp-ilmi.html

[..]
> > That said, I tried to reproduce it on our boxes, and neither the ASR920
> > nor the lone ASR1000 reponds to SNMP v1 or v2c queries with community
> > "ILMI", with nothing in the config to block it (same source host can
> > query with one of the configured SNMP communities). This is on IOS XE
> > 16.6.10 and 15.5(3)S10 respectively. Seems you need something extra.
>
> It is V3. Here is a Shodan snippet from one of dozens of alerts we get
> per day:

Good to know. Looking at shodan, I see that both types of devices here
are listed as well (ewww!).

So, need to figure out what the magic -v3 incantation of snmpget is
to make this work... (every time I tried v3 so far has led to
"more grey hair").

thanks for the heads up

gert


--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: How to disable ILMI/SNMP CSCvs33325 [ In reply to ]
Gert Doering via cisco-nsp writes:
> Hi,
> On Mon, Sep 19, 2022 at 03:47:09PM +0300, Hank Nussbacher via cisco-nsp wrote:
>> On 19/09/2022 15:40, Gert Doering wrote:
> https://www.cisco.com/c/dam/en/us/support/docs/csa/cisco-sa-20010227-ios-snmp-ilmi.html

> [..]
>> > That said, I tried to reproduce it on our boxes, and neither the ASR920
>> > nor the lone ASR1000 reponds to SNMP v1 or v2c queries with community
>> > "ILMI", with nothing in the config to block it (same source host can
>> > query with one of the configured SNMP communities). This is on IOS XE
>> > 16.6.10 and 15.5(3)S10 respectively. Seems you need something extra.
>>
>> It is V3. Here is a Shodan snippet from one of dozens of alerts we get
>> per day:

> Good to know. Looking at shodan, I see that both types of devices here
> are listed as well (ewww!).

> So, need to figure out what the magic -v3 incantation of snmpget is
> to make this work... (every time I tried v3 so far has led to
> "more grey hair").

Yeah, I'd like to reproduce/understand that too. I actually remember
both ILMI (in ATM, sigh) and SNMPv3. One of SNMPv3's distinguishing
features is that it DOESN'T use community strings anymore. So I'm a bit
confused as to what the problem is. Is there some implicit mapping from
SNMPv1/2c communities to SNMPv3 usernames/passwords? Or are the Shodan
reports referring to information leaks from SNMPv3 engine-ID discovery?
(e.g. CSCtw74132)

Cheers,
--
Simon.
_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: How to disable ILMI/SNMP CSCvs33325 [ In reply to ]
On 20/09/2022 15:54, Simon Leinen wrote:
> Gert Doering via cisco-nsp writes:
>> Hi,
>> On Mon, Sep 19, 2022 at 03:47:09PM +0300, Hank Nussbacher via cisco-nsp wrote:
>>> On 19/09/2022 15:40, Gert Doering wrote:
>> https://www.cisco.com/c/dam/en/us/support/docs/csa/cisco-sa-20010227-ios-snmp-ilmi.html
>
>> [..]
>>>> That said, I tried to reproduce it on our boxes, and neither the ASR920
>>>> nor the lone ASR1000 reponds to SNMP v1 or v2c queries with community
>>>> "ILMI", with nothing in the config to block it (same source host can
>>>> query with one of the configured SNMP communities). This is on IOS XE
>>>> 16.6.10 and 15.5(3)S10 respectively. Seems you need something extra.
>>>
>>> It is V3. Here is a Shodan snippet from one of dozens of alerts we get
>>> per day:
>
>> Good to know. Looking at shodan, I see that both types of devices here
>> are listed as well (ewww!).
>
>> So, need to figure out what the magic -v3 incantation of snmpget is
>> to make this work... (every time I tried v3 so far has led to
>> "more grey hair").
>
> Yeah, I'd like to reproduce/understand that too. I actually remember
> both ILMI (in ATM, sigh) and SNMPv3. One of SNMPv3's distinguishing
> features is that it DOESN'T use community strings anymore. So I'm a bit
> confused as to what the problem is. Is there some implicit mapping from
> SNMPv1/2c communities to SNMPv3 usernames/passwords? Or are the Shodan
> reports referring to information leaks from SNMPv3 engine-ID discovery?
> (e.g. CSCtw74132)

Indeed the SNMP leaks appear to be exactly CSCtw74132 which we did not
know about nor did Cisco TAC :-(

Good to know the people here are more knowledgeable than Cisco :-)

Regards,
Hank

>
> Cheers,

_______________________________________________
cisco-nsp mailing list cisco-nsp@puck.nether.net
https://puck.nether.net/mailman/listinfo/cisco-nsp
archive at http://puck.nether.net/pipermail/cisco-nsp/
Re: How to disable ILMI/SNMP CSCvs33325 [ In reply to ]
Hi,

On Wed, Sep 21, 2022 at 08:14:30AM +0300, Hank Nussbacher wrote:
> Indeed the SNMP leaks appear to be exactly CSCtw74132 which we did not
> know about nor did Cisco TAC :-(

The more I dive into this, the more I want to return to my bed and
pull the blanket over my head...

So, the Cisco bug ID claims "this has been fixed in some versions",
but none of those are "ASR920 IOS trains" (except 03.9(00)E, which
is sort of weird).

The bug also claims "CVE ID CVE-2012-5719 has been assigned", but
MITRE says "** RESERVED ** This candidate has been reserved by an
organization or individual that will use it when announcing a new
security problem", so it got never published...


That said, I then went to test our Junipers and Aristas, and they
all do the same silly shit - no SNMPv3 configured, strict ACLs for
all configured SNMP communities, and *still* SNMP engine discovery
works from arbitrary sources out there. On the switches it's not
that annoying (management interface is in a well-isolated network
segment) but on the routers, customer-facing IPs are reachable
"from the world".

Sounds like a nice reflection attack in the coming...

*grumble*

gert
--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de
Re: How to disable ILMI/SNMP CSCvs33325 [ In reply to ]
Hi,

so, more on this...

- on ASR9k, SNMPv3 is subject to regular control plane ACLs, so
unless a SNMPv3 sender shows up in

control-plane
management-plane
inband
interface all
allow all peer
address ipv4 1.2.3.4/32
!
allow SNMP peer
address ipv4 3.4.5.6/32

the ASR9k will not reply (I assume that's generic IOS XR). Good.

- on IOS XE, I found something that "seems to do the right thing", as
in, block all SNMPv3 packets, including discovery, while still permitting
SNMPv2

asr920(config)#access-list 99 deny any log
asr920(config)#snmp-server drop report access 99
asr920(config)#do term mon
asr920(config)#
Sep 21 12:25:07: %SEC-6-IPACCESSLOGS: list 99 denied 1.1.27.20 1 packet
Sep 21 12:25:11: %SEC-6-IPACCESSLOGS: list 99 denied 1.1.0.18 1 packet
Sep 21 12:31:03: %SEC-6-IPACCESSLOGS: list 99 denied 1.1.27.20 5 packets
Sep 21 12:31:03: %SEC-6-IPACCESSLOGS: list 99 denied 1.1.0.18 5 packets

(these are the two test hosts that could do SNMP v3 discovery before)

- since we're not using SNMPv3 anywhere, that is good enough for us.

This is on IOS XE 16.06.10.

Older IOS XE and IOS versions have "snmp-server drop unknown-user", but
that still permits discovery.


So maybe the "snmp-server drop report" will at least help Hank... :-)

gert

--
"If was one thing all people took for granted, was conviction that if you
feed honest figures into a computer, honest figures come out. Never doubted
it myself till I met a computer with a sense of humor."
Robert A. Heinlein, The Moon is a Harsh Mistress

Gert Doering - Munich, Germany gert@greenie.muc.de