Mailing List Archive

> On 20210601, at 15:15, Moritz Müller via NANOG <nanog@nanog.org> wrote:
>
> Hi,
>
> DANE for SMTP is not deployed on large scale. Together with researchers from Seoul National University, Virginia Tech and the University of Twente, we would like to understand which challenges operators face when deploying DANE for SMTP.

DNSSEC?

... ;)

No, not even kidding. For many organisations DNSSEC is 'scary' and a burden as it feels 'fragile' for them.

Now, over the last few years this fragility has become less, especially with DNS servers already doing most of the work for you, but people still find it scary, as when DNS breaks (and "it is always DNS", unless it is the network full of packets eh, or broken routes, etc), then you lose all your eggs.

And replacing a DNS key can take a few moments, especially with caching of records etc.
Thus downtime is then ensured.


Combine that with many shops not having much DNS knowledge in the first place, they won't easily get their heads around that barrier.

Hosted offerings (where the shop has 24/7 people just for DNS) are then the only way to go, but then why have an Internet, we could just let everything be done by a single Monopoly and be done with it.


As for solutions: better education, more improvements to the tools & making it easier. CDS records already help a lot. But we might also need to improve recovery mechanisms, as f-ups are made, and you don't want to be off this Internet thing for too long.

Greets,
Jeroen

On 6/2/21 11:07, Jeroen Massar via NANOG wrote:

> As for solutions: better education, more improvements to the tools & making it easier. CDS records already help a lot. But we might also need to improve recovery mechanisms, as f-ups are made, and you don't want to be off this Internet thing for too long.

I think DNSSEC implementation needs to be made less scary for folk who
are apprehensive, and broken down into two steps, where step 1 is most
emphasized:

* Enable DNSSEC on your resolvers. Does not require you to sign your
zones. Does not require you to read up on what it takes to sign and
maintain your zones. Does not require you to worry and test for the
next 60 days whether DNSSEC will break your e-mail delivery, e.t.c.:

             dnssec-enable yes;
             dnssec-validation auto;

        Done! Two lines (BIND, in this case), and off you go.

* Step 2 - take your time cluing up on getting your zone signed, and
being part of the solution toward a more secure Internet. No
pressure, at your pace.

Mark.

Jeroen Massar via NANOG <nanog@nanog.org> writes:

> For many organisations DNSSEC is 'scary' and a burden as it feels
> 'fragile' for them.

For "many"? Can you name one that doesn't feel like that?

https://www.arin.net/vault/announcements/2019/20190204.html
https://www.ripe.net/support/service-announcements/dnssec-validation-problem-with-ripe.net

DNSSEC *is* scary, fragile and a burden. The question is whether the
alternatives are worse.


Bjørn

On 2021-06-02 15:47, Bjørn Mork wrote:
> Jeroen Massar via NANOG <nanog@nanog.org> writes:
>
>> For many organisations DNSSEC is 'scary' and a burden as it feels
>> 'fragile' for them.
>
> For "many"? Can you name one that doesn't feel like that?

Large organisations with 24/7 NOC teams where at least a few folks work
more or less the whole week on keeping DNS up and running....

And as you note, even then, things happen. Noting that ARIN & RIPE NOC
are not that super big, they are rather on the small size... their
problems are just felt really quickly.

Their advantage is that many people will help them out and more
importantly: they have the right contacts.

Greets,
Jeroen

On Wed, Jun 2, 2021 at 8:54 AM Bjørn Mork <bjorn@mork.no> wrote:

> Jeroen Massar via NANOG <nanog@nanog.org> writes:
>
> > For many organisations DNSSEC is 'scary' and a burden as it feels
> > 'fragile' for them.
>
> For "many"? Can you name one that doesn't feel like that?
>
> https://www.arin.net/vault/announcements/2019/20190204.html
>
> https://www.ripe.net/support/service-announcements/dnssec-validation-problem-with-ripe.net
>
> DNSSEC *is* scary, fragile and a burden. The question is whether the
> alternatives are worse.
>

Certainly. It's probably not even difficult. My large organization doesn't
find DNSSEC 'scary' or a burden and didn't even when we started in 2011. It
was simply a technical challenge that needed to be designed and integrated.
We have authoritative DNSSEC signing and recursive DNSSEC validation in
place. Public and internal authoritative DNS is mostly signed and
validation is enforced throughout our multi-layered recursive
infrastructure. A number of other organizations also do DNSSEC and have
been doing it.

I know our Internet email team is aware of the use of DANE for SMTP TLS and
are considering it, but DNSSEC is not a factor at all in their evaluation.
It's a given, just part of the DNS infrastructure they use. And I'm not
aware of any "alternative" that does what DNSSEC does. The only choice is
whether you care about verifiable DNS response integrity or not. As far as
'fragile' goes, I assume that would be implementation dependent. There's no
inherent reason an architecture would be 'fragile'. Or if you mean failure
to properly maintain your zones will break your DNS, that would be true for
any strong integrity assurance mechanism that could possibly be designed.
Integrity assurance means if things are supposed to be secured and cannot
be validated they will not resolve. That's an underlying requirement any
mechanism would have to meet or it's not providing integrity assurance.

Scott

Hello Mark ,

On Wed, 2 Jun 2021, Mark Tinka wrote:
> On 6/2/21 11:07, Jeroen Massar via NANOG wrote:
>
>> As for solutions: better education, more improvements to the tools & making
>> it easier. CDS records already help a lot. But we might also need to
>> improve recovery mechanisms, as f-ups are made, and you don't want to be
>> off this Internet thing for too long.
>
> I think DNSSEC implementation needs to be made less scary for folk who are
> apprehensive, and broken down into two steps, where step 1 is most
> emphasized:
>
> * Enable DNSSEC on your resolvers. Does not require you to sign your
> zones. Does not require you to read up on what it takes to sign and
> maintain your zones. Does not require you to worry and test for the
> next 60 days whether DNSSEC will break your e-mail delivery, e.t.c.:
>
> ??? ??? ???? dnssec-enable yes;
> ???????????? dnssec-validation auto;
>
> ??? ??? Done! Two lines (BIND, in this case), and off you go.

Will this handle the case of self-signed only ?
And as Jeroen Massar mentioned the resignation of a certificate is a tad
troubles some for both DNSSEC & DANE .

> * Step 2 - take your time cluing up on getting your zone signed, and
> being part of the solution toward a more secure Internet. No
> pressure, at your pace.

Again , Will this handle the case of self-signed only ?

> Mark.
Tia , JimL
--
+---------------------------------------------------------------------+
| James W. Laferriere | System Techniques | Give me VMS |
| Network & System Engineer | 3237 Holden Road | Give me Linux |
| jiml@system-techniques.com | Fairbanks, AK. 99709 | only on AXP |
+---------------------------------------------------------------------+

[.
The kicker about DNSSEC is in the dnsviz links, enjoy ;)

TLDR: As long as the very big providers don't demand DNSSEC / DANE, why bother as a small network (just, be prepared to deploy when it starts affecting spam scoring or your search rankings), but small networks do benefit unlike the large providers with direct peerings.
]


> On 20210602, at 16:57, Scott Morizot <tmorizot@gmail.com> wrote:

[..]
> My large organization [..]

> I know our Internet email team [..]

You are hitting it right on the head as I noted in my previous comment: you have multiple (and then possibly even >10) people working on just those separate subjects of DNS and email.

Many many shops do not have that luxury, where the email and DNS person is the same person, maybe there are 2 people, but that is heaven already.

DNSSEC just becomes really scary for smaller teams, or for teams that are small, as it is keys to the kingdom when that fails, and you can't rely on an external helper to then run and help you when you need the help. And the benefits do not really always outweigh the fragility chances...



Now, in reality, it all should not really matter: the email market, where DANE should be prevalently used, is dominated by a few duo-ish-opolies of really large mail providers and then there are a bunch of smaller ones.
And these have those large teams and could do DNSSEC and if they want DANE indeed just fine, you would think.

If those market 'leaders' decide to enforce yet another new standard, and they pick DANE, if you want to either receive or send email to them, things get implemented really quickly everywhere, as you'll have to. (in the same way that we have SPF/DKIM/DMARC/ARC/STARTTTLS/MTA-STS/etc etc)


But lets check the leaders with many great engineers on staff, what they think of this DNSSEC thing (and thus also DANE):

https://dnsviz.net/d/gmail.com/dnssec/ <https://dnsviz.net/d/gmail.com/dnssec/>
https://dnsviz.net/d/google.com/dnssec/ <https://dnsviz.net/d/google.com/dnssec/>
https://dnsviz.net/d/microsoft.com/dnssec/ <https://dnsviz.net/d/microsoft.com/dnssec/> (along with 0x20 errors at my time of test)
https://dnsviz.net/d/outlook.com/dnssec/ <https://dnsviz.net/d/outlook.com/dnssec/>
https://dnsviz.net/d/icloud.com/dnssec/ <https://dnsviz.net/d/icloud.com/dnssec/>
https://dnsviz.net/d/aol.com/dnssec/ <https://dnsviz.net/d/aol.com/dnssec/> (no UDP over IPv6)
https://dnsviz.net/d/facebook.com/dnssec/ <https://dnsviz.net/d/facebook.com/dnssec/>

One outlier:
https://dnsviz.net/d/comcast.com/dnssec/ <https://dnsviz.net/d/comcast.com/dnssec/> (but various RRSIG issues, due to alg 5)

seems your team has to be really big to dare to enable DNSSEC ;)

Though, I would not be surprised if they simply don't care about DNSSEC, as they have direct links to most networks, thus spoofing DNS becomes a rarer option, and that the larger responses are considered to slow things down, thus they won't want to enable it because of performance reasons (gotta get those ads quicker to the eyeball before they dismiss it).


Thus yes, for a smaller network it is likely a good idea to DNSSEC, because your packets will transit multiple links that might change bits, but for the very very big, where you mostly peer directly with most networks, DNSSEC is not really going to bring you much in terms of authentication of data.

And with the move to DoT/DoH and centralised DNS Recursor services, they really don't care about that as TLS solves many (not all, eg auth) of the 'man-in-the-middle' attacks.

But I am also sure that these large networks can simply flip a switch and enable it easily if they really wanted to.

Greets,
Jeroen
(who has the majority of domains under my control DNSSEC signed, but... not all; need to do the DANE part though still)

On 6/3/21 04:53, Jeroen Massar via NANOG wrote:

>  Jeroen
>   (who has the majority of domains under my control DNSSEC signed,
> but... not all; need to do the DANE part though still)

You and me both, on the DANE bit :-).

Mark.

On 6/3/21 00:25, babydr DBA James W. Laferriere wrote:

>
>     Again ,  Will this handle the case of self-signed only ?

Not sure I understand your question, in both cases of recursion and
authoritative.

Mark.

DANE works with self generated CERTs. The TLSA record provides the cryptographic link back to the DNSSEC root.

--
Mark Andrews

> On 3 Jun 2021, at 22:32, babydr DBA James W. Laferriere <babydr@baby-dragons.com> wrote:
>
> ? Hello Mark ,
>
>> On Wed, 2 Jun 2021, Mark Tinka wrote:
>>> On 6/2/21 11:07, Jeroen Massar via NANOG wrote:
>>>
>>> As for solutions: better education, more improvements to the tools & making it easier. CDS records already help a lot. But we might also need to improve recovery mechanisms, as f-ups are made, and you don't want to be off this Internet thing for too long.
>>
>> I think DNSSEC implementation needs to be made less scary for folk who are apprehensive, and broken down into two steps, where step 1 is most emphasized:
>>
>> * Enable DNSSEC on your resolvers. Does not require you to sign your
>> zones. Does not require you to read up on what it takes to sign and
>> maintain your zones. Does not require you to worry and test for the
>> next 60 days whether DNSSEC will break your e-mail delivery, e.t.c.:
>>
>> dnssec-enable yes;
>> dnssec-validation auto;
>>
>> Done! Two lines (BIND, in this case), and off you go.
>
> Will this handle the case of self-signed only ?
> And as Jeroen Massar mentioned the resignation of a certificate is a tad troubles some for both DNSSEC & DANE .
>
>> * Step 2 - take your time cluing up on getting your zone signed, and
>> being part of the solution toward a more secure Internet. No
>> pressure, at your pace.
>
> Again , Will this handle the case of self-signed only ?
>
>> Mark.
> Tia , JimL
> --
> +---------------------------------------------------------------------+
> | James W. Laferriere | System Techniques | Give me VMS |
> | Network & System Engineer | 3237 Holden Road | Give me Linux |
> | jiml@system-techniques.com | Fairbanks, AK. 99709 | only on AXP |
> +---------------------------------------------------------------------+

Hello Mr. Tinka & Mr. Andrews , Please see below .

On Thu, 3 Jun 2021, Mark Tinka wrote:
> On 6/3/21 00:25, babydr DBA James W. Laferriere wrote:

The Below is to keep thread of thought accurate ...
>> On Wed, 2 Jun 2021, Mark Tinka wrote:
>>> * Step 2 - take your time cluing up on getting your zone signed, and
>>> being part of the solution toward a more secure Internet. No
>>> pressure, at your pace.

>>
>> ????Again ,? Will this handle the case of self-signed only ?
>
> Not sure I understand your question, in both cases of recursion and
> authoritative.

The Signing of the 'Zone' , Can the 'Zone' be signed by a self-signed
key ? Or MUST I (and others) rely on a external certificate authority ?

Mind you I notice in rfc6487 (note(s)) about self-signed certificates .
So Maybe I am being a bit over worried about having to spend more money
just to keep my 2 ip-ranges routing in light of the RPKI initative(s) .

Which Mr. Andrews response below answers quite succinctly ,

On Thu, 3 Jun 2021, Mark Andrews wrote:

> DANE works with self generated CERTs. The TLSA record provides the
> cryptographic link back to the DNSSEC root.

Thank You Mr. Andrews , Muchly . Is what I was hoping for .

Thank You Both . JimL
--
+---------------------------------------------------------------------+
| James W. Laferriere | System Techniques | Give me VMS |
| Network & System Engineer | 3237 Holden Road | Give me Linux |
| jiml@system-techniques.com | Fairbanks, AK. 99709 | only on AXP |
+---------------------------------------------------------------------+

On 6/3/21 23:41, babydr DBA James W. Laferriere wrote:

>
>     The Signing of the 'Zone' ,  Can the 'Zone' be signed by a
> self-signed key ?  Or MUST I (and others) rely on a external
> certificate authority ?
>
>     Mind you I notice in rfc6487 (note(s)) about self-signed
> certificates .
>     So Maybe I am being a bit over worried about having to spend more
> money just to keep my 2 ip-ranges routing in light of the RPKI
> initative(s) .
>
>     Which Mr. Andrews response below answers quite succinctly ,

Indeed! Thanks, Mark.

Yeah, it's never been obvious or apparent to me that self-signed keys
for DNSSEC would not be honoured.

My personal zone, as well as my company's one, are both self-signed.
They've both been working reasonably well, so far.

Mark.

Jeroen Massar via NANOG <nanog@nanog.org> writes:

> No, not even kidding. For many organisations DNSSEC is 'scary' and a
> burden as it feels 'fragile' for them.

Unfortunately, yes. And those of us who use it know that this is a
myth. With modern software, DNSSEC is quick and easy to set up, and
works just fine, with no reason for any problems. The effort invested
is a very low price to pay for the added protection, both directly (by
making sure that spoofing attacks &c make resolving fail noticeably),
and through the various added mechanisms you can then apply, such as CAA
records.

> And replacing a DNS key can take a few moments, especially with
> caching of records etc.
> Thus downtime is then ensured.

Not if you do it right. Add the new key, wait a while, then remove the
old key. On installations I manage, this is scripted, and done from
cron, rotating ZSKs on a monthly basis.

> Combine that with many shops not having much DNS knowledge in the
> first place, they won't easily get their heads around that barrier.

Now that's a real problem. If you're going to do X, you should have
someone on staff who knows enough about X to do it right, safely.

-tih
--
Most people who graduate with CS degrees don't understand the significance
of Lisp. Lisp is the most important idea in computer science. --Alan Kay

It appears that Tom Ivar Helbekkmo via NANOG <tih@hamartun.priv.no> said:
>Jeroen Massar via NANOG <nanog@nanog.org> writes:
>
>> No, not even kidding. For many organisations DNSSEC is 'scary' and a
>> burden as it feels 'fragile' for them.
>
>Unfortunately, yes. And those of us who use it know that this is a
>myth. With modern software, DNSSEC is quick and easy to set up, and
>works just fine, with no reason for any problems. ...

I wish that were true. I have signed all 300 zones on my DNS
servers, but only about half of them have working DNSSEC because there
is no practical way to install the DS records. For names that are
registered through my registrar reseller account, it's easy since my
registrar (Tucows) has an API. But for the rest of them that my
users have registered somewhere else, either I have to try and walk
them through the process of uploading the DS data themselves, or
they have to give me their account passwords, neither of which is
workable if you have 100 domains, much less thousands.

I know about CDS, and have tried publishing CDS, but none of my
unsigned domains are at the handful of registries that do CDS
bootstrapping.

I've been grousing about this at the IETF and ICANN for years,
people say yes, that's a problem, and nothing happens.

John Levine <johnl@iecc.com> writes:

> I have signed all 300 zones on my DNS servers, but only about half of
> them have working DNSSEC because there is no practical way to install
> the DS records.

Sounds like ICANN, having told us for a very long time that they want
DNSSEC everywhere, should attempt to get a requirement in place that
registrars have to make it reasonably easy for customers to get those DS
records installed. Certificate authorities are now required to honor
CAA records, which need DNSSEC in place to really make sense, so it
would, IMHO, be natural to follow up like this.

-tih
--
Most people who graduate with CS degrees don't understand the significance
of Lisp. Lisp is the most important idea in computer science. --Alan Kay

It appears that Tom Ivar Helbekkmo via NANOG <tih@hamartun.priv.no> said:
>John Levine <johnl@iecc.com> writes:
>
>> I have signed all 300 zones on my DNS servers, but only about half of
>> them have working DNSSEC because there is no practical way to install
>> the DS records.
>
>Sounds like ICANN, having told us for a very long time that they want
>DNSSEC everywhere, should attempt to get a requirement in place that
>registrars have to make it reasonably easy for customers to get those DS
>records installed. ...

Hahahahaha. At ICANN, anything that might put even the tiniest extra
cost on registrars is a non-starter. Look at the endless fight about
WHOIS redaction.