Mailing List Archive

Keyserver: List of available ones (dynamic)
Am Freitag 24 September 2021 01:00:04 schrieb Andrew Gallagher via
Gnupg-devel:
> > https://github.com/philpennock/sks_spider
> I have updated Phil's code to support Hockeypuck servers, and it is now
> running at https://spider.pgpkeys.eu .

Very cool!

Did you make your code available?

> It also hosts pretty graphs at https://spider.pgpkeys.eu/graphs

In the graphs I can see keyserver.ubuntu.com, but in the list
I cannot. Why is that?

Regards,
Bernhard

--
www.intevation.de/~bernhard   +49 541 33 508 3-3
Intevation GmbH, Osnabrück, DE; Amtsgericht Osnabrück, HRB 18998
Geschäftsführer Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner
Re: Keyserver: List of available ones (dynamic) [ In reply to ]
On 24/09/2021 11:27, Bernhard Reiter wrote:
> Am Freitag 24 September 2021 01:00:04 schrieb Andrew Gallagher via
> Gnupg-devel:
>>> https://github.com/philpennock/sks_spider
>> I have updated Phil's code to support Hockeypuck servers, and it is now
>> running at https://spider.pgpkeys.eu .
>
> Very cool!
>
> Did you make your code available?

Ref this PR:

https://github.com/philpennock/sks_spider/pull/8

>> It also hosts pretty graphs at https://spider.pgpkeys.eu/graphs
>
> In the graphs I can see keyserver.ubuntu.com, but in the list
> I cannot. Why is that?

Well spotted. :-)

The graph code (written in Ruby) polls both port 80 and port 11371,
while the list code (written in Go) only polls 11371. Furthermore,
keyserver.ubuntu.com is not referenced directly by any peer, but
keyserver.syseleven.de is -- it runs a proxy on port 80 (only) that
forwards requests to keyserver.ubuntu.com. This means that the Ruby
spider can find keyserver.ubuntu.com by traversing syseleven port 80,
but the Go spider can't.

Note that the Ruby code records every peer reference no matter what the
error, whereas the Go code records only HTTP errors, and silently
ignores connection failures. The Go code performs URL deduplication by
detecting the hostname that the target server returns. The Ruby code
doesn't; instead I bodged it by hard-coding a list of duplicates by hand
(unresponsive servers don't return hostnames).

In any case, keyserver.ubuntu.com is not syncing because it has no
mutual peers.

--
Andrew Gallagher
Re: Keyserver: List of available ones (dynamic) [ In reply to ]
On 24/09/2021 12:49, Andrew Gallagher wrote:
> On 24/09/2021 11:27, Bernhard Reiter wrote:
>
>> In the graphs I can see keyserver.ubuntu.com, but in the list
>> I cannot. Why is that?
>
> Well spotted. :-)
>
> The graph code (written in Ruby) polls both port 80 and port 11371,
> while the list code (written in Go) only polls 11371. Furthermore,
> keyserver.ubuntu.com is not referenced directly by any peer, but
> keyserver.syseleven.de is -- it runs a proxy on port 80 (only) that
> forwards requests to keyserver.ubuntu.com. This means that the Ruby
> spider can find keyserver.ubuntu.com by traversing syseleven port 80,
> but the Go spider can't.

After investigating the above further, I have now removed the port 80
check entirely from the Ruby graphing code - the only "keyservers" that
listen on port 80 and not 11371 are either proxies/redirectors like
syseleven and pgp.pm, or DNS registrar holding pages. In any case, a
keyserver that does not listen on port 11371 will almost certainly not
sync, so should not be listed as functional.

This change has allowed me to quickly detect some broken mutuals,
exposing some hidden fragilities in the graph (I did/will follow these
up by private mail).

Andrew.