Mailing List Archive

Varnish only resolve the ip on startup
Hello,
While setting up a docker-compose with a cache powered by varnish I came across an issue when recreating a container that varnish depends on. After the recreation varnish doesn't try to retrieve the new local ip. I've been told this is an upstream varnish issue (https://github.com/varnish/docker-varnish/issues/41). I have found some ways to solve the problem, but will an appropriate solution be considered ?

Thanks in advance,
Have a good day,
Léo Lelievre
Re: Varnish only resolve the ip on startup [ In reply to ]
I think it makes sense for Varnish to natively support backends changing
their IPs. I do get the performance argument but now that there is a
cloud/container market and that Varnish has proven to be useful in it, this
basic functionality should be brought in.

Would it be acceptable to add a "host_string" to vrt_endpoint and fill it
if the VCL backend isn't an IP, then, we can add another cp_methods to
cache_conn_pool.c to use it? This way IPs are still super fast, and
hostnames become actually useful and a bit less confusing?

--
Guillaume Quintard


On Sun, Oct 3, 2021 at 9:07 AM Léo <leo.lelievre@protonmail.com> wrote:

> Hello,
> While setting up a docker-compose with a cache powered by varnish I came
> across an issue when recreating a container that varnish depends on. After
> the recreation varnish doesn't try to retrieve the new local ip. I've been
> told this is an upstream varnish issue (
> https://github.com/varnish/docker-varnish/issues/41). I have found some
> ways to solve the problem, but will an appropriate solution be considered ?
>
> Thanks in advance,
> Have a good day,
> Léo Lelievre
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: Varnish only resolve the ip on startup [ In reply to ]
On Mon, Oct 4, 2021 at 3:45 PM Guillaume Quintard
<guillaume.quintard@gmail.com> wrote:
>
> I think it makes sense for Varnish to natively support backends changing their IPs. I do get the performance argument but now that there is a cloud/container market and that Varnish has proven to be useful in it, this basic functionality should be brought in.

I would assume the primary argument was simplicity, not performance,
but I wasn't around. One could argue it's turned into simplism in
today's cloudy cloud cloudy world.

> Would it be acceptable to add a "host_string" to vrt_endpoint and fill it if the VCL backend isn't an IP, then, we can add another cp_methods to cache_conn_pool.c to use it? This way IPs are still super fast, and hostnames become actually useful and a bit less confusing?

One problem I have (and that you should be familiar with) is that
portable interfaces we have that *respect* the system configuration
(hosts file, nsswitch configuration etc) are not providing enough
information. For example it becomes cumbersome to resolve SRV records
or even get the TTL of individual records for a DNS resolution in a
*portable* fashion.

When you put it like this, it sounds simple enough (dare I say
simplistic?) but what I see is a sizeable can of worms.

I do think we could do something about it, I don't know what would be
satisfying.

Cheers,
Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish only resolve the ip on startup [ In reply to ]
On Mon, Oct 4, 2021 at 9:49 AM Dridi Boukelmoune <dridi@varni.sh> wrote:

> One problem I have (and that you should be familiar with) is that
> portable interfaces we have that *respect* the system configuration
> (hosts file, nsswitch configuration etc) are not providing enough
> information. For example it becomes cumbersome to resolve SRV records
> or even get the TTL of individual records for a DNS resolution in a
> *portable* fashion.
>
> When you put it like this, it sounds simple enough (dare I say
> simplistic?) but what I see is a sizeable can of worms.
>

That sounds like a bit of a strawman to me. getaddrinfo and connect are
standard, and that's about all we should need. Applications are supposed
(in general) to just use whatever gai give them. We can call them every
time we need a new connection so we don't worry about TTL, and we just
disregard SRV records.
The vast majority of users don't need SRV (yet?), and don't expect the
application to optimize DNS calls, but they do complain that giving a
hostname to VCL doesn't work.

Let's just provide basic, expected functionality out of the box, and leave
the fancier features to vmod_goto and vmod_dynamic.

--
Guillaume Quintard
RE: Varnish only resolve the ip on startup [ In reply to ]
I’m definitely watching this topic, considering I’m planning on moving to Varnish Enterprise next year and putting a cluster in ECS, if not Fargate, so being able to easily handle dynamic IPs would be extremely helpful.

From: varnish-misc <varnish-misc-bounces+justinl=arena.net@varnish-cache.org> On Behalf Of Guillaume Quintard
Sent: Monday, October 4, 2021 10:16 AM
To: Dridi Boukelmoune <dridi@varni.sh>
Cc: varnish-misc@varnish-cache.org
Subject: Re: Varnish only resolve the ip on startup

On Mon, Oct 4, 2021 at 9:49 AM Dridi Boukelmoune <dridi@varni.sh<mailto:dridi@varni.sh>> wrote:
One problem I have (and that you should be familiar with) is that
portable interfaces we have that *respect* the system configuration
(hosts file, nsswitch configuration etc) are not providing enough
information. For example it becomes cumbersome to resolve SRV records
or even get the TTL of individual records for a DNS resolution in a
*portable* fashion.

When you put it like this, it sounds simple enough (dare I say
simplistic?) but what I see is a sizeable can of worms.

That sounds like a bit of a strawman to me. getaddrinfo and connect are standard, and that's about all we should need. Applications are supposed (in general) to just use whatever gai give them. We can call them every time we need a new connection so we don't worry about TTL, and we just disregard SRV records.
The vast majority of users don't need SRV (yet?), and don't expect the application to optimize DNS calls, but they do complain that giving a hostname to VCL doesn't work.

Let's just provide basic, expected functionality out of the box, and leave the fancier features to vmod_goto and vmod_dynamic.

--
Guillaume Quintard
Re: Varnish only resolve the ip on startup [ In reply to ]
On Mon, Oct 4, 2021 at 5:29 PM Justin Lloyd <justinl@arena.net> wrote:
>
> I’m definitely watching this topic, considering I’m planning on moving to Varnish Enterprise next year and putting a cluster in ECS, if not Fargate, so being able to easily handle dynamic IPs would be extremely helpful.

Like I implied in my initial response you have this ability out of the
box with Varnish Enterprise, and with Varnish Cache there are
third-party VMODs.

Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish only resolve the ip on startup [ In reply to ]
On Mon, Oct 4, 2021 at 5:15 PM Guillaume Quintard
<guillaume.quintard@gmail.com> wrote:
>
> On Mon, Oct 4, 2021 at 9:49 AM Dridi Boukelmoune <dridi@varni.sh> wrote:
>>
>> One problem I have (and that you should be familiar with) is that
>> portable interfaces we have that *respect* the system configuration
>> (hosts file, nsswitch configuration etc) are not providing enough
>> information. For example it becomes cumbersome to resolve SRV records
>> or even get the TTL of individual records for a DNS resolution in a
>> *portable* fashion.
>>
>> When you put it like this, it sounds simple enough (dare I say
>> simplistic?) but what I see is a sizeable can of worms.
>
>
> That sounds like a bit of a strawman to me. getaddrinfo and connect are standard, and that's about all we should need. Applications are supposed (in general) to just use whatever gai give them. We can call them every time we need a new connection so we don't worry about TTL, and we just disregard SRV records.
> The vast majority of users don't need SRV (yet?), and don't expect the application to optimize DNS calls, but they do complain that giving a hostname to VCL doesn't work.

The "portable" interface I was referring to includes getaddrinfo, and
I guess your suggestion would be to always resolve and leave A/AAAA
records caching to your stub and/or recursive resolver.

Fair enough, simplicity.

That still doesn't seal the can of worms: once there are more than one
address per family or addresses change, it's our connection and
pooling models that need to be revisited, how many different addresses
to try connecting to, how to retry, and how we account for all of that
(stats for example). Again, it's a bit more complex that just saying
"change the connect callback to one that combines resolve+connect".

I'm not against a holistic management of connections, there are
actually more aspects that I think should be at the core of our
connection management like certificates attached to a session (and in
particular their domains) once we have TLS support.

I just think it needs to be better defined and not give the impression
that messing about a new set of backend callbacks that will
systematically resolve endpoints is as simple as it gets.

Cheers
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish only resolve the ip on startup [ In reply to ]
On Mon, Oct 4, 2021 at 11:55 AM Dridi Boukelmoune <dridi@varni.sh> wrote:

> That still doesn't seal the can of worms: once there are more than one
> address per family or addresses change, it's our connection and
> pooling models that need to be revisited, how many different addresses
> to try connecting to, how to retry, and how we account for all of that
> (stats for example). Again, it's a bit more complex that just saying
> "change the connect callback to one that combines resolve+connect".
>

I do understand that it's the core of the problem, and I'm probably being
pig-headed on this, but it feels to me that it's not really different from
a server with a floating IP, or a level-4 load-balancer fronting the
backend. The addresses may change, there may be more than one, but once the
connection is open, you can trust it and keep using it.
We trust connect() to get us to our goal, using getaddrinfo() on top of it
just means that we trust the DNS server to provide good info. So:
- if we prefer IPv6, we go through the list and pick the first IPv6 entry,
if we don't find one, we just grab the first IPv4 entry, no second chance
- if you try to reuse a connection and it died on you, you try to
resolve+connect a new one. Maybe you get the same IP, maybe you don't, but
we trust the DNS implementation to shuffle the entries around
- stats is an interesting issue, but again, only if you let it be. Backends
are already "IP address" or "UDS path", I'm fine with hiding all the DNS
entries behind "non-numerical host".

I agree, things get gnarly when we start fiddling with DNs, but there's
that portable interface that allows us to not have to. And if you want
fancy stuff like stats per IP, and fun load-balancing options that are
DNS-aware, there are vmods for that.

For the sake of transparency, I must admit that I do have an issue with my
proposal though: there's no timeout option for getaddrinfo, which sucks.

--
Guillaume Quintard
Re: Varnish only resolve the ip on startup [ In reply to ]
On 03.10.21 18:05, Léo wrote:
> I have found some ways to solve the problem

https://github.com/nigoroll/libvmod-dynamic exists to do just that

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Varnish only resolve the ip on startup [ In reply to ]
I'm fine with adding vmod_dynamic to the varnish docker images so that
users have access to this basic functionality, but, for the record:
- I really feel this should be considered a basic feature and exist in core
- if the docker image starts adding downstream vmods, it opens the gates to
a flood of "can you add this vmod too?" questions, which I'm not looking
forward to.

--
Guillaume Quintard


On Tue, Oct 5, 2021 at 6:39 AM Nils Goroll <slink@schokola.de> wrote:

> On 03.10.21 18:05, Léo wrote:
> > I have found some ways to solve the problem
>
> https://github.com/nigoroll/libvmod-dynamic exists to do just that
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: Varnish only resolve the ip on startup [ In reply to ]
mailto:slink@schokola.de

> On 03.10.21 18:05, Léo wrote:
>> I have found some ways to solve the problem
>
> https://github.com/nigoroll/libvmod-dynamic exists to do just that

The point was to keep the docker compose as simple as possible without the need to install other services