Mailing List Archive: Caching OSM tiles - how to not overload backends?

Caching OSM tiles - how to not overload backends?

Jun 13, 2020, 7:31 PM

Post #1 of 7 (1235 views)

Hello!

Please forgive my bad English, I live in France.

Summary: how to cache - with Varnish - Open Street Map PNG images
without overloading OSM tiles servers?

The question seems related to Varnish backends and ".max_connections"
parameter.

A far as I know, if ".max_connections" is reached for a backend, Varnish
sends 503 http errors.

I understand the logic but would it be possible to queue these incoming
requests and wait until the selected backend is really available?

backend a_tile                  {
.host                         = "a.tile.openstreetmap.org";
.port                         = "80";
.max_connections              = 2;
}

If Varnish have, let's say 100 incoming requests in 1 second, how can I
handle this "spike" without overloading the backend?

All my google searches were "dead ends" so I think the question is
poorly formulated.

Note 1 : using [random|round_robin] directors could be a temporary solution
Note 2 : libvmod-dynamic is great but does not limit backend
simultaneous connexions

Many thanks for your help!

--
tranxene50
tranxene50@openvz-diff-backups.fr

Re: Caching OSM tiles - how to not overload backends? [ In reply to ]

dridi at varni

Jun 15, 2020, 11:00 AM

Post #2 of 7 (1232 views)

Permalink

On Sun, Jun 14, 2020 at 2:32 AM tranxene50
<tranxene50@openvz-diff-backups.fr> wrote:
>
> Hello!
>
> Please forgive my bad English, I live in France.
>
> Summary: how to cache - with Varnish - Open Street Map PNG images without overloading OSM tiles servers?
>
> The question seems related to Varnish backends and ".max_connections" parameter.
>
> A far as I know, if ".max_connections" is reached for a backend, Varnish sends 503 http errors.
>
> I understand the logic but would it be possible to queue these incoming requests and wait until the selected backend is really available?
>
> backend a_tile {
> .host = "a.tile.openstreetmap.org";
> .port = "80";
> .max_connections = 2;
> }
>
> If Varnish have, let's say 100 incoming requests in 1 second, how can I handle this "spike" without overloading the backend?
>
> All my google searches were "dead ends" so I think the question is poorly formulated.
>
> Note 1 : using [random|round_robin] directors could be a temporary solution
> Note 2 : libvmod-dynamic is great but does not limit backend simultaneous connexions
>
> Many thanks for your help!

Bonsoir,

Unfortunately we don't have any sort of queuing on the backend side,
so besides implementing your own backend transport from scratch in a
VMOD there is currently no solution.

Cordialement,
Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Caching OSM tiles - how to not overload backends? [ In reply to ]

xavier at ccmbenchmark

Jun 15, 2020, 3:17 PM

Post #3 of 7 (1232 views)

Permalink

Hello,

@tranxene50 if implementing a vmod can be very challenging, using haproxy
can be a great solution here. Please refer to this blog post:
https://www.haproxy.com/fr/blog/four-examples-of-haproxy-rate-limiting/ (or
in french ;)
https://www.haproxy.com/fr/blog/four-examples-of-haproxy-rate-limiting/ ).
The very first step is about setting a maximum connections number and a
queuing. Using haproxy as your backend would require low engineering and a
minimum overage.

Regards,

Le lun. 15 juin 2020 à 20:02, Dridi Boukelmoune <dridi@varni.sh> a écrit :

> On Sun, Jun 14, 2020 at 2:32 AM tranxene50
> <tranxene50@openvz-diff-backups.fr> wrote:
> >
> > Hello!
> >
> > Please forgive my bad English, I live in France.
> >
> > Summary: how to cache - with Varnish - Open Street Map PNG images
> without overloading OSM tiles servers?
> >
> > The question seems related to Varnish backends and ".max_connections"
> parameter.
> >
> > A far as I know, if ".max_connections" is reached for a backend, Varnish
> sends 503 http errors.
> >
> > I understand the logic but would it be possible to queue these incoming
> requests and wait until the selected backend is really available?
> >
> > backend a_tile {
> > .host = "a.tile.openstreetmap.org";
> > .port = "80";
> > .max_connections = 2;
> > }
> >
> > If Varnish have, let's say 100 incoming requests in 1 second, how can I
> handle this "spike" without overloading the backend?
> >
> > All my google searches were "dead ends" so I think the question is
> poorly formulated.
> >
> > Note 1 : using [random|round_robin] directors could be a temporary
> solution
> > Note 2 : libvmod-dynamic is great but does not limit backend
> simultaneous connexions
> >
> > Many thanks for your help!
>
> Bonsoir,
>
> Unfortunately we don't have any sort of queuing on the backend side,
> so besides implementing your own backend transport from scratch in a
> VMOD there is currently no solution.
>
> Cordialement,
> Dridi
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>

Re: Caching OSM tiles - how to not overload backends? [ In reply to ]

tranxene50 at openvz-diff-backups

Jun 16, 2020, 3:21 PM

Post #4 of 7 (1226 views)

Permalink

Hello!

Many thanks for your answers! :-)

@Dridi:

You are right, writing a specific VMOD would be the ideal solution but
unfortunately I am not qualified for the job. ^^

By the way, I would like to thank all the people who are working hard to
enhance and maintain Varnish.

This software is absolutely awesome!

@Xavier:

Before considering HAProxy, I searched if quick and dirty hacks were
possible with iptables to limit simultaneous connexions and tc to shape
the traffic.

But, after a quick reading of the documentation of HAProxy, it became
clear that - as you said - it is a reliable solution.

So, thanks the for the hint!

Have a great day!

Le 16/06/2020 à 00:17, Xavier Leune a écrit :
> Hello,
>
> @tranxene50 if implementing a vmod can be very challenging, using
> haproxy can be a great solution here. Please refer to this blog post:
> https://www.haproxy.com/fr/blog/four-examples-of-haproxy-rate-limiting/ (or
> in french ;)
> https://www.haproxy.com/fr/blog/four-examples-of-haproxy-rate-limiting/ ).
> The very first step is about setting a maximum connections number and
> a queuing. Using haproxy as your backend would require low engineering
> and a minimum overage.
>
> Regards,
>
> Le lun. 15 juin 2020 à 20:02, Dridi Boukelmoune <dridi@varni.sh> a écrit :
>
>
> Bonsoir,
>
> Unfortunately we don't have any sort of queuing on the backend side,
> so besides implementing your own backend transport from scratch in a
> VMOD there is currently no solution.
>
> Cordialement,
> Dridi
>
> On Sun, Jun 14, 2020 at 2:32 AM tranxene50
> <tranxene50@openvz-diff-backups.fr
> <mailto:tranxene50@openvz-diff-backups.fr>> wrote:
> >
> > Hello!
> >
> > Please forgive my bad English, I live in France.
> >
> > Summary: how to cache - with Varnish - Open Street Map PNG
> images without overloading OSM tiles servers?
> >
> > The question seems related to Varnish backends and
> ".max_connections" parameter.
> >
> > A far as I know, if ".max_connections" is reached for a backend,
> Varnish sends 503 http errors.
> >
> > I understand the logic but would it be possible to queue these
> incoming requests and wait until the selected backend is really
> available?
> >
> > backend a_tile {
> > .host = "a.tile.openstreetmap.org
> <http://a.tile.openstreetmap.org>";
> > .port = "80";
> > .max_connections = 2;
> > }
> >
> > If Varnish have, let's say 100 incoming requests in 1 second,
> how can I handle this "spike" without overloading the backend?
> >
> > All my google searches were "dead ends" so I think the question
> is poorly formulated.
> >
> > Note 1 : using [random|round_robin] directors could be a
> temporary solution
> > Note 2 : libvmod-dynamic is great but does not limit backend
> simultaneous connexions
> >
> > Many thanks for your help!
>

Re: Caching OSM tiles - how to not overload backends? [ In reply to ]

dwetzel at atanar

Jun 17, 2020, 6:08 AM

Post #5 of 7 (1224 views)

Permalink

Hi Tranx?ne50
I'm a reseller of Fastly in France a cdn based on a fork of varnish 2
maybe it could be a solution to minimize the backend traffic you receive somehow ?
i can make you test if needed
akamai has a solution they call waiting room, to regulate the traffic to the origin,
i think this is the kind of things to try to achieve ?
Best Regards,
Damien

tranxene50 writes:
> Hello!
>
> Many thanks for your answers! :-)
>
> @Dridi:
>
> You are right, writing a specific VMOD would be the ideal solution but unfortunately I am not qualified for the job. ^^
>
> By the way, I would like to thank all the people who are working hard to enhance and maintain Varnish.
>
> This software is absolutely awesome!
>
> @Xavier:
>
> Before considering HAProxy, I searched if quick and dirty hacks were possible with iptables to limit simultaneous connexions and tc to shape the traffic.
>
> But, after a quick reading of the documentation of HAProxy, it became clear that - as you said - it is a reliable solution.
>
> So, thanks the for the hint!
>
> Have a great day!
>
> Le 16/06/2020 ? 00:17, Xavier Leune a ?crit?:
>
> Hello,
>
> @tranxene50 if implementing a vmod can be very challenging, using haproxy can be a great solution here. Please refer to this blog post:?
> https://www.haproxy.com/fr/blog/four-examples-of-haproxy-rate-limiting/?(or in french ;)?https://www.haproxy.com/fr/blog/four-examples-of-haproxy-rate-limiting/?). The very first step is about setting a
> maximum connections number and a queuing. Using haproxy as your backend would require low engineering and a minimum overage.
>
> Regards,
>
> Le?lun. 15 juin 2020 ??20:02, Dridi Boukelmoune <dridi@varni.sh> a ?crit?:
>
> Bonsoir,
>
> Unfortunately we don't have any sort of queuing on the backend side,
> so besides implementing your own backend transport from scratch in a
> VMOD there is currently no solution.
>
> Cordialement,
> Dridi
>
> On Sun, Jun 14, 2020 at 2:32 AM tranxene50
> <tranxene50@openvz-diff-backups.fr> wrote:
> >
> > Hello!
> >
> > Please forgive my bad English, I live in France.
> >
> > Summary: how to cache - with Varnish - Open Street Map PNG images without overloading OSM tiles servers?
> >
> > The question seems related to Varnish backends and ".max_connections" parameter.
> >
> > A far as I know, if ".max_connections" is reached for a backend, Varnish sends 503 http errors.
> >
> > I understand the logic but would it be possible to queue these incoming requests and wait until the selected backend is really available?
> >
> > backend a_tile? ? ? ? ? ? ? ? ? {
> >? ?.host? ? ? ? ? ? ? ? ? ? ? ? ?= "a.tile.openstreetmap.org";
> >? ?.port? ? ? ? ? ? ? ? ? ? ? ? ?= "80";
> >? ?.max_connections? ? ? ? ? ? ? = 2;
> > }
> >
> > If Varnish have, let's say 100 incoming requests in 1 second, how can I handle this "spike" without overloading the backend?
> >
> > All my google searches were "dead ends" so I think the question is poorly formulated.
> >
> > Note 1 : using [random|round_robin] directors could be a temporary solution
> > Note 2 : libvmod-dynamic is great but does not limit backend simultaneous connexions
> >
> > Many thanks for your help!
>
>
> ----------------------------------------------------------------------
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

--

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Damien WETZEL (ATANAR TECHNOLOGIES) ("`-/")_.-'"``-._
http://www.atanar.com . . `; -._ )-;-,_`)
(v_,)' _ )`-.\ ``-'
Phone:+33 9 67 35 09 05 _.- _..-_/ / ((.'
- So much to do, so little time - ((,.-' ((,/
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Caching OSM tiles - how to not overload backends? [ In reply to ]

tranxene50 at openvz-diff-backups

Jun 17, 2020, 7:23 AM

Post #6 of 7 (1224 views)

Permalink

Hello Damien.

Thank you for your proposal, I appreciate!

But, as I can see/read, Fastly operates the same way as Varnish when the
backend is overloaded (it rejects new connections):

https://docs.fastly.com/en/guides/common-503-errors#error-503-backendmax_conn-reached

The "Waiting room" could be an option (but this is not what I try to
achieve because it throttles clients):

https://developer.fastly.com/solutions/patterns/waiting-room/

However, I prefer to stick to GPL or Open-Source projects and HAProxy
seems perfect to regulate the traffic sent to the backend. ;-)

Regards.

Le 17/06/2020 à 15:08, Damien Wetzel a écrit :
> Hi Tranxène50
> I'm a reseller of Fastly in France a cdn based on a fork of varnish 2
> maybe it could be a solution to minimize the backend traffic you receive somehow ?
> i can make you test if needed
> akamai has a solution they call waiting room, to regulate the traffic to the origin,
> i think this is the kind of things to try to achieve ?
> Best Regards,
> Damien
>
> tranxene50 writes:
> > Hello!
> >
> > Many thanks for your answers! :-)
> >
> > @Dridi:
> >
> > You are right, writing a specific VMOD would be the ideal solution but unfortunately I am not qualified for the job. ^^
> >
> > By the way, I would like to thank all the people who are working hard to enhance and maintain Varnish.
> >
> > This software is absolutely awesome!
> >
> > @Xavier:
> >
> > Before considering HAProxy, I searched if quick and dirty hacks were possible with iptables to limit simultaneous connexions and tc to shape the traffic.
> >
> > But, after a quick reading of the documentation of HAProxy, it became clear that - as you said - it is a reliable solution.
> >
> > So, thanks the for the hint!
> >
> > Have a great day!
> >
> > Le 16/06/2020 à 00:17, Xavier Leune a écrit :
> >
> > Hello,
> >
> > @tranxene50 if implementing a vmod can be very challenging, using haproxy can be a great solution here. Please refer to this blog post:
> > https://www.haproxy.com/fr/blog/four-examples-of-haproxy-rate-limiting/ (or in french ;) https://www.haproxy.com/fr/blog/four-examples-of-haproxy-rate-limiting/ ). The very first step is about setting a
> > maximum connections number and a queuing. Using haproxy as your backend would require low engineering and a minimum overage.
> >
> > Regards,
> >
> > Le lun. 15 juin 2020 à 20:02, Dridi Boukelmoune <dridi@varni.sh> a écrit :
> >
> > Bonsoir,
> >
> > Unfortunately we don't have any sort of queuing on the backend side,
> > so besides implementing your own backend transport from scratch in a
> > VMOD there is currently no solution.
> >
> > Cordialement,
> > Dridi
> >
> > On Sun, Jun 14, 2020 at 2:32 AM tranxene50
> > <tranxene50@openvz-diff-backups.fr> wrote:
> > >
> > > Hello!
> > >
> > > Please forgive my bad English, I live in France.
> > >
> > > Summary: how to cache - with Varnish - Open Street Map PNG images without overloading OSM tiles servers?
> > >
> > > The question seems related to Varnish backends and ".max_connections" parameter.
> > >
> > > A far as I know, if ".max_connections" is reached for a backend, Varnish sends 503 http errors.
> > >
> > > I understand the logic but would it be possible to queue these incoming requests and wait until the selected backend is really available?
> > >
> > > backend a_tile {
> > > .host = "a.tile.openstreetmap.org";
> > > .port = "80";
> > > .max_connections = 2;
> > > }
> > >
> > > If Varnish have, let's say 100 incoming requests in 1 second, how can I handle this "spike" without overloading the backend?
> > >
> > > All my google searches were "dead ends" so I think the question is poorly formulated.
> > >
> > > Note 1 : using [random|round_robin] directors could be a temporary solution
> > > Note 2 : libvmod-dynamic is great but does not limit backend simultaneous connexions
> > >
> > > Many thanks for your help!
> >
> >
> > ----------------------------------------------------------------------
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc@varnish-cache.org
> > https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
--
tranxene50
tranxene50@openvz-diff-backups.fr

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Re: Caching OSM tiles - how to not overload backends? [ In reply to ]

info at massivescale

Jun 29, 2020, 2:46 AM

Post #7 of 7 (1207 views)

Permalink

Hi,

You may want to look into OSM-supported tile proxy software. It's not
Varnish-based but maybe one of the options would be good enough for
your use:

https://wiki.openstreetmap.org/wiki/Tile_proxy

Regards,
Andrzej
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc