Mailing List Archive

Limit requests to backend based on requested domain?
Hi,

Is there a way to limit the number of concurrent requests sent to a backend based on the requested domain name? Or more broadly, based on some key/value?

I know I can limit the number of requests to a backend with max requests, but our backend services multiple domains, and I’m wanting to limit the number of requests per domain (so that too many requests to one doesn’t cut off resources for others). I’ve thought of maybe doing this by defining multiple backends (pointed at the same server), but then I worry that the health checks we run would stack up (i.e. if there are a 100 domains, going from 1 backend to 100 backends would presumable increase the health checks 100 times).

Thanks,
Jeff

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Limit requests to backend based on requested domain? [ In reply to ]
On Wed, Aug 30, 2017 at 9:24 AM, <jpotter-varnish@codepuppy.com> wrote:
>
> Hi,
>
> Is there a way to limit the number of concurrent requests sent to a backend based on the requested domain name? Or more broadly, based on some key/value?

Have a look at the vsthrottle VMOD, it's usually used for client
requests but you can probably use it on the backend side too.

https://github.com/varnish/varnish-modules
https://github.com/varnish/varnish-modules/blob/master/docs/vmod_vsthrottle.rst

Dridi

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Limit requests to backend based on requested domain? [ In reply to ]
Hi Jeff,

have a look at vmod_vsthrottle in varnish-modules, that should do it.

--
Guillaume Quintard

On Wed, Aug 30, 2017 at 9:24 AM, <jpotter-varnish@codepuppy.com> wrote:

>
> Hi,
>
> Is there a way to limit the number of concurrent requests sent to a
> backend based on the requested domain name? Or more broadly, based on some
> key/value?
>
> I know I can limit the number of requests to a backend with max requests,
> but our backend services multiple domains, and I’m wanting to limit the
> number of requests per domain (so that too many requests to one doesn’t cut
> off resources for others). I’ve thought of maybe doing this by defining
> multiple backends (pointed at the same server), but then I worry that the
> health checks we run would stack up (i.e. if there are a 100 domains, going
> from 1 backend to 100 backends would presumable increase the health checks
> 100 times).
>
> Thanks,
> Jeff
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: Limit requests to backend based on requested domain? [ In reply to ]
Thanks, Guillaume and Dridi — I’m not seeing a way to limit concurrent requests however. We’re using vsthrottle to limit the total number of requests to a domain — i.e. “100 requests in 10 seconds; 600 requests in 5 minutes”; but that doesn’t guard against the case of someone’s PHP script blocking (say, external API request from some wordpress plugin), and when that happens, those PHP workers pile up pretty fast. Normally, requests to the backend at the rate of something like 100 per second in total would be reasonable, but in the case of bad backend PHP code in one domain, well; it’d resource starve workers for all the domains on that backend server (shared hosting).

Perhaps this is something that’s similar enough to what vsthrottle does that I should fork it and figure out how to implement it as a counter instead?

Thanks,
Jeff

> On Aug 30, 2017, at 4:22 AM, Guillaume Quintard <guillaume@varnish-software.com> wrote:
>
> Hi Jeff,
>
> have a look at vmod_vsthrottle in varnish-modules, that should do it.
>
> --
> Guillaume Quintard
>
> On Wed, Aug 30, 2017 at 9:24 AM, <jpotter-varnish@codepuppy.com <mailto:jpotter-varnish@codepuppy.com>> wrote:
>
> Hi,
>
> Is there a way to limit the number of concurrent requests sent to a backend based on the requested domain name? Or more broadly, based on some key/value?
>
> I know I can limit the number of requests to a backend with max requests, but our backend services multiple domains, and I’m wanting to limit the number of requests per domain (so that too many requests to one doesn’t cut off resources for others). I’ve thought of maybe doing this by defining multiple backends (pointed at the same server), but then I worry that the health checks we run would stack up (i.e. if there are a 100 domains, going from 1 backend to 100 backends would presumable increase the health checks 100 times).
>
> Thanks,
> Jeff
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org <mailto:varnish-misc@varnish-cache.org>
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
Re: Limit requests to backend based on requested domain? [ In reply to ]
On Wed, Aug 30, 2017 at 6:28 PM, <jpotter-varnish@codepuppy.com> wrote:
>
> Thanks, Guillaume and Dridi — I’m not seeing a way to limit concurrent
> requests however. We’re using vsthrottle to limit the total number of
> requests to a domain — i.e. “100 requests in 10 seconds; 600 requests in 5
> minutes”; but that doesn’t guard against the case of someone’s PHP script
> blocking (say, external API request from some wordpress plugin), and when
> that happens, those PHP workers pile up pretty fast. Normally, requests to
> the backend at the rate of something like 100 per second in total would be
> reasonable, but in the case of bad backend PHP code in one domain, well;
> it’d resource starve workers for all the domains on that backend server
> (shared hosting).

Correct.

> Perhaps this is something that’s similar enough to what vsthrottle does that
> I should fork it and figure out how to implement it as a counter instead?

Browse the issue tracker, we may have discussed that at some point but
I'm not sure. Please also have a look at the old repositories before
it got bundled to varnish-modules. We are close to the next Varnish
release and I can't spend much time on this.

https://github.com/varnish/libvmod-vsthrottle

The problem with explicit inc/dec operations is that they may not be
balanced, so you'll need a PRIV_TASK to keep track of that and clear
the state if the transaction ends with a positive balance.

Dridi

_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc