Mailing List Archive

On 03/31/2018 10:21 AM, Danila Vershinin wrote:
>
> If a constant PHP error occurs in the backend and there is no cache -
> we see Backend fetch failed.
[...]

> ... it always takes time to explain to the clients that the Varnish
> is not to blame ...
Welcome to Varnish administration! Such is the life.

When the backend fetch fails, always look to the FetchError entry in the
backend log, which diagnoses the problem. It helps with those conversations.

> ... and ideally there would be a way to deliver different Varnish
> error page for these 3 cases:>
> * 500 error-ed backend
> * 404-ed backend
> * actual problem talking to the backend (HTTP etc.)

Varnish can generate the response itself with vcl_synth -- when you see
the Guru Meditation and "Backend fetch failed", you're seeing the
buitlin.vcl version of vcl_synth.

vcl_synth is called automatically for response codes 500 & 503, for 404
or anything else, your VCL has to direct to there:

sub vcl_deliver {
if (resp.status == 404) {
return (synth(404));
}
}

So you could have something like this:

sub vcl_synth {
if (resp.status == 500) {
synthetic("500 error-ed backend");
}
elsif (resp.status == 404) {
synthetic("404-ed backend");
}
elsif (resp.reason == "Backend fetch failed") {
synthetic("actual problem talking to the backend");
}
}

Since I believe about Varnish 5.0 or so you can use also this syntax to
generate the synthetic response (synthetic() works as well):

set resp.body = "mumble";

Of course you'll probably want to have HTML markup in the generated
response -- look to the Guru Meditation in builtin.vcl's vcl_synth for
an example.


HTH,
Geoff
--
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de

Hi Geoff,

Thank you. I understand about vcl_synth being used when getting 500 & 503 better :)
However, I’m also interested how can we distinguish backend errors while using grace mode and failed probe.

Suppose that you have configured grace mode and a health check against homepage. And homepage fails with 500.
The backend is marked as sick, then synthetic response (guru meditation) is delivered from vcl_backend_error with resp.status pre-filled to 503.
So there seems to be no way to do the same conditional error display in that case?

Best Regards,
Danila

> On 31 Mar 2018, at 14:25, Geoff Simmons <geoff@uplex.de> wrote:
>
> On 03/31/2018 10:21 AM, Danila Vershinin wrote:
>>
>> If a constant PHP error occurs in the backend and there is no cache -
>> we see Backend fetch failed.
> [...]
>
>> ... it always takes time to explain to the clients that the Varnish
>> is not to blame ...
> Welcome to Varnish administration! Such is the life.
>
> When the backend fetch fails, always look to the FetchError entry in the
> backend log, which diagnoses the problem. It helps with those conversations.
>
>> ... and ideally there would be a way to deliver different Varnish
>> error page for these 3 cases:>
>> * 500 error-ed backend
>> * 404-ed backend
>> * actual problem talking to the backend (HTTP etc.)
>
> Varnish can generate the response itself with vcl_synth -- when you see
> the Guru Meditation and "Backend fetch failed", you're seeing the
> buitlin.vcl version of vcl_synth.
>
> vcl_synth is called automatically for response codes 500 & 503, for 404
> or anything else, your VCL has to direct to there:
>
> sub vcl_deliver {
> if (resp.status == 404) {
> return (synth(404));
> }
> }
>
> So you could have something like this:
>
> sub vcl_synth {
> if (resp.status == 500) {
> synthetic("500 error-ed backend");
> }
> elsif (resp.status == 404) {
> synthetic("404-ed backend");
> }
> elsif (resp.reason == "Backend fetch failed") {
> synthetic("actual problem talking to the backend");
> }
> }
>
> Since I believe about Varnish 5.0 or so you can use also this syntax to
> generate the synthetic response (synthetic() works as well):
>
> set resp.body = "mumble";
>
> Of course you'll probably want to have HTML markup in the generated
> response -- look to the Guru Meditation in builtin.vcl's vcl_synth for
> an example.
>
>
> HTH,
> Geoff
> --
> ** * * UPLEX - Nils Goroll Systemoptimierung
>
> Scheffelstraße 32
> 22301 Hamburg
>
> Tel +49 40 2880 5731
> Mob +49 176 636 90917
> Fax +49 40 42949753
>
> http://uplex.de
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

On 03/31/2018 02:36 PM, Danila Vershinin wrote:
>
> However, I’m also interested how can we distinguish backend errors
> while using grace mode and failed probe.
>
> Suppose that you have configured grace mode and a health check
> against homepage. And homepage fails with 500.
> The backend is marked as sick, then synthetic response (guru
> meditation) is delivered from vcl_backend_error with resp.status
> pre-filled to 503.
> So there seems to be no way to do the same conditional error display
> in that case?

Oh I'm sorry, I think I may have misunderstood your question.

I assume in this scenario that Varnish never got a valid response that
could be cached -- otherwise, the cached object could have been
delivered while it's still in grace.

As long as the health probe is failing, Varnish doesn't attempt any
fetch in response to a client request at all. Varnish makes a note of
the backend sick state, and then any request meant for that backend goes
right to the 503 response and vcl_backend_error, logging FetchError=="no
backend connection". There's no attempt to re-use or initiate a network
connection, let alone forward a request, it just becomes 503 and that's
it, until the health probe goes back to healthy.

So the 503 you get in that situation is only indirectly related to the
500 health probe response -- the health probes see the 500, running in
their own threads and independent of any request/response traffic.

That means that you can't establish a relation between the 503 fetch
failure and the 500 health probe response in VCL. What you can do is
look at std.healthy(backend), something like:

import std;

if (!std.healthy(mybackend)) {
set resp.body = "..." + "... because the backend is sick ...";
}

And of course there's the log -- FetchError=="no backend connection" and
the Backend_health entries.


Hope that helped a little better this time,
Geoff
--
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de