Mailing List Archive

backend web node keep-alive option
I was observing a behavior of a PHP script on one backend server that got executed/requested
twice (but not by the client/browser).

The plain script just processes data and only outputs a response until its done. The observed
behavior is; when first_byte_timeout passes but the script is still processing data, the varnish
node answers with an 503 and the script gets listed twice in the webserver process list. So, either
it gets requested on this event again or the TCP connection close triggers an new execution. In any
case its a bug and because of missing resources to dive into a more deeper analyses, I just disabled
the enabled keep-alive option of the backend webserver (apache/httpd). The results: it helped to avoid
a further execution of the script.

The question is: What is the best practice to configure the backend webservers? Do you keep the
keep-alive option enabled or not?

I'd appreciate hearing your suggestions.

Leon







_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: backend web node keep-alive option [ In reply to ]
Hi Leon,

Why not just up the timeout value? Usually, you want to keep the keep-alive
as it's much more efficient.

Cheers,

--
Guillaume Quintard

On Sat, Nov 11, 2017 at 9:48 PM, <info+varnish@shee.org> wrote:

> I was observing a behavior of a PHP script on one backend server that got
> executed/requested
> twice (but not by the client/browser).
>
> The plain script just processes data and only outputs a response until its
> done. The observed
> behavior is; when first_byte_timeout passes but the script is still
> processing data, the varnish
> node answers with an 503 and the script gets listed twice in the webserver
> process list. So, either
> it gets requested on this event again or the TCP connection close triggers
> an new execution. In any
> case its a bug and because of missing resources to dive into a more deeper
> analyses, I just disabled
> the enabled keep-alive option of the backend webserver (apache/httpd). The
> results: it helped to avoid
> a further execution of the script.
>
> The question is: What is the best practice to configure the backend
> webservers? Do you keep the
> keep-alive option enabled or not?
>
> I'd appreciate hearing your suggestions.
>
> Leon
>
>
>
>
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc@varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
Re: backend web node keep-alive option [ In reply to ]
On Sat, Nov 11, 2017 at 9:48 PM, <info+varnish@shee.org> wrote:
> I was observing a behavior of a PHP script on one backend server that got executed/requested
> twice (but not by the client/browser).
>
> The plain script just processes data and only outputs a response until its done. The observed
> behavior is; when first_byte_timeout passes but the script is still processing data, the varnish
> node answers with an 503 and the script gets listed twice in the webserver process list. So, either
> it gets requested on this event again or the TCP connection close triggers an new execution. In any
> case its a bug and because of missing resources to dive into a more deeper analyses, I just disabled
> the enabled keep-alive option of the backend webserver (apache/httpd). The results: it helped to avoid
> a further execution of the script.
>
> The question is: What is the best practice to configure the backend webservers? Do you keep the
> keep-alive option enabled or not?
>
> I'd appreciate hearing your suggestions.

I think you're getting a retry from varnishd, independent from VCL's
return(retry) which only happens on GET/HEAD requests that are
supposed to be side-effect-free and safely retry-able. On top of that
you might be hitting a bug where Varnish retries more than once.
Seeing some log would help confirm this hypothesis.

https://github.com/varnishcache/varnish-cache/pull/2135

Dridi
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: backend web node keep-alive option [ In reply to ]
Am 12.11.2017 um 15:42 schrieb Guillaume Quintard <guillaume@varnish-software.com>:
>
> On Sat, Nov 11, 2017 at 9:48 PM, <info+varnish@shee.org> wrote:
>
>> I was observing a behavior of a PHP script on one backend server that got executed/requested
>> twice (but not by the client/browser).
>>
>> The plain script just processes data and only outputs a response until its done. The observed
>> behavior is; when first_byte_timeout passes but the script is still processing data, the varnish
>> node answers with an 503 and the script gets listed twice in the webserver process list. So, either
>> it gets requested on this event again or the TCP connection close triggers an new execution. In any
>> case its a bug and because of missing resources to dive into a more deeper analyses, I just disabled
>> the enabled keep-alive option of the backend webserver (apache/httpd). The results: it helped to avoid
>> a further execution of the script.
>>
>> The question is: What is the best practice to configure the backend webservers? Do you keep the
>> keep-alive option enabled or not?
>>

Hi Guillaume,

>
> Why not just up the timeout value?


Sure, the mentioned script(task) is for my taste conceptually bad designed and I could
convinced the developer to run some kind of a task queue.

The primary focus of my question targeted the keep-alive part but while here: does it
have negative implications having e.g. a 2h first_byte_timeout configuration? Is this
common?


> Usually, you want to keep the keep-alive as it's much more efficient.

Also sure, but my big picture while asking was - heavy traffic. While the costs of the TCP handshake
persists, the scenario is at the backend of the architecture where high bandwidth links exits.
So, i can imagine that in heavy traffic scenarios, the keep-alive feature would lead to more
memory consumption, more file descriptors and be more prone to denial of service attacks.

Any experiences out there?
--
Thanks,
Leon


_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: backend web node keep-alive option [ In reply to ]
> Am 13.11.2017 um 08:11 schrieb Dridi Boukelmoune <dridi@varni.sh>:
>
> On Sat, Nov 11, 2017 at 9:48 PM, <info+varnish@shee.org> wrote:
>> I was observing a behavior of a PHP script on one backend server that got executed/requested
>> twice (but not by the client/browser).
>>
>> The plain script just processes data and only outputs a response until its done. The observed
>> behavior is; when first_byte_timeout passes but the script is still processing data, the varnish
>> node answers with an 503 and the script gets listed twice in the webserver process list. So, either
>> it gets requested on this event again or the TCP connection close triggers an new execution. In any
>> case its a bug and because of missing resources to dive into a more deeper analyses, I just disabled
>> the enabled keep-alive option of the backend webserver (apache/httpd). The results: it helped to avoid
>> a further execution of the script.
>>
>> The question is: What is the best practice to configure the backend webservers? Do you keep the
>> keep-alive option enabled or not?
>>
>> I'd appreciate hearing your suggestions.
>
> I think you're getting a retry from varnishd, independent from VCL's
> return(retry) which only happens on GET/HEAD requests that are
> supposed to be side-effect-free and safely retry-able. On top of that
> you might be hitting a bug where Varnish retries more than once.
> Seeing some log would help confirm this hypothesis.
>
> https://github.com/varnishcache/varnish-cache/pull/2135


Ah, okay. Is this still valid for version 5.1/5.2?

The mentioned script doesn't exist anymore. I will try to spend some time to reconstruct this case.

--
Thanks,
Leon






_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
Re: backend web node keep-alive option [ In reply to ]
>> I think you're getting a retry from varnishd, independent from VCL's
>> return(retry) which only happens on GET/HEAD requests that are
>> supposed to be side-effect-free and safely retry-able. On top of that
>> you might be hitting a bug where Varnish retries more than once.
>> Seeing some log would help confirm this hypothesis.
>>
>> https://github.com/varnishcache/varnish-cache/pull/2135
>
>
> Ah, okay. Is this still valid for version 5.1/5.2?

Yes, but fixed two days ago so it will be part of the next release.

https://github.com/varnishcache/varnish-cache/commit/fa3884391f85565cb7d564cfa5f22d066948d7bc

> The mentioned script doesn't exist anymore. I will try to spend some time to reconstruct this case.

Happy to see you worked around the problem on your side.

Cheers
_______________________________________________
varnish-misc mailing list
varnish-misc@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc