Mailing List Archive

#1794: varnish*-tools do not die when terminal disconnects
#1794: varnish*-tools do not die when terminal disconnects
-------------------+--------------------
Reporter: kwy | Type: defect
Status: new | Priority: normal
Milestone: | Component: build
Version: 4.0.3 | Severity: normal
Keywords: |
-------------------+--------------------
Open a varnishtop or varnishhist in a terminal. Then close the terminal,
and the program just launched keeps running.
Often the program will spin on 100% CPU and need a SIGKILL to terminate.

We expect the program to terminate.

Verified on RHEL6 and Arch current, so it's not a distribution-specific
issue.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+--------------------
Reporter: kwy | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by lkarsten):

I can reproduce this on 82a09dc.

varnishist process:
{{{
root@sierra:~# gdb -p 21658 --batch --eval-command="bt full"
[New LWP 21659]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fde11b9153d in nanosleep () at ../sysdeps/unix/syscall-
template.S:81
81 ../sysdeps/unix/syscall-template.S: No such file or directory.
#0 0x00007fde11b9153d in nanosleep () at ../sysdeps/unix/syscall-
template.S:81
No locals.
#1 0x00000000004051e0 in VTIM_sleep (t=0,01) at
../../lib/libvarnish/vtim.c:365
ts = {tv_sec = 0, tv_nsec = 10000000}
#2 0x000000000040683e in VUT_Main () at
../../lib/libvarnishtools/vut.c:397
c = 0x0
i = 0
__func__ = "VUT_Main"
#3 0x000000000040342f in main (argc=1, argv=0x7fffe14de6f8) at
varnishhist.c:468
i = 2
colon = 0x40691d <__libc_csu_init+77>
"H\203\303\001H9\353u\352H\203\304\b[]A\\A]A^A_Ðf.\017\037\204"
profile = 0x4069d4 "responsetime"
thr = 140591663175424
fnum = 3
cli_p = {name = 0x0, tag = SLT__Bogus, prefix = 0x0, field = 0,
hist_low = 0, hist_high = 0}
__func__ = "main"
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+--------------------
Reporter: kwy | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by lkarsten):

This is most likely related to there being no sighup handler defined.

Following patch removes the behavior for me:

{{{
diff --git a/bin/varnishhist/varnishhist.c b/bin/varnishhist/varnishhist.c
index 310eccc..d40264f 100644
--- a/bin/varnishhist/varnishhist.c
+++ b/bin/varnishhist/varnishhist.c
@@ -133,6 +133,12 @@ profiles[] = {

static struct profile *active_profile;

+static int __match_proto__(VUT_cb_f)
+sighup_handler(void)
+{
+ exit(1);
+}
+
static void
update(void)
{
@@ -464,6 +470,7 @@ main(int argc, char **argv)
exit(1);
}
VUT.dispatch_f = &accumulate;
+ VUT.sighup_f = &sighup_handler;
VUT.dispatch_priv = NULL;
VUT_Main();
end_of_file = 1;
}}}

Downside is that we don't clean up what curses has done, so the terminal
becomes garbled if it still exists. ("reset" fixes, of course)

Same sitation on varnishtop.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:2>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+--------------------
Reporter: kwy | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by fgsch):

From a quick look at the code I think the right way is to setup a sighup_f
function that returns 1, so the loop in VUT_Main() is broken:

{{{
if (VUT.sighup && VUT.sighup_f) {
/* sighup callback */
VUT.sighup = 0;
i = (VUT.sighup_f)();
if (i)
break;
}
}}}

btw, rotateout should only be used if the programs are running as daemon
(VUT.d_opt equals 1).
This looks like a bug to me in the current code.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+--------------------
Reporter: kwy | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by fgsch):

Attached patch should fix this and refuse -D without -w.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:4>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+--------------------
Reporter: kwy | Owner:
Type: defect | Status: new
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
--------------------+--------------------

Comment (by fgsch):

Tested and confirms this patch solves the issue.

Updated patch with doc fixes and amended test added.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:5>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+---------------------------------------------
Reporter: kwy | Owner: Federico G. Schwindt <fgsch@…>
Type: defect | Status: closed
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution: fixed
Keywords: |
--------------------+---------------------------------------------
Changes (by Federico G. Schwindt <fgsch@…>):

* status: new => closed
* owner: => Federico G. Schwindt <fgsch@…>
* resolution: => fixed


Comment:

In [b214d4806ed9f63a8bf95db29c176f1da86be13b]:
{{{
#!CommitTicketReference repository=""
revision="b214d4806ed9f63a8bf95db29c176f1da86be13b"
Handle terminal disconnections correctly

Change SIGHUP handling depending on whether we're running in daemon mode
or foreground. The former will continue rotating the logs, the latter
will abort the loop and die gracefully.

Fixes #1794.
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:6>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+---------------------------------------------
Reporter: kwy | Owner: Federico G. Schwindt <fgsch@…>
Type: defect | Status: closed
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution: fixed
Keywords: |
--------------------+---------------------------------------------

Comment (by Lasse Karstensen <lkarsten@…>):

In [01a6d89a58500111d9b92749a267b3c22ae4b847]:
{{{
#!CommitTicketReference repository=""
revision="01a6d89a58500111d9b92749a267b3c22ae4b847"
Handle terminal disconnections correctly

Change SIGHUP handling depending on whether we're running in daemon mode
or foreground. The former will continue rotating the logs, the latter
will abort the loop and die gracefully.

Fixes #1794.
}}}

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:7>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+---------------------------------------------
Reporter: kwy | Owner: Federico G. Schwindt <fgsch@…>
Type: defect | Status: reopened
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
--------------------+---------------------------------------------
Changes (by lkarsten):

* status: closed => reopened
* resolution: fixed =>


Comment:

Reopening this.

In 4.1.1 packaging we changed the startup options when running under
systemd to not let varnishncsa fork by itself. This makes packaging easier
since we don't have to bother with a writeable location for the pid file,
and so on.

With this change, logrotation hup-ing varnishncsa running "in foreground
mode" leads to the daemon shutting down on SIGHUP.

Short term fix is to roll back the systemd service file change and run new
4.1.1 packages, but I find the simplicity of not writing a pid file
appealing. We should consider if a better indicator can be used than -D.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:8>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+---------------------------------------------
Reporter: kwy | Owner: Federico G. Schwindt <fgsch@…>
Type: defect | Status: reopened
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution:
Keywords: |
--------------------+---------------------------------------------

Comment (by martin):

I have been discussing this issue with both Federico and Lasse, and these
are the conclusions:

* The current behaviour of exiting on sighup when in foreground mode is
the correct one.
* The main issue is for the startup scripts for running varnishlog and
varnishncsa as daemons. These scripts are part of the packages.
* The scripts want to start the utilities as the Varnish user (both
because they shouldn't run as root, and because the need to be the Varnish
user in order to get read access to the shmlog).
* When running as the Varnish user, they utilities will not be able to
write a PID file in /run, as /run is owned by root.

This can be worked around by having the scripts prepare a
/run/varnishutils directory owned by the Varnish user, and make the utils
store the PID file in there.

But the real issue is that the utils only implements half of what proper
daemonizing programs should be doing. The utils should have user/group
arguments and changed to that u/g upon daemonizing.

Martin

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:9>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs
Re: #1794: varnish*-tools do not die when terminal disconnects [ In reply to ]
#1794: varnish*-tools do not die when terminal disconnects
--------------------+---------------------------------------------
Reporter: kwy | Owner: Federico G. Schwindt <fgsch@…>
Type: defect | Status: closed
Priority: normal | Milestone:
Component: build | Version: 4.0.3
Severity: normal | Resolution: fixed
Keywords: |
--------------------+---------------------------------------------
Changes (by lkarsten):

* status: reopened => closed
* resolution: => fixed


Comment:

It would be nice to have the HUP handler for reopening the file in
foreground mode to use the new systemd simplicity, but that is really a
future feature and not a bug.

The current handling works fine, as far as I can tell. Closing.

--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1794#comment:10>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator

_______________________________________________
varnish-bugs mailing list
varnish-bugs@varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-bugs