https://bz.apache.org/bugzilla/show_bug.cgi?id=65697
Bug ID: 65697
Summary: Initial state of proxy backend is 'Ok' when
healthcheck was not run yet
Product: Apache httpd-2
Version: 2.4.51
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: mod_proxy_hcheck
Assignee: bugs@httpd.apache.org
Reporter: hendrik.jaeger@dswiss.com
Target Milestone: ---
Symptoms
mod_status reports a proxy balancer backend as 'Init Ok' for the duration of 1
hcinterval after startup
Environment
I compiled apache 2.4.51 from source on Debian stable (bullseye).
httpd -V output:
Server version: Apache/2.4.51 (Unix)
Server built: Nov 15 2021 16:33:58
Server's Module Magic Number: 20120211:118
Server loaded: APR 1.7.0, APR-UTIL 1.6.1
Compiled using: APR 1.7.0, APR-UTIL 1.6.1
Architecture: 64-bit
Server MPM: event
threaded: yes (fixed thread count)
forked: yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_PROC_PTHREAD_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/home/henk/usr/local"
-D SUEXEC_BIN="/home/henk/usr/local/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
Steps to reproduce
httpd.conf:
ServerRoot "/home/henk/usr/local"
Listen 80
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule watchdog_module modules/mod_watchdog.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_hcheck_module modules/mod_proxy_hcheck.so
LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule status_module modules/mod_status.so
ServerAdmin you@example.com
ErrorLog "logs/error_log"
LogLevel warn
LogLevel proxy_hcheck:TRACE2 proxy:TRACE2 watchdog:TRACE2
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"
%I %O" combinedio
</IfModule>
CustomLog "logs/access_log" common
</IfModule>
<Proxy balancer://bz54657>
BalancerMember http://hopefully.nonexistant.example.org hcmethod=GET
</Proxy>
<Location "/server-status">
SetHandler server-status
</Location>
ProxyStatus On
request:
curl localhost/server-status?auto 2>/dev/null | grep -e Status -e Uptime
Expected result:
The worker should be marked 'Init HcFl' because its name cannot even be
resolved.
Actual result:
% curl localhost/server-status?auto | grep -i -e status -e uptime
ServerUptimeSeconds: 29
ServerUptime: 29 seconds
Uptime: 29
ProxyBalancer[0]Worker[0]Status: Init Ok
% curl localhost/server-status?auto | grep -i -e status -e uptime
ServerUptimeSeconds: 30
ServerUptime: 30 seconds
Uptime: 30
ProxyBalancer[0]Worker[0]Status: Init HcFl
Suspected issues:
* The first health check is done 30s after startup because the default value of
hcinterval (how often to do a healthcheck) is 30s.
* Backend workers are by default in the state 'Init Ok' until a healthcheck is
run.
Expected behaviour:
* To be on the 'safe' side, backend workers should either not have a state
directly after httpd startup until a healthcheck has run, that state should be
'undetermined' or 'failed' but not 'Ok'.
* The first healthcheck should be run during or immediately after httpd
startup, not hcinterval after httpd startup.
Impact:
The mod_status page could be used for monitoring backend status and alerting if
all backends are down but its content needs to be accurate for that to be
useful.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
Bug ID: 65697
Summary: Initial state of proxy backend is 'Ok' when
healthcheck was not run yet
Product: Apache httpd-2
Version: 2.4.51
Hardware: PC
OS: Linux
Status: NEW
Severity: normal
Priority: P2
Component: mod_proxy_hcheck
Assignee: bugs@httpd.apache.org
Reporter: hendrik.jaeger@dswiss.com
Target Milestone: ---
Symptoms
mod_status reports a proxy balancer backend as 'Init Ok' for the duration of 1
hcinterval after startup
Environment
I compiled apache 2.4.51 from source on Debian stable (bullseye).
httpd -V output:
Server version: Apache/2.4.51 (Unix)
Server built: Nov 15 2021 16:33:58
Server's Module Magic Number: 20120211:118
Server loaded: APR 1.7.0, APR-UTIL 1.6.1
Compiled using: APR 1.7.0, APR-UTIL 1.6.1
Architecture: 64-bit
Server MPM: event
threaded: yes (fixed thread count)
forked: yes (variable process count)
Server compiled with....
-D APR_HAS_SENDFILE
-D APR_HAS_MMAP
-D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
-D APR_USE_PROC_PTHREAD_SERIALIZE
-D APR_USE_PTHREAD_SERIALIZE
-D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
-D APR_HAS_OTHER_CHILD
-D AP_HAVE_RELIABLE_PIPED_LOGS
-D DYNAMIC_MODULE_LIMIT=256
-D HTTPD_ROOT="/home/henk/usr/local"
-D SUEXEC_BIN="/home/henk/usr/local/bin/suexec"
-D DEFAULT_PIDLOG="logs/httpd.pid"
-D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
-D DEFAULT_ERRORLOG="logs/error_log"
-D AP_TYPES_CONFIG_FILE="conf/mime.types"
-D SERVER_CONFIG_FILE="conf/httpd.conf"
Steps to reproduce
httpd.conf:
ServerRoot "/home/henk/usr/local"
Listen 80
LoadModule authz_core_module modules/mod_authz_core.so
LoadModule watchdog_module modules/mod_watchdog.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_hcheck_module modules/mod_proxy_hcheck.so
LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so
LoadModule unixd_module modules/mod_unixd.so
LoadModule status_module modules/mod_status.so
ServerAdmin you@example.com
ErrorLog "logs/error_log"
LogLevel warn
LogLevel proxy_hcheck:TRACE2 proxy:TRACE2 watchdog:TRACE2
<IfModule log_config_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""
combined
LogFormat "%h %l %u %t \"%r\" %>s %b" common
<IfModule logio_module>
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"
%I %O" combinedio
</IfModule>
CustomLog "logs/access_log" common
</IfModule>
<Proxy balancer://bz54657>
BalancerMember http://hopefully.nonexistant.example.org hcmethod=GET
</Proxy>
<Location "/server-status">
SetHandler server-status
</Location>
ProxyStatus On
request:
curl localhost/server-status?auto 2>/dev/null | grep -e Status -e Uptime
Expected result:
The worker should be marked 'Init HcFl' because its name cannot even be
resolved.
Actual result:
% curl localhost/server-status?auto | grep -i -e status -e uptime
ServerUptimeSeconds: 29
ServerUptime: 29 seconds
Uptime: 29
ProxyBalancer[0]Worker[0]Status: Init Ok
% curl localhost/server-status?auto | grep -i -e status -e uptime
ServerUptimeSeconds: 30
ServerUptime: 30 seconds
Uptime: 30
ProxyBalancer[0]Worker[0]Status: Init HcFl
Suspected issues:
* The first health check is done 30s after startup because the default value of
hcinterval (how often to do a healthcheck) is 30s.
* Backend workers are by default in the state 'Init Ok' until a healthcheck is
run.
Expected behaviour:
* To be on the 'safe' side, backend workers should either not have a state
directly after httpd startup until a healthcheck has run, that state should be
'undetermined' or 'failed' but not 'Ok'.
* The first healthcheck should be run during or immediately after httpd
startup, not hcinterval after httpd startup.
Impact:
The mod_status page could be used for monitoring backend status and alerting if
all backends are down but its content needs to be accurate for that to be
useful.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org