Mailing List Archive

[mod_backhand-users] Balancing many sites with BIG-IP and mod_backhand...
In light of the recent BIG-IP conversation, I'm wondering how other
people handle this scenario:

Internet
|
|
| [.BIG-IP listens on many public IP addresses that maps IPs to ports]
V
BIG-IP
|
|
| [Only a dozen or so web servers listening on 8000-8256]
V
Small handful of servers all running mod_backhand

A few things to note:

1) All servers use the same port for a host (8000 is always site
foo.com, 8001 is always bar.com).

2) Some servers may not be listening to port 8001.

3) If a server is listening on a port (ex: 8001) it is a valid server
for the domain (ex: bar.com).


The only thing that I can think of is to have every server listen on
all the same ports and then double check that backhand will forward
from port X to port X (and not from port X to port 80), but I don't
think it does (would break the model of incoming data on port 443 ->
decrypt by mod_ssl -> backhand to port 80 on other server). Anyway,
if someone has any ideas/experiences with this, I'd be curious to know
how you've handled it. -sc
[mod_backhand-users] Balancing many sites with BIG-IP and mod_backhand... [ In reply to ]
If I were to do this, I would modify the server stats structure to
include a 256 bit vector and augment the function that fills out the
servers stats to mark the bits for the domains that it serves.

I know someone else did this for NFS mounts. The had a handful of
machines and each machine had some subset of the available NFS
mountpoints mounted. Obviously, if the machine didn't have it mounted,
it couldn't serve the data, so NFS mounts were a "resource". And custom
candidacy function was written to check the if the bit was set based on
the request.

In this case it would check the appropriate bit field for the port.

On Sunday, October 28, 2001, at 07:41 PM, Sean Chittenden wrote:

> In light of the recent BIG-IP conversation, I'm wondering how other
> people handle this scenario:
>
> Internet
> |
> |
> | [.BIG-IP listens on many public IP addresses that maps IPs to
> ports]
> V
> BIG-IP
> |
> |
> | [Only a dozen or so web servers listening on 8000-8256]
> V
> Small handful of servers all running mod_backhand
>
> A few things to note:
>
> 1) All servers use the same port for a host (8000 is always site
> foo.com, 8001 is always bar.com).
>
> 2) Some servers may not be listening to port 8001.
>
> 3) If a server is listening on a port (ex: 8001) it is a valid server
> for the domain (ex: bar.com).
>
>
> The only thing that I can think of is to have every server listen on
> all the same ports and then double check that backhand will forward
> from port X to port X (and not from port X to port 80), but I don't
> think it does (would break the model of incoming data on port 443 ->
> decrypt by mod_ssl -> backhand to port 80 on other server). Anyway,
> if someone has any ideas/experiences with this, I'd be curious to know
> how you've handled it. -sc
>
> _______________________________________________
> backhand-users mailing list
> backhand-users@lists.backhand.org
> http://lists.backhand.org/mailman/listinfo/backhand-users
>
--
Theo Schlossnagle
1024D/82844984/95FD 30F1 489E 4613 F22E 491A 7E88 364C 8284 4984
2047R/33131B65/71 F7 95 64 49 76 5D BA 3D 90 B9 9F BE 27 24 E7
[mod_backhand-users] Balancing many sites with BIG-IP and mod_backhand... [ In reply to ]
This is a multi-part message in MIME format.
--------------030908000607010809050809
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit

Theo Schlossnagle wrote:

> If I were to do this, I would modify the server stats structure to
> include a 256 bit vector and augment the function that fills out the
> servers stats to mark the bits for the domains that it serves.
>
> I know someone else did this for NFS mounts. The had a handful of
> machines and each machine had some subset of the available NFS
> mountpoints mounted. Obviously, if the machine didn't have it mounted,
> it couldn't serve the data, so NFS mounts were a "resource". And custom
> candidacy function was written to check the if the bit was set based on
> the request.
>
> In this case it would check the appropriate bit field for the port.


I am doing something very similar to this: I added a
"BackhandSetCapabilities" config directive and a "byCapability"
candadicy function. They both use a long integer I've added to the
server stats structure. The "BackhandSetCapabilities" sets bits in the
out-going server stats field, and the byCapability looks for a specific
set of bits on the incoming structures.

A patch is attached that should work off the latest CVS source. Also,
the "BackhandSetCapabilities" directive takes a bit position as its
argument (basically, an integer from 0 to 31), but the byCapability
function takes an integer mask (like "42" to look for servers with bits
1, 3, and 5 set -- 2^1 + 2^3 + 2^5 = 2+8+32 = 42). Not a particularly
intuitive system, I know. But it's a work in progress.

Anyway...you can set bits according to certian conditions on the servers
-- things like: "has perl", "has php", etc. -- and then send requests to
servers which support the right stuff.

There you go. Hope that is somewhat helpful.

--
Michael Cramer / Consultant, PBS Interactive
mcramer@pbs.org

--------------030908000607010809050809
Content-Type: text/plain;
name="capabilities2.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="capabilities2.patch"

Index: mod_backhand/back_util.c
===================================================================
RCS file: /storage/cvs/jesus/mod_backhand/back_util.c,v
retrieving revision 1.16
diff -u -r1.16 back_util.c
--- mod_backhand/back_util.c 2001/10/23 20:25:17 1.16
+++ mod_backhand/back_util.c 2001/10/26 15:05:53
@@ -169,7 +169,7 @@
<TD align=center><B># ready servers/<BR># total servers</B></TD>\
<TD align=center><B>~ms/req [#req]</B></TD>\
<TD align=right><B>Arriba</B></TD><TD align=center><B># CPUs</B></TD>\
-<TD align=right><B>Load/HWM</B></TD><TD align=right><B>CPU Idle</B></TD></TR>\n",
+<TD align=right><B>Load/HWM</B></TD><TD align=right><B>CPU Idle</B></TD><TD><B>Capabilities</B></TD></TR>\n",
r);
for(i=0;i<MAXSERVERS;i++) {
tempport = ntohs(serverstats[i].contact.sin_port);
@@ -183,7 +183,7 @@
<TD align=right>%d</TD><TD align=center>%s</TD><TD align=right>%d</TD>\
<TD align=left>%d</TD><TD align=center>%d/%d</TD><TD align=center>%d [%d]</TD>\
<TD align=right>%d</TD>\
-<TD align=center>%d</TD><TD align=right>%f/%d</TD><TD align=right>%f</TD></TR>\n",
+<TD align=center>%d</TD><TD align=right>%f/%d</TD><TD align=right>%f</TD><TD align=center>0x%4.4x</TD></TR>\n",
(!is_alive(i,SERVER_TIMEOUT,now))?"#ff4444":
(i%2)?"#aaaaaa":"#cccccc",
i, /* Entry */
@@ -200,7 +200,8 @@
serverstats[i].ncpu, /* Number of CPUs */
serverstats[i].load/1000.0, /* load on machine */
serverstats[i].load_hwm/1000, /* load_hwm for cluster */
- serverstats[i].cpu/1000.0); /* CPU utilization on machine*/
+ serverstats[i].cpu/1000.0, /* CPU utilization on machine*/
+ serverstats[i].capabilities); /* capabilities */
}
ap_rputs("</TABLE>\n", r);
} else {
Index: mod_backhand/builtins.c
===================================================================
RCS file: /storage/cvs/jesus/mod_backhand/builtins.c,v
retrieving revision 1.24
diff -u -r1.24 builtins.c
--- mod_backhand/builtins.c 2001/05/30 21:13:35 1.24
+++ mod_backhand/builtins.c 2001/10/26 15:05:53
@@ -192,6 +192,21 @@
return mycount;
}

+int byCapability(request_rec *r, ServerSlot *servers, int *n, char *arg) {
+ int i, mycount, capabilities;
+ if(!arg) return -1;
+
+ capabilities = strtol(arg, (char**)NULL, 10);
+
+ mycount=0;
+ for(i=0;i<*n;i++)
+ if( (serverstats[servers[i].id].capabilities & capabilities) == capabilities )
+ servers[mycount++] = servers[i];
+
+ *n=mycount;
+ return mycount;
+}
+
int byRandom(request_rec *r, ServerSlot *servers, int *n, char *arg) {
/* Returns n or n server as a random permutation of the input
vector */
Index: mod_backhand/mod_backhand.c
===================================================================
RCS file: /storage/cvs/jesus/mod_backhand/mod_backhand.c,v
retrieving revision 1.44
diff -u -r1.44 mod_backhand.c
--- mod_backhand/mod_backhand.c 2001/10/02 19:18:58 1.44
+++ mod_backhand/mod_backhand.c 2001/10/26 15:05:54
@@ -441,6 +441,21 @@
}
*/

+static const char *cmd_backhand_capabilities(cmd_parms *cmd,
+ void *mconfig, char *word1) {
+ static char *sr_error = "BackhandSetCapabilities BitPosition";
+ mbcfg *cfg = (mbcfg *) mconfig;
+ mbcfg *scfg = (mbcfg *) ap_get_module_config(cmd->server->module_config, &backhand_module);
+
+ /* OR together multiple BackhandSetCapabilities directives
+ so you don't have to do the math yourself. Just put one
+ directive near each of the "capabilities" in your Apache
+ config file and this takes care of adding it up. */
+ scfg->capabilities |= (1<<strtol(word1, (char**)NULL, 10));
+
+ return NULL;
+}
+
static int build_request_headers(void *nuttin,
const char *key, const char *val) {
char buffer[DEFAULT_LIMIT_REQUEST_FIELDSIZE + 2];
@@ -1206,6 +1221,7 @@
cfg->connectionpools = MBCP_INHERIT;
cfg->selfredirect = MBSR_INHERIT;
cfg->dfunc = NULL;
+ cfg->capabilities = 0;
return (void *) cfg;
}

@@ -1222,6 +1238,7 @@
merged_config->cmode =
(s1conf->cmode == s2conf->cmode) ? s1conf->cmode : CONFIG_MODE_COMBO;
merged_config->local = s2conf->local;
+ merged_config->capabilities = s2conf->capabilities;
merged_config->congenital = (s1conf->congenital | s1conf->local);
merged_config->loc = ap_pstrdup(p, s2conf->loc);
merged_config->loglevel =
@@ -1521,6 +1538,15 @@
ACCESS_CONF, /* where available */
TAKE23, /* arguments */
"Backhand directive - sharedlib function [arg]"
+ /* directive description */
+ },
+ {
+ "BackhandSetCapabilities", /* directive name */
+ cmd_backhand_capabilities, /* config action routine */
+ NULL, /* argument to include in call */
+ RSRC_CONF, /* where available */
+ TAKE1, /* arguments */
+ "BackhandSetCapabilities directive - set the capabilities flags for this server"
/* directive description */
},
{NULL}
Index: mod_backhand/mod_backhand.h
===================================================================
RCS file: /storage/cvs/jesus/mod_backhand/mod_backhand.h,v
retrieving revision 1.22
diff -u -r1.22 mod_backhand.h
--- mod_backhand/mod_backhand.h 2001/08/05 02:23:14 1.22
+++ mod_backhand/mod_backhand.h 2001/10/26 15:05:54
@@ -104,6 +104,7 @@
#define MBLL_DCSNALL (MBLL_DCSN1|MBLL_DCSN2|MBLL_DCSN3|MBLL_DCSN4)
char *loc;
struct cpd *dfunc;
+ int capabilities;
} mbcfg;

struct lognames {
Index: mod_backhand/platform.c
===================================================================
RCS file: /storage/cvs/jesus/mod_backhand/platform.c,v
retrieving revision 1.10
diff -u -r1.10 platform.c
--- mod_backhand/platform.c 2001/04/22 16:49:51 1.10
+++ mod_backhand/platform.c 2001/10/26 15:05:54
@@ -112,7 +112,11 @@
float ourload=0.0;
float states[CPUSTATES] = {0.0}, ourcpu=0.0;
int i, nstates, tmem, amem;
+ mbcfg *scfg;

+ scfg = our_sconfig(s);
+ ss->capabilities = scfg->capabilities;
+
/* Fill out hostname */
strncpy(ss->hostname, s->server_hostname, 40);
ss->hostname[39]='\0';
@@ -326,6 +330,10 @@
kstat_io_t dstat;
kstat_named_t *kn;
struct kstat_list *k;
+ mbcfg *scfg;
+
+ scfg = our_sconfig(s);
+ ss->capabilities = scfg->capabilities;

/* CPU stuff */
total=0.0;
@@ -452,6 +460,10 @@
vm_statistics_data_t vminfo;
host_load_info_data_t loadinfo;
register host_cpu_load_info_t temp;
+ mbcfg *scfg;
+
+ scfg = our_sconfig(s);
+ ss->capabilities = scfg->capabilities;
count = HOST_VM_INFO_COUNT;
kr = host_statistics(mach_host_self(), HOST_VM_INFO,
(host_info_t)&vminfo, &count);
@@ -552,6 +564,10 @@
float ourcpu=0.0;
struct loadavg ourload;
struct vmtotal ourmem;
+ mbcfg *scfg;
+
+ scfg = our_sconfig(s);
+ ss->capabilities = scfg->capabilities;
#ifdef KERN_CPUSTATS
mib[0] = CTL_KERN;
mib[1] = KERN_CPUSTATS;
Index: mod_backhand/serverstat.h
===================================================================
RCS file: /storage/cvs/jesus/mod_backhand/serverstat.h,v
retrieving revision 1.5
diff -u -r1.5 serverstat.h
--- mod_backhand/serverstat.h 2000/04/08 05:12:51 1.5
+++ mod_backhand/serverstat.h 2001/10/26 15:05:54
@@ -33,6 +33,7 @@
int amem; /* available memory */
int numbacked; /* Number of requests backhanded */
int tatime; /* average ms to serve a request (backhanded) */
+ int capabilities; /* a bitmap of user defined flags */
} serverstat;

/* NOTE!!!!! If we decide to add a short value into the serverstat
@@ -52,6 +53,7 @@
(a).amem = f((a).amem);\
(a).numbacked = f((a).numbacked);\
(a).tatime = f((a).tatime);\
+ (a).capabilities = f((a).capabilities);\
}

/* These are like hton(s|l) and ntoh(s|l) except for serverstats You

--------------030908000607010809050809--