Mailing List Archive

On Sun, Sep 10, 2000 at 09:09:23PM -0600, Alan Robertson wrote:
> Hi,
>
> I've been working on a Python-based test tool for high-availability
> clusters. I'm going to make it work for at least heartbeat and FailSafe.
>
> In the process, I wrote a python version of the heartbeat client-side API.
> It isn't yet in CVS, but I'll put it there as soon as I figure out how and
> where to install Python libraries ;-)
>
> I've been thinking about prototyping some the higher-level membership
> algorithm in Python. It looks like it's about 1/3 the size of the
> corresponding C code for the things I've done so far. [I'm a Python newbie]

Hi,
I'm curiosu to know why you expect the code for python to be so
much smaller than that for C.

No really, I am curious :)

--
Horms

horms wrote:

> Hi,
> I'm curiosu to know why you expect the code for python to be so
> much smaller than that for C.
>
> No really, I am curious :)

It depends who's counting:

* If you count only the python code per se, then it is smaller than the C
code because the C code links in a bunch of stuff. E.g. hello_world.c
compiles to 12KB on my Linux box.
* If you also include the python interpreter plus all the crap it needs to
run, then the python code may well be larger :-)

For a really small footprint, it is important to choose exactly one programming
language, so that you can share all the ancillory crap. If you have part of
your code in C, part in Perl, and part in Python, then you end up with
essentially 3 different standard libraries, i.e. bloat.

Question: who gives a crap about footprint in an HA cluster? No, really: an
HA box is big because of redundant hardware, so why would one care about
shrinking it?

Crispin

--
Crispin Cowan, Ph.D.
Chief Research Scientist, WireX Communications, Inc. http://wirex.com
Free Hardened Linux Distribution: http://immunix.org
Olympics: The Corruption Games

horms wrote:
>
> On Sun, Sep 10, 2000 at 09:09:23PM -0600, Alan Robertson wrote:
> > Hi,
> >
> > I've been working on a Python-based test tool for high-availability
> > clusters. I'm going to make it work for at least heartbeat and FailSafe.
> >
> > In the process, I wrote a python version of the heartbeat client-side API.
> > It isn't yet in CVS, but I'll put it there as soon as I figure out how and
> > where to install Python libraries ;-)
> >
> > I've been thinking about prototyping some the higher-level membership
> > algorithm in Python. It looks like it's about 1/3 the size of the
> > corresponding C code for the things I've done so far. [I'm a Python newbie]
>
> Hi,
> I'm curiosu to know why you expect the code for python to be so
> much smaller than that for C.
>
> No really, I am curious :)

What I said was:

> It looks like it's about 1/3 the size of the
> corresponding C code for the things I've done so far.


This was not a theoretical value, but was more an observation of what had
occured so far more than an expectation for the future. At this point, I've
written the API code, and this observation seems to have roughly held out (a
2 or 3 to 1 ratio of the number of lines). Since I'm translating the same
code from 'C' to python, this is probably a reasonable comparison.

There are several things that seem to have contributed to this:

heartbeat messages map very nicely onto python dictionaries
so using them is very concise and neat.

Since I sling around ASCII messages, they are a bit verbose to handle
in 'C'

In 'C', every single impossible return code has explicit tests and
return logic. In python, I let these become unhandled
exceptions. Maybe this is sloppiness? I do handle things like
bad messages on the wire, where something outside my control
gave me bad data.

There are no declarations in python

I spend no time managing storage in python.

My 'C' code is somewhat object-oriented in style. In python this is
easier and more direct.


Here's an example function from the python API (20 lines):

def nodestatus(self, node):

'''Retrieve the status of the given node'''

msg = hb_api.__api_msg(self, hb_api.NODESTATUS)
msg[ha_msg.F_NODENAME]=node


msg.tofile(self.MsgFIFO)

try:

reply = self.__get_reply()
rc = reply[ha_msg.F_APIRESULT]

if rc == hb_api.FAILURE : return None

return reply[ha_msg.F_STATUS]

except (KeyError, ValueError):
return None


And here's the corresponding 'C' code (55 lines)

static const char *
get_nodestatus(ll_cluster_t* lcl, const char *host)
{
struct ha_msg* request;
struct ha_msg* reply;
const char * result;
const char * status;
static char statbuf[128];
const char * ret;
llc_private_t* pi;

ClearLog();
if (!ISOURS(lcl)) {
ha_log(LOG_ERR, "get_nodestatus: bad cinfo");
return NULL;
}
pi = (llc_private_t*)lcl->ll_cluster_private;

if (!pi->SignedOn) {
ha_log(LOG_ERR, "not signed on");
return NULL;
}

if ((request = hb_api_boilerplate(API_NODESTATUS)) == NULL) {
return NULL;
}
if (ha_msg_add(request, F_NODENAME, host) != HA_OK) {
ha_log(LOG_ERR, "get_nodestatus: cannot add field");
ZAPMSG(request);
return NULL;
}

/* Send message */
if (msg2stream(request, pi->MsgFIFO) != HA_OK) {
ZAPMSG(request);
ha_perror("Can't send message to MsgFIFO");
return NULL;
}
ZAPMSG(request);

/* Read reply... */
if ((reply=read_api_msg(pi)) == NULL) {
ZAPMSG(request);
return NULL;
}
if ((result = ha_msg_value(reply, F_APIRESULT)) != NULL
&& strcmp(result, API_OK) == 0
&& (status = ha_msg_value(reply, F_STATUS)) != NULL) {
strncpy(statbuf, status, sizeof(statbuf));
ret = statbuf;
}else{
ret = NULL;
}
ZAPMSG(reply);

return ret;
}

Crispin Cowan wrote:
>
> horms wrote:
>
> > Hi,
> > I'm curiosu to know why you expect the code for python to be so
> > much smaller than that for C.
> >
> > No really, I am curious :)
>
> It depends who's counting:
>
> * If you count only the python code per se, then it is smaller than the C
> code because the C code links in a bunch of stuff. E.g. hello_world.c
> compiles to 12KB on my Linux box.
> * If you also include the python interpreter plus all the crap it needs to
> run, then the python code may well be larger :-)
>
> For a really small footprint, it is important to choose exactly one programming
> language, so that you can share all the ancillory crap. If you have part of
> your code in C, part in Perl, and part in Python, then you end up with
> essentially 3 different standard libraries, i.e. bloat.
>
> Question: who gives a crap about footprint in an HA cluster? No, really: an
> HA box is big because of redundant hardware, so why would one care about
> shrinking it?


I wrote the python API for two reasons:
1) to learn python
2) to use it as a basis for a test tool.

For my test tool, I don't care about footprint.

In the real world, I care VERY MUCH about footprint because much of the code
of an HA system needs to run locked into memory. Wombat has said that they
took the attitude you describe towards their code, and now it really bogs a
system down for a customer to load it in.

So...

I think python is great for prototyping, and making test tools. I wouldn't
write the heartbeat code in it (and I didn't - just the API). It clearly
makes sense for a GUI and user interface. I reserve judgement on whether
it makes sense to write production versions of other pieces of an HA system
in it. It might make sense for an SNMP agent (for example).

Beyond that, I really don't have enough experience to comment.

-- Alan Robertson
alanr@suse.com

On 2000-09-18T19:21:48,
Crispin Cowan <crispin@wirex.com> said:

> Question: who gives a crap about footprint in an HA cluster? No, really: an
> HA box is big because of redundant hardware, so why would one care about
> shrinking it?

Because you may not be targetting the high-end HA market.

Sincerely,
Lars Marowsky-Brée <lmb@suse.de>
Development HA

--
Perfection is our goal, excellence will be tolerated. -- J. Yahl