Mailing List Archive

* Joshua Rodman <jrodman@example.com> [001122 23:42]:
> I was working on reimplementing the drbd configuration file(s) so that
> they're a bit more comprehensible to the administrator.
>
> Ie. One file would be nice.
> One file which can be read pretty much all at once would be nice.
>
> My first take was a fancy approach involving a parser written in bash.
> It's far too slow so I'm throwing it away.
>
> Before I start again, I'd like to know if this conflicts with other
> efforts. At this point my idea of a file looks something like this:
>
> ---cut--here---
> # comment
>
> group: <groupname> <protocol> <timeout> <anyother group thing>
> <host> <port> <upper_dev> <lower_dev> <other_host_thing>
> <host> <port> <upper_dev> <lower_dev> <other_host_thing>
>
> group: <groupname> <protocol> <timeout> <anyother group thing>
> <host> <port> <upper_dev> <lower_dev> <other_host_thing>
> <host> <port> <upper_dev> <lower_dev> <other_host_thing>
>
> ---cut--here---
>
> It's a sort of drbdtab. The verbosity of comments in the current file are
> nice for the first time working with the file, but very much in the way on
> successive visits. Other things this format should feature are stub
> hosts.. ie
>
> group: <groupname> <protocol> <timeout> <anyother group thing>
> <host> <port> <upper_dev> <lower_dev> <other_host_thing>
> <host> [<port>]
>
> As the local drbd software doesn't need anymore. Fully specified hosts
> are nice for keeping one file across all machines.
>
> I think commands which attempt to start up drbd services should work like
> mount.. matching on groupname, either of the device names, etc.
>
> In the case where two fully specified host lines are given the software
> should be able to determine which one is approrpriate for local use based
> on ip addresses, hostnames, etc. available locally.
>
>
> Please flame gently.
>

Hi Joshua,

to my knowledge there is currenty no other effort to improve
the format of the DRBD configuration files. -- Please base
your work on the latest stuff in CVS.

-Philipp

> Hi Joshua,
>
> to my knowledge there is currenty no other effort to improve
> the format of the DRBD configuration files. -- Please base
> your work on the latest stuff in CVS.

... Hum, .. I changed the code a lot so to help things:

I am reading the file line by line and look for all GLOBAL_ , NODE(X) and
for the current node number. If I find once I strip it from it's PREFIX PART
and pass it through an eval ... See lines 178 + in drbd_common function
setLocal.
This is the only function to change.

Thomas

* Thomas Mangin (thomas.mangin@example.com) [001124 01:25]:
> > Hi Joshua,
> >
> > to my knowledge there is currenty no other effort to improve
> > the format of the DRBD configuration files. -- Please base
> > your work on the latest stuff in CVS.
>
> ... Hum, .. I changed the code a lot so to help things:
>
> I am reading the file line by line

And sourcing it.
I'd recommend doing one or othe other, not both.

> and look for all GLOBAL_ , NODE(X) and for the current node number. If I
> find once I strip it from it's PREFIX PART and pass it through an eval
> ... See lines 178 + in drbd_common function setLocal.
>
> This is the only function to change.

I'll play with it.

-josh
--
_,,wwww,,_
,wØØØ^"^ØØØØØØw,
.dØØØØI,_,Ø***ØØØØb. _____ _____ ______ ____
.ØØØ^ØØ*^'¸ ,¯'ØØØØØ. / ___/__ __/ ___// ____/ / _/___ _____
:ØØØbwww** `7,'ØØØØ: \__ \/ / / /\__ \/ __/ / // __ \/ ___/
`ØØØØØØØ7' dØØØØØØ' ___/ / /_/ /___/ / /___ _/ // / / / /___
`7ØØØØØ' ,* ØØØØØ7' /____/\__,_//____/_____/ /___/_/ /_/\___(_)
`*Øbb. "Ø, ØØØ*'
¯'^^ ^' '¯

> > and look for all GLOBAL_ , NODE(X) and for the current node number. If I
> > find once I strip it from it's PREFIX PART and pass it through an eval
> > ... See lines 178 + in drbd_common function setLocal.
> >
> > This is the only function to change.
>
> I'll play with it.

Okay, I'm not sure what the data is you're expecting for the arbitrary
number of nodes.

Here's a working model of what I think would be useful. I haven't written
the function which determines if a hostname is the host running the code
yet... but pretend I have. I think this should be autodetermined. Tell
me I'm wrong if I am.



-----------drbdtab-----------
# This is a test config file

# this is the first group
group: - A 300 r=30
host1 80 /dev/upper0 /dev/lower0
host2 80 /dev/upper0 /dev/lower0

# This group has multiple members
group: dwarves B 200 r=80
dopey 210 /dev/nb1 /dev/dop1
bashful 220 /dev/nb1 /dev/bas2
sleepy 230 /dev/nb1 /dev/bas2

# This group has an undefined partner
group: localdef C 200 r=80
localhost 210 /dev/nb2 /dev/hda2
partner - - -
-----------drbdtab-----------


This would become part of drbd_common for now, though in the future if
this makes sense, I'd like to make more of it able to access the data
directly. I don't see any advantages to the current resource scheme of
"DRBD0 DRBD1" etc...

I have a configuration reading function and then another which attempts to
mimic setLocal as written (which I do not understand .. :-( )
-----------parsey2-----------
drbdtab=/etc/drbdtab

num_group_params=4 # name, protocol, timout, options
num_host_params=4 # host, port, upperdev, lowerdev

# set LOCAL_DEVICE : set environment values with node configuration info
# $1 unused -- legacy
# $2 'RESOURCE FILE' -- mapped to consecutive group found in /etc/drbdtab
setLocal()
{
local number res_filename_len conf_name

# make sure data is available
if [ ! "$conf_count" ]; then
drbd_conf_read $drbdtab
fi

res_filename_len=${#2} # determine which config being requested
number=${2:$res_filename_len - 1}

conf_name=drbd_config$number # set up for, and
set -- ${!conf_name} # access the data

GLOBAL_PROTOCOL=$2
GLOBAL_TIME=$3
GLOBAL_OPTIONS=$4

shift $num_group_params # move over to the host data
while [ $# -gt 0 ]; do # walk the host list
if this_host $1; then
NODE1_HOST="$1:$2"

NODE1_DEVICE=$3
NODE1_PARTITION=$4
else # assume those that are not us are 'other'
NODE2_HOST="$1:$2" # this can change when I know how the
# data is to be formed
NODE2_DEVICE=$3
NODE2_PARTITION=$4
fi
shift $num_host_params # next host
done
}

this_host()
{
if [ ! "$foober" ]; then
foober=blah
return 0
else
return 1
fi
}

drbd_conf_read()
{
local drbd_conf=$1

local line in_group curr_conf


conf_count=0

check_end_group()
{
if [ "$in_group" ]; then
set -- ${!curr_conf}
if [ $# -ge $(( $num_group_params +
($num_host_params*2) )) ];then # if the group is valid
let conf_count='conf_count + 1' # end the current group
fi
fi
}

# This block runs with redirection from the configfile
{
while read line; do
# skip blank lines and comments
if [ ! "$line" ] || [ "${line:0:1}" == "#" ]; then continue; fi

set -- $line # seperate on whitespace

if [ "$1" == "group:" ]; then # If it's a group...
shift # drop the identifier

check_end_group

if [ $# -ne $num_group_params ]; then
unset in_group # skip invalid groups
echo "invalid group defn: $*, skipping..." >&2
else
curr_conf=drbd_config$conf_count
eval $curr_conf=\"$*\" # begin a new group
in_group=true
fi
elif [ $# -eq $num_host_params ]; then # If it's a host...
if [ "$in_group" ]; then
eval $curr_conf=\"\$$curr_conf \$*\"
else
echo "host defn outside group: $*, skipping..." >&2
fi
else # All else are errors
echo "invalid host defn: $*, skipping..." >&2
fi
done
} < $drbd_conf

check_end_group
}
-----------parsey2-----------

--
_,,wwww,,_
,wØØØ^"^ØØØØØØw,
.dØØØØI,_,Ø***ØØØØb. _____ _____ ______ ____
.ØØØ^ØØ*^'¸ ,¯'ØØØØØ. / ___/__ __/ ___// ____/ / _/___ _____
:ØØØbwww** `7,'ØØØØ: \__ \/ / / /\__ \/ __/ / // __ \/ ___/
`ØØØØØØØ7' dØØØØØØ' ___/ / /_/ /___/ / /___ _/ // / / / /___
`7ØØØØØ' ,* ØØØØØ7' /____/\__,_//____/_____/ /___/_/ /_/\___(_)
`*Øbb. "Ø, ØØØ*'
¯'^^ ^' '¯

> Okay, I'm not sure what the data is you're expecting for the arbitrary
> number of nodes.
>
> Here's a working model of what I think would be useful. I haven't written
> the function which determines if a hostname is the host running the code
> yet... but pretend I have. I think this should be autodetermined. Tell
> me I'm wrong if I am.

I will have to read the thread about your new configuration structure ...
But setLocal set global environnement variable extracted from the drbd
configuration file.

I used the NODE(NUMBER) as a filter on the line to only get configuration
relative to the
good node.

Once found I was stripping the line from its NODE part :
ie : NODE(NUMBER)_IF="192.168.0.1" become IF="192.168.0.1"
I was then running an eval on the string extracted to set the IF value.

Same with GLOBAL_ ...

This looks better for me as sample ;*)

> -----------drbdtab-----------
# This is a test config file

# this is the first group

#group: name protocol timeout options
group: - A 300 r=30

#host port upperdev lowerdev
host1 80 /dev/upper0 /dev/lower0
host2 80 /dev/upper0 /dev/lower0

# This group has multiple members

#group: name protocol timeout options
group: dwarves B 200 r=80

#host port upperdev lowerdev
dopey 210 /dev/nb1 /dev/dop1
bashful 220 /dev/nb1 /dev/bas2
sleepy 230 /dev/nb1 /dev/bas2

# This group has an undefined partner

#group: name protocol timeout options
group: localdef C 200 r=80

#host port upperdev lowerdev
localhost 210 /dev/nb2 /dev/hda2
partner - - -

> -----------drbdtab-----------

> This would become part of drbd_common for now, though in the future if
> this makes sense, I'd like to make more of it able to access the data
> directly. I don't see any advantages to the current resource scheme of
> "DRBD0 DRBD1" etc...

I like you new format.

> I have a configuration reading function and then another which attempts to
> mimic setLocal as written (which I do not understand .. :-( )

> -----------parsey2-----------
> drbdtab=/etc/drbdtab
>
> num_group_params=4 # name, protocol, timout, options
> num_host_params=4 # host, port, upperdev, lowerdev
>
> # set LOCAL_DEVICE : set environment values with node configuration info
> # $1 unused -- legacy
> # $2 'RESOURCE FILE' -- mapped to consecutive group found in /etc/drbdtab
> setLocal()
> {
> local number res_filename_len conf_name
>
> # make sure data is available
> if [ ! "$conf_count" ]; then
> drbd_conf_read $drbdtab
> fi
>
> res_filename_len=${#2} # determine which config being
requested
> number=${2:$res_filename_len - 1}
>
> conf_name=drbd_config$number # set up for, and
> set -- ${!conf_name} # access the data
>

Your must remove the GLOCAL_ prefix .. But I am ready to discuss to change
the reste of
the code instead if we can find a good naming convention.

> GLOBAL_PROTOCOL=$2
> GLOBAL_TIME=$3
> GLOBAL_OPTIONS=$4

Same you must set DEVICE and PARTITION

> shift $num_group_params # move over to the host data
> while [ $# -gt 0 ]; do # walk the host list
> if this_host $1; then
> NODE1_HOST="$1:$2"
>
> NODE1_DEVICE=$3
> NODE1_PARTITION=$4
> else # assume those that are not us are
'other'
> NODE2_HOST="$1:$2" # this can change when I know how the
> # data is to be formed
> NODE2_DEVICE=$3
> NODE2_PARTITION=$4
> fi
> shift $num_host_params # next host
> done
> }

foober ???
foobar ???

> this_host()
> {
> if [ ! "$foober" ]; then
> foober=blah
> return 0
> else
> return 1
> fi
> }


> drbd_conf_read()
> {
> local drbd_conf=$1
>
> local line in_group curr_conf
>
> conf_count=0
>
> check_end_group()
> {
> if [ "$in_group" ]; then
> set -- ${!curr_conf}
> if [ $# -ge $(( $num_group_params +
> ($num_host_params*2) )) ];then # if the group is
valid
> let conf_count='conf_count + 1' # end the current
group
> fi
> fi
> }
>
> # This block runs with redirection from the configfile
> {
> while read line; do
> # skip blank lines and comments

Can you can't skip " # Comment" with this code too ?

> if [ ! "$line" ] || [ "${line:0:1}" == "#" ]; then continue;
fi
>
> set -- $line # seperate on
whitespace
>
> if [ "$1" == "group:" ]; then # If it's a
group...
> shift # drop the
identifier
>
> check_end_group
>
> if [ $# -ne $num_group_params ]; then
> unset in_group # skip invalid
groups
> echo "invalid group defn: $*, skipping..." >&2
> else
> curr_conf=drbd_config$conf_count
> eval $curr_conf=\"$*\" # begin a new group
> in_group=true
> fi
> elif [ $# -eq $num_host_params ]; then # If it's a host...
> if [ "$in_group" ]; then
> eval $curr_conf=\"\$$curr_conf \$*\"
> else
> echo "host defn outside group: $*, skipping..." >&2
> fi
> else # All else are
errors
> echo "invalid host defn: $*, skipping..." >&2
> fi
> done
> } < $drbd_conf
>
> check_end_group
> }


I like your script coding style .. You are visibly using bash more than me.
I will try to look more deeply in it today ...

Thomas

* Thomas Mangin (thomas.mangin@example.com) [001128 01:48]:
> > Okay, I'm not sure what the data is you're expecting for the arbitrary
> > number of nodes.
> >
> > Here's a working model of what I think would be useful. I haven't written
> > the function which determines if a hostname is the host running the code
> > yet... but pretend I have. I think this should be autodetermined. Tell
> > me I'm wrong if I am.
>
> I used the NODE(NUMBER) as a filter on the line to only get configuration
> relative to the
> good node.
>
> Once found I was stripping the line from its NODE part :
> ie : NODE(NUMBER)_IF="192.168.0.1" become IF="192.168.0.1"
> I was then running an eval on the string extracted to set the IF value.
>
> Same with GLOBAL_ ...

Ok. I wasn't sure exactly.
That's easy enough to change in mine :-)

> This looks better for me as sample ;*)
>
> > -----------drbdtab-----------
> # This is a test config file
>
> # this is the first group
>
> #group: name protocol timeout options
> group: - A 300 r=30
>
> #host port upperdev lowerdev
> host1 80 /dev/upper0 /dev/lower0
> host2 80 /dev/upper0 /dev/lower0
> > -----------drbdtab-----------

> Your must remove the GLOCAL_ prefix .. But I am ready to discuss to
> change the reste of the code instead if we can find a good naming
> convention.

Well basically there's two options:

1) pull the data into global ENVs, as you do.
2) have everything comprehend something more complex, like lists, as I am
doing inside the config reader, or possibly arrays, if whitespace
sensitivity was needed, which I've tried to avoid.

#2 makes for more flexible but more complex code.

> foober ???
> foobar ???

This code is not meaningful! :D
foobar was already in my env space when i was writing this.. so.

I will write it correctly today.
> > this_host()
> > {
> > if [ ! "$foober" ]; then
> > foober=blah
> > return 0
> > else
> > return 1
> > fi
> > }

> I like your script coding style .. You are visibly using bash more than me.
> I will try to look more deeply in it today ...

Thanks. I started with the O'Reilly book, but have just learned along the
way. It's an ugly little language, but you can make it pretty readable if
you stay within its problem domain.

I do write stuff that's pretty heavily bash-dependent, which isn't
necessarily ideal in all cases, but I prefer it to dropping bits of awk,
sed, grep, and friends everywhere. Which can really decrease readability
fast.

> Thomas

-josh

[...]
> > Your must remove the GLOCAL_ prefix .. But I am ready to discuss to
> > change the reste of the code instead if we can find a good naming
> > convention.
>
> Well basically there's two options:
>
> 1) pull the data into global ENVs, as you do.

I would perhaps like to rename the env variable in order to make them more
readable something like CONFIG_IF, CONFIG_XXX ..

> 2) have everything comprehend something more complex, like lists, as I am
> doing inside the config reader, or possibly arrays, if whitespace
> sensitivity was needed, which I've tried to avoid.
>
> #2 makes for more flexible but more complex code.

The simplest the better ;*)
We will still be able to go toward list if needed later ..

> > foober ???
> > foobar ???
>
> This code is not meaningful! :D
> foobar was already in my env space when i was writing this.. so.

I was hoping so...

> > I like your script coding style .. You are visibly using bash more than
me.
> > I will try to look more deeply in it today ...
>
> Thanks. I started with the O'Reilly book, but have just learned along the
> way. It's an ugly little language, but you can make it pretty readable if
> you stay within its problem domain.

Yes but is pretty usefull when you consider it is coming whith EVERY unix
system.

> I do write stuff that's pretty heavily bash-dependent, which isn't
> necessarily ideal in all cases, but I prefer it to dropping bits of awk,
> sed, grep, and friends everywhere. Which can really decrease readability
> fast.

I was more sh + sed + grep without awk ..
But I can leave with bash dependancy without problem.

Thomas