Mailing List Archive

Hi,
On 7 Jan 2004, at 16:51, Brian Blood wrote:

> for 2.0, can we add a parameter to have a prefix for dbmail owned
> tables?
>
> Or, even better, have the table names be abstracted out of the code
> conf
> options.
Sounds like a good idea. I'd like to postpone it to a later time
though. I think we should first get 2.0 out of the door and add new
features after that.

But of course, if you need this and like to start working on it, that
would be greatly appreciated :)

Ilja
--
IC&S
Stadhouderslaan 57
3583 JD Utrecht
telnr. 030-6355730
faxnr. 030-6355731

PGP-key:
http://www.ic-s.nl/keys/ilja.txt

I disagree on getting 2.0 out the door first. Table prefixes are critical
infrastructure for people who may have to share a single database with tables
for different applications. Since we may already need a migration script /
program to get 1.2.x people up to 2.0, table prefixes won't hurt much.

Aaron


Ilja Booij <ilja@ic-s.nl> said:

> Hi,
> On 7 Jan 2004, at 16:51, Brian Blood wrote:
>
> > for 2.0, can we add a parameter to have a prefix for dbmail owned
> > tables?
> >
> > Or, even better, have the table names be abstracted out of the code
> > conf
> > options.
> Sounds like a good idea. I'd like to postpone it to a later time
> though. I think we should first get 2.0 out of the door and add new
> features after that.
>
> But of course, if you need this and like to start working on it, that
> would be greatly appreciated :)
>
> Ilja
> --
> IC&S
> Stadhouderslaan 57
> 3583 JD Utrecht
> telnr. 030-6355730
> faxnr. 030-6355731
>
> PGP-key:
> http://www.ic-s.nl/keys/ilja.txt
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

Aaron Stone writes:

> I disagree on getting 2.0 out the door first. Table prefixes are critical
> infrastructure for people who may have to share a single database with tables
> for different applications. Since we may already need a migration script /
> program to get 1.2.x people up to 2.0, table prefixes won't hurt much.


hm I disagree. if someone is going to add prefix to the tables, you better
abstract them through the conf file.. however mind then fact that the conf
file shouldn't be too overwhelming otherwise it will get useless and bulky a
bit like sendmail cf and others..

best scenario will be to have only DB settings and table names in the conf
file.. and moving all the bulk like AUTO_* EFFECTIVE_* BIND into the
database , there's no point for them being there, anyway without the DB
dbmail* is useless, also consider the fact the dbmail-smtp doesn't even know
what DEFER
is (at least for postfix).(well LMTP will solve that - more abstraction!)

but that still aint the best solution around, it can be even more
abstract... nevertheless sometimes being over abstract/general is considered
harmfull.

and after that I'd like to see somethin like FIFO-like dbmail/ctl
to do echo 'table users=someprefix_users' > dbmail/ctl.

for what is worth it i think Ilja should put more effort on core rather than
features like filtering or different auth mechanisms (not that they wont be
usefull).


Also can you get rid of my_malloc() and my_free(). those were good
10-15y ago, now they cause more problems then they solve..
its bad practice.


cheers,
-l


>
>
> Ilja Booij <ilja@ic-s.nl> said:
>
>> Hi,
>> On 7 Jan 2004, at 16:51, Brian Blood wrote:
>>
>> > for 2.0, can we add a parameter to have a prefix for dbmail owned
>> > tables?
>> >
>> > Or, even better, have the table names be abstracted out of the code
>> > conf
>> > options.
>> Sounds like a good idea. I'd like to postpone it to a later time
>> though. I think we should first get 2.0 out of the door and add new
>> features after that.
>>
>> But of course, if you need this and like to start working on it, that
>> would be greatly appreciated :)

lou@0xffff.org wrote:

> hm I disagree. if someone is going to add prefix to the tables, you
> better
> abstract them through the conf file..


I think this is moving towards over-engineered. The only possible thing
that you might want to put into a config file is the table prefix. I
feel that this is a solution in need of a problem. Have we actually
heard from people that they are having name collision problems becuase
they share their dbmail database with other applications? I would think
that anyone setting up dbmail has to configure the MTA and other asorted
items, the idea that they would be constrained to one database doesn't
seem logical to me. If this is really a problem, perhaps it would be
better to allow the user to specify a schema to put dbmail tables in
(does mysql support schema's?).

On 1/7/04 4:44 PM, "Matthew T. O'Connor" <matthew@zeut.net> wrote:

>> hm I disagree. if someone is going to add prefix to the tables, you
>> better
>> abstract them through the conf file..
>
>
> I think this is moving towards over-engineered. The only possible thing
> that you might want to put into a config file is the table prefix.

My particular reason for wanting a prefix was for organizational purposes.

I would personally use something like MAIL or dbmail_ or something else so
that I could keep all the dbmail related tables grouped together.

Another benefit would be to make it a LOT easier to have only my tables
replicated in MySQL using the wildcard pattern in the conf file.

In one particular project, we will likely have the aliases and users tables
managed one on server and replicated over to the actual dbmail server.


> I
> feel that this is a solution in need of a problem. Have we actually
> heard from people that they are having name collision problems becuase
> they share their dbmail database with other applications? I would think
> that anyone setting up dbmail has to configure the MTA and other asorted
> items, the idea that they would be constrained to one database doesn't
> seem logical to me.

I agree with your sense of logic here. I think we need a prefix AT LEAST.
Abstracting the table names out may not be necessary in the real world.


> If this is really a problem, perhaps it would be
> better to allow the user to specify a schema to put dbmail tables in
> (does mysql support schema's?).

not that I'm aware of.


Brian

> best scenario will be to have only DB settings and table
> names in the conf
> file.. and moving all the bulk like AUTO_* EFFECTIVE_* BIND into the
> database , there's no point for them being there, anyway
> without the DB
> dbmail* is useless, also consider the fact the dbmail-smtp
> doesn't even know
> what DEFER
> is (at least for postfix).(well LMTP will solve that - more
> abstraction!)

Noooo! Settings like log-level, bindaddress, ports, etc need to be able to
be specified on a per-client basis (so via command line or config file
selected by daemon), not on a global scale only (although happy with
defaults accessible via database provided they can be overridden by a local
config).

/Mark

Matthew T. O'Connor writes:

> lou@0xffff.org wrote:
>
>> hm I disagree. if someone is going to add prefix to the tables, you
>> better
>> abstract them through the conf file..
>
>
> I think this is moving towards over-engineered. The only possible thing
> that you might want to put into a config file is the table prefix. I
> feel that this is a solution in need of a problem. Have we actually heard
> from people that they are having name collision problems becuase they
> share their dbmail database with other applications? I would think that
> anyone setting up dbmail has to configure the MTA and other asorted items,
> the idea that they would be constrained to one database doesn't seem
> logical to me. If this is really a problem, perhaps it would be better to
> allow the user to specify a schema to put dbmail tables in (does mysql
> support schema's?).

not exactly, I assume you didnt understand my point, problems are easily
created, but how many of them can you solve with one solution.
its like writing foo1 and then redesigning it 100 times to get
foo-100.21.22.2release. there's a difference in creating design which can
easily accomodate any of the new features nativaly and there's re-designing
the software to accomodate the new features... which one will it be?

currently the 2nd is happening.

cheers

Mark Mackay - Orcon writes:

>> best scenario will be to have only DB settings and table
>> names in the conf
>> file.. and moving all the bulk like AUTO_* EFFECTIVE_* BIND into the
>> database , there's no point for them being there, anyway
>> without the DB
>> dbmail* is useless, also consider the fact the dbmail-smtp
>> doesn't even know
>> what DEFER
>> is (at least for postfix).(well LMTP will solve that - more
>> abstraction!)
>
> Noooo! Settings like log-level, bindaddress, ports, etc need to be able to
> be specified on a per-client basis (so via command line or config file
> selected by daemon), not on a global scale only (although happy with
> defaults accessible via database provided they can be overridden by a local
> config).

right. well that could be solved by having prefixes and still end up with
%{DB,SMTP,POP,IMAP}_TRACE_LEVEL=%d

same for having a multihomed box. (well in both case you have to hack
dbmail.h to get it to use different pathnames)

and when you have 2 different hosts accessing the same db
dbmail1:x.x.x.x -> conftable=xxxx_conf
dbmail2:y.y.y.y -> conftable=yyyy_conf
and myprefix_users, etc

and i'm sure you can think of many other ways how you can utilize
abstract tablenames & prefixes ..

hope my point is a bit clearer now.

cheers

So trying to put together some of the ideas from the past few emails, here's
what I'm thinking:

dbmail.conf:
Is a configuration file in /etc or wherever.
- database information
- logging info
- hostname / clustering info

dbmail_conf:
Is a table in the database.
- Has detailed config info in host/key/value pairs:

CREATE TABLE dbmail_conf (
hostname VARCHAR(128) NOT NULL,
key VARCHAR(32) NOT NULL,
value TEXT NULL,
UNIQUE INDEX (hostname, key)
)

I suspect that a hostname value of '*' would be useful for basic, common
config options and for non-clustered installations.

By doing this, we provide a method for multiple hosts to connect to a common
database, retrieve host-specific configuration information, and begin
operation as a member of the cluster.

Regarding clustered operations, I've done a little bit of research, and
literally the only thing that needs to be done is to remove the auto_increment
columns from every table and to provide for a unique, or at least
statistically unique, ID number for messages.

Aaron


lou@0xffff.org said:

> Mark Mackay - Orcon writes:
>
> >> best scenario will be to have only DB settings and table
> >> names in the conf
> >> file.. and moving all the bulk like AUTO_* EFFECTIVE_* BIND into the
> >> database , there's no point for them being there, anyway
> >> without the DB
> >> dbmail* is useless, also consider the fact the dbmail-smtp
> >> doesn't even know
> >> what DEFER
> >> is (at least for postfix).(well LMTP will solve that - more
> >> abstraction!)
> >
> > Noooo! Settings like log-level, bindaddress, ports, etc need to be able to
> > be specified on a per-client basis (so via command line or config file
> > selected by daemon), not on a global scale only (although happy with
> > defaults accessible via database provided they can be overridden by a local
> > config).
>
> right. well that could be solved by having prefixes and still end up with
> %{DB,SMTP,POP,IMAP}_TRACE_LEVEL=%d
>
> same for having a multihomed box. (well in both case you have to hack
> dbmail.h to get it to use different pathnames)
>
> and when you have 2 different hosts accessing the same db
> dbmail1:x.x.x.x -> conftable=xxxx_conf
> dbmail2:y.y.y.y -> conftable=yyyy_conf
> and myprefix_users, etc
>
> and i'm sure you can think of many other ways how you can utilize
> abstract tablenames & prefixes ..
>
> hope my point is a bit clearer now.
>
> cheers
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

I really like having a local config file [.much better than the old days of
having to compile the database name in, etc]; and I like being able to just
'vi' a file to enable logging, adjust process levels on one of the hosts,
etc.

If you want to do database configs as well, then the syntax below would work
-- but rather than via hostname, I'd recommend making it some form of
'clientid/configid' which could be supplied on command lines (e.g.
./dbmail-pop3d -configkey DEFAULT) That would allow configs to be
experimented with and changed between without having to affect other daemons
on the same host, etc.

Personally I just say keep the config files :)

Cheers,
Mark.

> dbmail.conf:
> Is a configuration file in /etc or wherever.
> - database information
> - logging info
> - hostname / clustering info
>
> dbmail_conf:
> Is a table in the database.
> - Has detailed config info in host/key/value pairs:
>
> CREATE TABLE dbmail_conf (
> hostname VARCHAR(128) NOT NULL,
> key VARCHAR(32) NOT NULL,
> value TEXT NULL,
> UNIQUE INDEX (hostname, key)
> )

We could also do both (config on file and in the db) and keep them in
sync. This enables easier administration through interfaces on the DB.


On 8-jan-04, at 9:44, Mark Mackay - Orcon wrote:

> I really like having a local config file [.much better than the old
> days of
> having to compile the database name in, etc]; and I like being able to
> just
> 'vi' a file to enable logging, adjust process levels on one of the
> hosts,
> etc.
>
> If you want to do database configs as well, then the syntax below
> would work
> -- but rather than via hostname, I'd recommend making it some form of
> 'clientid/configid' which could be supplied on command lines (e.g.
> ../dbmail-pop3d -configkey DEFAULT) That would allow configs to be
> experimented with and changed between without having to affect other
> daemons
> on the same host, etc.
>
> Personally I just say keep the config files :)
>
> Cheers,
> Mark.
>
>> dbmail.conf:
>> Is a configuration file in /etc or wherever.
>> - database information
>> - logging info
>> - hostname / clustering info
>>
>> dbmail_conf:
>> Is a table in the database.
>> - Has detailed config info in host/key/value pairs:
>>
>> CREATE TABLE dbmail_conf (
>> hostname VARCHAR(128) NOT NULL,
>> key VARCHAR(32) NOT NULL,
>> value TEXT NULL,
>> UNIQUE INDEX (hostname, key)
>> )
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
_________________________
E.J.A. van Beek
ICT Manager
IC&S
T: +31 30 6355730
F: +31 30 6355731

PGP-key:
www.ic-s.nl/keys/eelco.txt

Hi,

On 7 Jan 2004, at 22:12, Aaron Stone wrote:

> I disagree on getting 2.0 out the door first. Table prefixes are
> critical
> infrastructure for people who may have to share a single database with
> tables
> for different applications. Since we may already need a migration
> script /
> program to get 1.2.x people up to 2.0, table prefixes won't hurt much.
I understand that table prefixes might be critical for some people, and
I'd like to have these custom prefixes in DBMail in the future.

One of the problems it that we'd have to change all queries which are
made in dbmail, which means changing all statements were queries are
made. This is quite a lot of work, for which I don't have any time at
the moment.

Also, getting 2.0 out the door would have (almost) no impact on the
addition of new features. I would like to move to a release model were
we can release more often, with smaller changes between releases. This
way, it should be much easier to do upgrades and bugfixes. I guess,
overall, it could mean that development can speed up, because changes
can be smaller.
If anybody disagrees with this: please yell and tell me why :)

Cheers,
Ilja
--
IC&S
Stadhouderslaan 57
3583 JD Utrecht
telnr. 030-6355730
faxnr. 030-6355731

PGP-key:
http://www.ic-s.nl/keys/ilja.txt

Eelco van Beek - IC&S writes:

> We could also do both (config on file and in the db) and keep them in
> sync. This enables easier administration through interfaces on the DB.
>

well do them redundant..
but as I said, big chunk of dbmail.conf is useless without the DB, except
if you've written some sort of caching which will fallback to fs if the
database is not there, else - stick it in the database. If there's no use
for
something being there - take it off.

Aaron's suggestion pretty much does the job. No one said anything about
specifing db tablenames in conf file? But then you have to modify all the
queries .. at least you'll be able to kill some static code (i.e. sql
queries)

tusers = my_users
taliases = my_aliases
[etc]

Hm give me 2-3 days and I'll give you a PoC on these on dbmail 1.2
(time is hard to find..)

> On 8-jan-04, at 9:44, Mark Mackay - Orcon wrote:
>
>> I really like having a local config file [.much better than the old days
>> of
>> having to compile the database name in, etc]; and I like being able to
>> just
>> 'vi' a file to enable logging, adjust process levels on one of the hosts,
>> etc.

well to be honest, having 1 config file is a pain.. but that's not dbmail's
fault that goes back years since unix sysv and old bsd.. the guys from bell
labs fixed them by creating a totally new OS. Shame that all the others
(most of them) adopted global namespace. so.. no advances in linux/bsd for
the last 10y..
On other hand conf files suck, for my local box 2/3 of postfix is in the DB,
dbmail goes there too, apache logs straight there too wey - even mrtg and
some other stuff i wrote. Easier to administrate and you don't need a genius
to write a simple php/perl/whatever to do few queries and show you some
data.

>> If you want to do database configs as well, then the syntax below would
>> work
>> -- but rather than via hostname, I'd recommend making it some form of
>> 'clientid/configid' which could be supplied on command lines (e.g.
>> ../dbmail-pop3d -configkey DEFAULT) That would allow configs to be
>> experimented with and changed between without having to affect other
>> daemons
>> on the same host, etc.
>>
>> Personally I just say keep the config files :)

Well give us a reason good enough, how that is going to be usefull in
general? looking at the logic - i dont see a reason.

>>> dbmail.conf:
>>> Is a configuration file in /etc or wherever.
>>> - database information
>>> - logging info
>>> - hostname / clustering info
>>>
>>> dbmail_conf:
>>> Is a table in the database.
>>> - Has detailed config info in host/key/value pairs:
>>>
>>> CREATE TABLE dbmail_conf (
>>> hostname VARCHAR(128) NOT NULL,
>>> key VARCHAR(32) NOT NULL,
>>> value TEXT NULL,
>>> UNIQUE INDEX (hostname, key)
>>> )

cheers

On 1/8/04 8:57 AM, "Ilja Booij" <ilja@ic-s.nl> wrote:

> If anybody disagrees with this: please yell and tell me why :)



I'm willing to do the work to add in table prefixes.


Brian

IMNSHO, the dbmail.conf file should have ONLY these 7 config options:

# Database settings
host=127.0.0.1
user=dbmail
pass=password
db=dbmail
configtable=dbmail_config
configprofile=PROFILE1
TRACE_LEVEL=2


EVERY other config option should be in the config table specified keyed
under the profile specified.

and there should be a dbmailctl program which you can call like the
following:

dbmailctl start
dbmailctl stop
dbmailctl reload


This is just my opinion of course.





Brian

On 8 Jan 2004, at 16:23, Brian Blood wrote:

> On 1/8/04 8:57 AM, "Ilja Booij" <ilja@ic-s.nl> wrote:
>
>> If anybody disagrees with this: please yell and tell me why :)
>
> I'm willing to do the work to add in table prefixes.
>
OK Brian, If you're willing to do this:
We need to decide on a design.

1. configurable prefix. I guess the default should be an empty prefix,
to default to the current table names.
2. do tablenames also need to configurable like lou@0xffff.org
mentioned in a previous message? If queries get table names from
variables, it shouldn't be so hard to make the whole table name
configurable.

Problem with configurable table names is, I don't see why one would
need this.. Prefixes make sense to me, but completely configurable
table names sound like over-complication to me.

Please shoot at my arguments. I think we need some strict requirements
on this before Brian gets to work ;)

Ilja
--
IC&S
Stadhouderslaan 57
3583 JD Utrecht
telnr. 030-6355730
faxnr. 030-6355731

PGP-key:
http://www.ic-s.nl/keys/ilja.txt

There are a couple more I'd like to see. For logging, I think a really
complete set of options should be in the config file. It is critical that if
the database cannot be contacted, that the errors are logged to the right place.

For the database settings, I've started to look into dl() and dynamically
loading our database abstraction at run-time. So when we get to coding that,
we'll need a dbtype={mysql,pgsql,...} option.

For authentication, I'm not sure if it should be in the config file or in the
database. On the one hand, you don't need the auth info immediately, and do
need to be in contact with the database for the auth info to be relevant
anyways, so it could be in the database. On the other hand, if you have LDAP
passwords it might be safer to keep them in a file.

I'm leaning towards auth info in the database, because I'd like to see the
config file be as short as is possible, and because a profile/hostname of '*'
would allow me to really easily add mail hosts and keep the auth information
synchronized. For example if there's an LDAP change and 20 hosts need to be
updated... ugh.

Aaron


Brian Blood <brian@macserve.net> said:

> IMNSHO, the dbmail.conf file should have ONLY these 7 config options:
>
> # Database settings
> host=127.0.0.1
> user=dbmail
> pass=password
> db=dbmail
> configtable=dbmail_config
> configprofile=PROFILE1
> TRACE_LEVEL=2
>
>
> EVERY other config option should be in the config table specified keyed
> under the profile specified.
>
> and there should be a dbmailctl program which you can call like the
> following:
>
> dbmailctl start
> dbmailctl stop
> dbmailctl reload
>
>
> This is just my opinion of course.
>
>
>
>
>
> Brian
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

Ilja Booij writes:

>
> On 8 Jan 2004, at 16:23, Brian Blood wrote:
>
>> On 1/8/04 8:57 AM, "Ilja Booij" <ilja@ic-s.nl> wrote:
>>
>>> If anybody disagrees with this: please yell and tell me why :)
>>
>> I'm willing to do the work to add in table prefixes.
>>
> OK Brian, If you're willing to do this:
> We need to decide on a design.
>
> 1. configurable prefix. I guess the default should be an empty prefix, to
> default to the current table names.
> 2. do tablenames also need to configurable like
> lou@0xffff.org

Lou

> mentioned
> in a previous message? If queries get table names from variables, it
> shouldn't be so hard to make the whole table name configurable.
>
> Problem with configurable table names is, I don't see why one would need
> this.. Prefixes make sense to me, but completely configurable table names
> sound like over-complication to me.
>
> Please shoot at my arguments. I think we need some strict requirements on
> this before Brian gets to work ;-)

well brian can start pretty much anytime, but i'd have a poc soon re
configurable tablenames. well if there's configurable tablenames.. there's
not much point
doing prefixes? i can think of one argument why i'd want configurable
tablenames, some months ago my pgsql data partition was corrupted somehow..
(reiserfs) there wasn't much damage done, but the indices on my users and
aliases tables were damaged.. i couldn't even select the data (no dump).
that can be easily solved, since the data on those two table is static and a
dump to replace them or just create new_users, new_aliases,
alter tusers & taliases, nevertheless the same wont apply for msgtables.
for sure there will be other ways to utilize it.

cheers

> and there should be a dbmailctl program which you can call like the
> following:
>
> dbmailctl start
> dbmailctl stop
> dbmailctl reload


A nice feature I've been meaning to put on the request list
is a clean shutdown signal - eg. "dbmailctl shutdown" signals all
daemons to close once the current connection is finished (maybe
with a timeout .. and don't know if you can signal an imap client
to go away) and no new connections are accepted. Another task that
could probably be added at the same time without much more work is
to signal the daemons to reload the config without dropping all
current connections (either gracefully finish the existing ones,
or reload the config with them still open if feasible).



--
Jesse Norell
jesse (at) kci.net

An idea that may work to suit everyone's preference of the config
layout: define specific section names for config items, like [SMTP],
and add that field to Aaron's table example below, and add an
[INCLUDE "/path/to/filename"] function to the config file parser.

Make the config reader get the db parameters from the config file
(also add an (optional) item to the config file to disable
database config lookups completely for those who don't use them),
read in all the default values from the db, then override with any
host specific values from the db, then read through the local config
file and override settings with anything found there.

Everyone could have the setup they wish - a huge dbmail.conf file,
or a minimal one with mostly db config settings, or even break the
local config into multiple files (/etc/dbmail/{dbmail,logging,host}.conf)
or whatever else you could think of.



---- Original Message ----
From: Eelco van Beek - IC&S <dbmail-dev@dbmail.org>
To: dbmail-dev@dbmail.org
Subject: Re: [Dbmail-dev] Re: table prefix
Sent: Thu, 8 Jan 2004 10:34:04 +0100

> We could also do both (config on file and in the db) and keep them in
> sync. This enables easier administration through interfaces on the DB.
>
>
> On 8-jan-04, at 9:44, Mark Mackay - Orcon wrote:
>
> > I really like having a local config file [.much better than the old
> > days of
> > having to compile the database name in, etc]; and I like being
able to
> > just
> > 'vi' a file to enable logging, adjust process levels on one of the
> > hosts,
> > etc.
> >
> > If you want to do database configs as well, then the syntax below
> > would work
> > -- but rather than via hostname, I'd recommend making it some form of
> > 'clientid/configid' which could be supplied on command lines (e.g.
> > ../dbmail-pop3d -configkey DEFAULT) That would allow configs to be
> > experimented with and changed between without having to affect other
> > daemons
> > on the same host, etc.
> >
> > Personally I just say keep the config files :)
> >
> > Cheers,
> > Mark.
> >
> >> dbmail.conf:
> >> Is a configuration file in /etc or wherever.
> >> - database information
> >> - logging info
> >> - hostname / clustering info
> >>
> >> dbmail_conf:
> >> Is a table in the database.
> >> - Has detailed config info in host/key/value pairs:
> >>
> >> CREATE TABLE dbmail_conf (
> >> hostname VARCHAR(128) NOT NULL,
> >> key VARCHAR(32) NOT NULL,
> >> value TEXT NULL,
> >> UNIQUE INDEX (hostname, key)
> >> )
> >
> > _______________________________________________
> > Dbmail-dev mailing list
> > Dbmail-dev@dbmail.org
> > http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >
> _________________________
> E.J.A. van Beek
> ICT Manager
> IC&S
> T: +31 30 6355730
> F: +31 30 6355731
>
> PGP-key:
> www.ic-s.nl/keys/eelco.txt
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
-- End Original Message --


--
Jesse Norell
jesse (at) kci.net

This does indeed give everyone what the want wrt config files, but also
introduces a lot of complexity. That's no necessarily bad, so long as the
simple cases are handled simply.

So, if you have a current config file, with everything in it, and no database
config table... works fine.

And, if you have a database only config file, with just the minimum needed to
"bootstrap" into the database config table... works fine.

Then, for crazy people, crazy things are possible if you're paying attention
to the order of inclusions... complicated, but possible.

Unless I left anything out, it sounds to me like this satisfies the "simple
things are simple, complicated things are possible" rule of UNIX Philosophy.

I can already hear Ilja groaning about this one, so if this is the code we
want, someone's gonna have to do it ;-)

There was one more issue, btw, which was the table names. I don't think that
fully configurable names are particularly necessary, but one place where we
might want to look into something like that is the users table. For people who
might already have a pam_mysql or an apache mod_auth_mysql database, it would
probably be nice for them to have dbmail use that existing database rather
than migrating the users into dbmail and reconfiguring everything else. Dunno
how to do this, just yet, although we should take a look at what pam_mysql and
mod_auth_mysql use in their config files.

Aaron


""Jesse Norell"" <jesse@kci.net> said:

>
> An idea that may work to suit everyone's preference of the config
> layout: define specific section names for config items, like [SMTP],
> and add that field to Aaron's table example below, and add an
> [INCLUDE "/path/to/filename"] function to the config file parser.
>
> Make the config reader get the db parameters from the config file
> (also add an (optional) item to the config file to disable
> database config lookups completely for those who don't use them),
> read in all the default values from the db, then override with any
> host specific values from the db, then read through the local config
> file and override settings with anything found there.
>
> Everyone could have the setup they wish - a huge dbmail.conf file,
> or a minimal one with mostly db config settings, or even break the
> local config into multiple files (/etc/dbmail/{dbmail,logging,host}.conf)
> or whatever else you could think of.
>
>
>
> ---- Original Message ----
> From: Eelco van Beek - IC&S <dbmail-dev@dbmail.org>
> To: dbmail-dev@dbmail.org
> Subject: Re: [Dbmail-dev] Re: table prefix
> Sent: Thu, 8 Jan 2004 10:34:04 +0100
>
> > We could also do both (config on file and in the db) and keep them in
> > sync. This enables easier administration through interfaces on the DB.
> >
> >
> > On 8-jan-04, at 9:44, Mark Mackay - Orcon wrote:
> >
> > > I really like having a local config file [.much better than the old
> > > days of
> > > having to compile the database name in, etc]; and I like being
> able to
> > > just
> > > 'vi' a file to enable logging, adjust process levels on one of the
> > > hosts,
> > > etc.
> > >
> > > If you want to do database configs as well, then the syntax below
> > > would work
> > > -- but rather than via hostname, I'd recommend making it some form of
> > > 'clientid/configid' which could be supplied on command lines (e.g.
> > > ../dbmail-pop3d -configkey DEFAULT) That would allow configs to be
> > > experimented with and changed between without having to affect other
> > > daemons
> > > on the same host, etc.
> > >
> > > Personally I just say keep the config files :)
> > >
> > > Cheers,
> > > Mark.
> > >
> > >> dbmail.conf:
> > >> Is a configuration file in /etc or wherever.
> > >> - database information
> > >> - logging info
> > >> - hostname / clustering info
> > >>
> > >> dbmail_conf:
> > >> Is a table in the database.
> > >> - Has detailed config info in host/key/value pairs:
> > >>
> > >> CREATE TABLE dbmail_conf (
> > >> hostname VARCHAR(128) NOT NULL,
> > >> key VARCHAR(32) NOT NULL,
> > >> value TEXT NULL,
> > >> UNIQUE INDEX (hostname, key)
> > >> )
> > >
> > > _______________________________________________
> > > Dbmail-dev mailing list
> > > Dbmail-dev@dbmail.org
> > > http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> > >
> > _________________________
> > E.J.A. van Beek
> > ICT Manager
> > IC&S
> > T: +31 30 6355730
> > F: +31 30 6355731
> >
> > PGP-key:
> > www.ic-s.nl/keys/eelco.txt
> >
> > _______________________________________________
> > Dbmail-dev mailing list
> > Dbmail-dev@dbmail.org
> > http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >
> -- End Original Message --
>
>
> --
> Jesse Norell
> jesse (at) kci.net
>
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>



--

> Also, getting 2.0 out the door would have (almost) no impact on the
> addition of new features. I would like to move to a release
> model were
> we can release more often, with smaller changes between
> releases. This
> way, it should be much easier to do upgrades and bugfixes. I guess,
> overall, it could mean that development can speed up, because changes
> can be smaller.
> If anybody disagrees with this: please yell and tell me why :)

Rapid development sounds good, but it would also be nice to have a set of
target milestones so that people can know what's coming out and approx when.


I guess the one thing that stands out to me which worries me a bit is schema
changes, and even the addition of new tables, etc. The change for v1.x ->
2.0 is not likely to be a trivial one for me. I'm currently sitting on about
110G of mail and I know the changeover is going to be 'fun'. A
once-every-so-often schema change is fine, but if you have to do it every
1-2 months just to keep current with the current version, then it starts
becoming a bit of an admin burden. This is especially true if security
patches are not applied to the previous versions (e.g if working on 2.4 and
2.2, 2.3 are not patched, etc).

There probably needs to be a better way to track schema changes and updates
to databsae required between versions. A "What's changed between v1.2 and
2.0" and "What's changed between v2.0 and 2.1" doc; which lists the
SQL/scripts that need to be run to upgrade things and the schema changes
made.

Of course if you can avoid schema changes to more major versions 2.0, 2.5,
3.0, etc -- even better.

Maybe a project plan would help here - and the schema could be set up with
the necessary fields.


Don't let the above rain on the development parade :) Just pointing out some
of the concerns from my end relating to the 'cost' of upgrades.

Cheers,
Mark.

> 1. configurable prefix. I guess the default should be an empty prefix,
> to default to the current table names.
> 2. do tablenames also need to configurable like lou@0xffff.org
> mentioned in a previous message? If queries get table names from
> variables, it shouldn't be so hard to make the whole table name
> configurable.
>
> Please shoot at my arguments. I think we need some strict
> requirements
> on this before Brian gets to work ;)

I don't think all tablenames need to be configurable , or certainly the ones
are 'dbmail-specific' [but prefixable may be nice]; but one that may be
worth considering is the users table. I've said this before, many months
ago, so forgive my repeat ramblings:

Most of us have a database of all our customers, for billing, radius, etc.
This to me it make sense to allow dbmail to be able to query our existing
user databases to ultimately find the dbmail "user_idnr" that we need to
handle delivery/checking of mail. Thus it may be worth making the "users"
table totally configurable, so you can hook into any existing table that
exists.

The real power with above, would be to allow the magic dbmail-specific
columns to be customisable (so if you already had a "uid" column which
mimicked the passwd file, you could use that column rather than having to
add a "user_idnr" column to your database, etc.

Even better -- allow custom queries from that table, to allow greater
flexibility. There are probably only a handful of queries which actually
interact with the users' table, so maybe these could be definable [.but not
required to be defined in a config, eg. default if not present].

Reason: We do wholesale ISP services in NZ and also buy a lot of ISPs, which
results in the need to support multiple "mark" usernames on our mailserver
(people over here don't log in with mark@ispa.co.nz and mark@ispb.co.nz --
usually just 'mark'). Dbmail because it's in theory all numeric post-auth is
great for this -- I can have mark from ispa with user_idnr 1000 and mark
from ispb with user_idnr 2000. So all I need to do now is make sure I can
look up the right user_idnr when 'mark' connects.

Currently we're having to proxy the pop3/imap streams and rewrite the
username so when someone goes "USER mark" the dbmail server actually gets
"USER mark.ispa" or similar. We have daemons bound to different IP addresses
and ports with different configs to do this.

What would be great, is to allow dbmail to do this job for us -- so I don't
have another intermediate step which doesn't need to be here. I'd love
dbmail to be able to do the following:

select user_idnr, password from users where userid='mark' and
SOMEOTHERCOLUMN='VALUE'

This SOMEOTHERCOLUMN=VALUE could be an ip address (host='10.0.0.1') based on
the IP address of the incoming connection, or perhaps some variable that is
per-daemon defined (e.g. start ./dbmail-pop3d with some config that says
ISPID="DEFAULT" or ISPID="ISPB").

Now I know not everyone wants this (or do you? -- there must be people in a
similar boat to me out there!). Having just the users-related queries
customisable, and allowing macros to be used such as

%u for username
%p for password
%h for incoming hostip
%{customvariableinconfig}

Would allow a very customisable solution for those people that needed it.
Then people could easily build up configs which queried their existing
user/billing databases.

My 2c.

/Mark

Brian Blood writes:

> IMNSHO, the dbmail.conf file should have ONLY these 7 config options:
>
> # Database settings
> host=127.0.0.1
> user=dbmail
> pass=password
> db=dbmail
> configtable=dbmail_config
> configprofile=PROFILE1
> TRACE_LEVEL=2
>
>
> EVERY other config option should be in the config table specified keyed
> under the profile specified.
>
> and there should be a dbmailctl program which you can call like the
> following:
>
> dbmailctl start
> dbmailctl stop
> dbmailctl reload


well i have one at dend.defx.org/~lou/dbmail
sometime ago i did a sample skel, the script is writtin in rc, plan9 shell.

conf has 2 const, /usr/local/bin/dbmail is the path where the bins are..
i.e. if yours are at /usr/bin put there /usr/bin only.. or just read the
src.
cheers