Mailing List Archive

Hello,

Dbmail logs via syslog, and you can adjust the verbosity
of what it logs in dbmail.conf. Probably your syslogd is
setup to print messages on the console or to some tty, which
is usually set in /etc/syslogd.conf (but if your don't find
anything there, check the manpage for your syslogd server).
If you can't figure out how to get it setup the way you want,
post back with the contents of your syslog.conf (you can strip
all commented lines if you want), and describe what you want
it to do/not do, and we can probably help get you going.

jn


---- Original Message ----
From: Bill Hacker <dbmail-dev@dbmail.org>
To: dbmail-dev@dbmail.org
Subject: [Dbmail-dev] Console Messages
Sent: Sun, 30 Nov 2003 17:25:58 +0800

> Odd one here:
>
> I have (AFAIK) all err, warning, emerg, debug, and such messages that
> would ordinarily be thrown onto the console redirected in syslogd to
> various /var/log<files> ...which I can then 'tail -f' in a separate
> login console... Handy..
>
> But - invoking a daemon on as-yet-not-fully-configured installation, to
> wit: dbmail-pop3d or dbmail-imapd - results in an error condition
> (which is not what I am posting about - the *error condition* and its
> 'warning' messages are quite legitimate at this stage...)
>
> What escapes me is that these error message still appear on the console.
>
> It would appear that syslogd is not being used, *but*....
>
> I do not see anything in dbmail.conf that specifies either syslog or
any
> other method of logging.
>
> Q: Where should I look, and/or is this a compile-time option that
cannot
> be changed at runtime?
>
> Bill Hacker
>
>
>
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
-- End Original Message --


--
Jesse Norell
jesse (at) kci.net

Thanks, Jesse,

- here is the /etc/syslog.conf (stripped of comments):
==============================================
*.err;kern.*;auth.*;*debug;mail.*;*.warning /var/log/console.log
*.notice;kern.debug;lpr.info;mail.crit;news.err /var/log/messages
security.* /var/log/security
auth.info;authpriv.info /var/log/auth.log
mail.info /var/log/maillog
lpr.info /var/log/lpd-errs
cron.* /var/log/cron
*.emerg /var/log/emerg.log
*.warning /var/log/warn.log
console.* /var/log/console.log
!startslip
*.* /var/log/slip.log
!ppp
*.* /var/log/ppp.log
==================================================================
Note that almost *nothing* should be going to the console... not even
kernel messages... (server is normally unattended..).

And all the /var/log<whatever> files exist, are owned by root:wheel, and
chmod'ed to 600.

The *real* problem (source of the messages) seems to be that DBMail's
daemons, dbmail-imapd and dbmail-pop3d go into runaway when they are
invoked. Error messages indicate they create all their authorized quota
of child processes before (or while) connecting to the DB. Not clear if
the first connection succeeeds, but the second and subsequent surely
fail with:

PQconnectdb failed: FATAL: Sorry, too many clients already

Thereafter, we get:

PQconnectdb failed: FATAL: Non-superuser connection limit exceeded

- So far, in dbmail.conf, I have backed down the number of processes and
children permitted to ONE, but each failure is 'FATAL' and spawns a new
one anyway, and does not always get the zombies cleared out (though a
killall <daemon-name> does do so - after a pause).

There *were* bugs discussed in this area of DBMail that consumed 100% of
CPU resources, BUT, IIRC they were allegedly fixed months ago (at least
w/r MySQL <G>). Maybe not w/r pgsql?

FWIW, dbmail-smtp works fine to the postgreSQL DB, so I am off to
compare source code in search of "MySQL'ism's" ...

Regards,

Bill

Jesse Norell wrote:

> Hello,
>
> Dbmail logs via syslog, and you can adjust the verbosity
> of what it logs in dbmail.conf. Probably your syslogd is
> setup to print messages on the console or to some tty, which
> is usually set in /etc/syslogd.conf (but if your don't find
> anything there, check the manpage for your syslogd server).
> If you can't figure out how to get it setup the way you want,
> post back with the contents of your syslog.conf (you can strip
> all commented lines if you want), and describe what you want
> it to do/not do, and we can probably help get you going.
>
> jn
>
>
> ---- Original Message ----
> From: Bill Hacker <dbmail-dev@dbmail.org>
> To: dbmail-dev@dbmail.org
> Subject: [Dbmail-dev] Console Messages
> Sent: Sun, 30 Nov 2003 17:25:58 +0800
>
>
>>Odd one here:
>>
>>I have (AFAIK) all err, warning, emerg, debug, and such messages that
>>would ordinarily be thrown onto the console redirected in syslogd to
>>various /var/log<files> ...which I can then 'tail -f' in a separate
>>login console... Handy..
>>
>>But - invoking a daemon on as-yet-not-fully-configured installation, to
>>wit: dbmail-pop3d or dbmail-imapd - results in an error condition
>>(which is not what I am posting about - the *error condition* and its
>>'warning' messages are quite legitimate at this stage...)
>>
>>What escapes me is that these error message still appear on the console.
>>
>>It would appear that syslogd is not being used, *but*....
>>
>>I do not see anything in dbmail.conf that specifies either syslog or
>
> any
>
>>other method of logging.
>>
>>Q: Where should I look, and/or is this a compile-time option that
>
> cannot
>
>>be changed at runtime?
>>
>>Bill Hacker
>>
>>
>>
>>
>>_______________________________________________
>>Dbmail-dev mailing list
>>Dbmail-dev@dbmail.org
>>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>>
>
> -- End Original Message --
>
>
> --
> Jesse Norell
> jesse (at) kci.net
>
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
>

Hello,

> - here is the /etc/syslog.conf (stripped of comments):
> ==============================================
> *.err;kern.*;auth.*;*debug;mail.*;*.warning /var/log/console.log

That should probably be "*.debug" - dunno what'll happen with
that particular typo (probably syslogd dependant).

> *.notice;kern.debug;lpr.info;mail.crit;news.err /var/log/messages
> security.* /var/log/security
> auth.info;authpriv.info /var/log/auth.log
> mail.info /var/log/maillog
> lpr.info /var/log/lpd-errs
> cron.* /var/log/cron
> *.emerg /var/log/emerg.log
> *.warning /var/log/warn.log

> console.* /var/log/console.log
> !startslip
> *.* /var/log/slip.log
> !ppp
> *.* /var/log/ppp.log

These last 3 look a bit odd - I think they would generate an error
on my syslogd (linux). There is no "console," "startslip," nor "ppp"
facility ... perhaps your syslogd handles those differently. In
any case, there is nothing writing to /dev/console or any other
device that would make those show up on the console. You might check
and see if you have another process running that tails console.log
and dumps it to your console (which indeed would include your dbmail
messages, per your first syslog.conf line). If so, take mail.*
out (maybe leave mail.crit or something), and make a new line for
just the mail logs, pointing to some other file (eg. mail.log); and
remember to log rotate that file, or your disk will be filled over
time.


The *real* problem (source of the messages) seems to be that DBMail's
> daemons, dbmail-imapd and dbmail-pop3d go into runaway when they are
> invoked. Error messages indicate they create all their authorized
quota
> of child processes before (or while) connecting to the DB. Not clear if
> the first connection succeeeds, but the second and subsequent surely
> fail with:
>
> PQconnectdb failed: FATAL: Sorry, too many clients already
>
> Thereafter, we get:
>
> PQconnectdb failed: FATAL: Non-superuser connection limit exceeded
>
> - So far, in dbmail.conf, I have backed down the number of processes
and
> children permitted to ONE, but each failure is 'FATAL' and spawns a new
> one anyway, and does not always get the zombies cleared out (though a
> killall <daemon-name> does do so - after a pause).


Perhaps we could add some rudimentary "something must be wrong"
checking that if a whole set of NCHILDREN children die FATALly
without handling any connections, the master process could assume
(or test) that further children will also die, and exit cleanly.
That of course doesn't fix your problem, but would probably be
good behavior.

As for your problem, check max_connections and
superuser_reserved_connections values in postgresql.conf (and
any command-line overrides).... other than that, nothing comes
to mind.


> There *were* bugs discussed in this area of DBMail that consumed
100% of
> CPU resources, BUT, IIRC they were allegedly fixed months ago (at least
> w/r MySQL <G>). Maybe not w/r pgsql?

The 2 I recall were a signal handling problem in the linux kernel
and the need to compile with -fomit-frame-pointer.

> FWIW, dbmail-smtp works fine to the postgreSQL DB, so I am off to
> compare source code in search of "MySQL'ism's" ...
>
> Regards,
>
> Bill
>
> Jesse Norell wrote:
>
> > Hello,
> >
> > Dbmail logs via syslog, and you can adjust the verbosity
> > of what it logs in dbmail.conf. Probably your syslogd is
> > setup to print messages on the console or to some tty, which
> > is usually set in /etc/syslogd.conf (but if your don't find
> > anything there, check the manpage for your syslogd server).
> > If you can't figure out how to get it setup the way you want,
> > post back with the contents of your syslog.conf (you can strip
> > all commented lines if you want), and describe what you want
> > it to do/not do, and we can probably help get you going.
> >
> > jn
> >
> >
> > ---- Original Message ----
> > From: Bill Hacker <dbmail-dev@dbmail.org>
> > To: dbmail-dev@dbmail.org
> > Subject: [Dbmail-dev] Console Messages
> > Sent: Sun, 30 Nov 2003 17:25:58 +0800
> >
> >
> >>Odd one here:
> >>
> >>I have (AFAIK) all err, warning, emerg, debug, and such messages that
> >>would ordinarily be thrown onto the console redirected in syslogd to
> >>various /var/log<files> ...which I can then 'tail -f' in a separate
> >>login console... Handy..
> >>
> >>But - invoking a daemon on as-yet-not-fully-configured
installation, to
> >>wit: dbmail-pop3d or dbmail-imapd - results in an error condition
> >>(which is not what I am posting about - the *error condition* and its
> >>'warning' messages are quite legitimate at this stage...)
> >>
> >>What escapes me is that these error message still appear on the
console.
> >>
> >>It would appear that syslogd is not being used, *but*....
> >>
> >>I do not see anything in dbmail.conf that specifies either syslog or
> >
> > any
> >
> >>other method of logging.
> >>
> >>Q: Where should I look, and/or is this a compile-time option that
> >
> > cannot
> >
> >>be changed at runtime?
> >>
> >>Bill Hacker
> >>
> >>
> >>
> >>
> >>_______________________________________________
> >>Dbmail-dev mailing list
> >>Dbmail-dev@dbmail.org
> >>http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >>
> >
> > -- End Original Message --
> >
> >
> > --
> > Jesse Norell
> > jesse (at) kci.net
> >
> >
> > _______________________________________________
> > Dbmail-dev mailing list
> > Dbmail-dev@dbmail.org
> > http://twister.fastxs.net/mailman/listinfo/dbmail-dev
> >
> >
>
>
> _______________________________________________
> Dbmail-dev mailing list
> Dbmail-dev@dbmail.org
> http://twister.fastxs.net/mailman/listinfo/dbmail-dev
>
-- End Original Message --


--
Jesse Norell
jesse (at) kci.net

Jesse Norell wrote:

> Hello,
>
>
>> - here is the /etc/syslog.conf (stripped of comments):
>> ==============================================

*SNIP*

>> !startslip *.*
>> /var/log/slip.log !ppp *.*
>> /var/log/ppp.log
>
>
> These last 3 look a bit odd - I think they would generate an error on
> my syslogd (linux). There is no "console," "startslip," nor "ppp"
> facility ...

ACK, will yank 'em. Irrelevant, actually, as the box has no need for,
hence never starts, either process. (dial-up or PPPoE are not in its
physical or logical environment).

> perhaps your syslogd handles those differently.

probably so (FreeBSd 4.9 RELEASE, w/o the Linux compatibility toolset <G>)

> In any case, there is nothing writing to /dev/console or any other
> device that would make those show up on the console. You might check
> and see if you have another process running that tails console.log
> and dumps it to your console (which indeed would include your dbmail
> messages, per your first syslog.conf line).

Not. These seem to be kernel PANIC outputs. I am running multitail on
another ssh session with colorized tail -f equvalent in split-screen
watching console.log and maillog. These are different! - with consol.log
showing about 5 times as much as hits the console, while maillog runs
too fast to read... (access is via PowerBook 17" running OS X 'Panther'
over reasonably fast cable modem)

> If so, take mail.* out (maybe leave mail.crit or something), and make
> a new line for just the mail logs, pointing to some other file (eg.
> mail.log); and remember to log rotate that file, or your disk will be
> filled over time.
>
No longer really the issue. See below.

>
> The *real* problem (source of the messages) seems to be that DBMail's
>
>
>> daemons, dbmail-imapd and dbmail-pop3d go into runaway when they
>> are invoked. Error messages indicate they create all their
>> authorized
>
> quota
>
>> of child processes before (or while) connecting to the DB. Not
>> clear if the first connection succeeeds, but the second and
>> subsequent surely fail with:
>>
>> PQconnectdb failed: FATAL: Sorry, too many clients already
>>
>> Thereafter, we get:
>>
>> PQconnectdb failed: FATAL: Non-superuser connection limit exceeded
>>
>>
>> - So far, in dbmail.conf, I have backed down the number of
>> processes
>
> and
>
>> children permitted to ONE, but each failure is 'FATAL' and spawns a
>> new one anyway, and does not always get the zombies cleared out
>> (though a killall <daemon-name> does do so - after a pause).
>
>
>
> Perhaps we could add some rudimentary "something must be wrong"
> checking that if a whole set of NCHILDREN children die FATALly
> without handling any connections, the master process could assume (or
> test) that further children will also die, and exit cleanly. That of
> course doesn't fix your problem, but would probably be good behavior.
>
Better 'fix' would be to exit the whole complex with an error the first
time a DB connection could not be opened, better yet - have it 'sleep'
then try again later.

>
> As for your problem, check max_connections and
> superuser_reserved_connections values in postgresql.conf (and any
> command-line overrides).... other than that, nothing comes to mind.

IIRC, the default for pgsql was 30 connections. I have bumped that to
512 buffers and 250 connections with no joy. More than that requires a
custom kernel with extra SYS V semaphores, some sysctl statements to do
with memory, etc. etc. I actually have such, written for use with
AOLServer+OpenACS when I needed to run multiple *instances* of the Pgsql
'engine' (not multiple databases, but whole extra independent
instances). But this isn't really a Pgsql failing, as dbmail-smtp gets
access just fine.....

Anyway - this occurs when invoked as root:wheel with dbmail.conf set the
same, or with dbmail:mail (as dbmail-smtp runs) with matching
dbmail.conf (save the use of ports above 1024).

A look at the source tree with a grep 'PQConnectdb' seems to find a lot
more attention paid to moving strings about, traceing and debugging
code, than to 'JFDI' w/r DB access. 'Tinker to Evers to Chance' as the
poem goes... MySQL forced this maybe?

It makes sense (to me) to replace the whole lot with a far more terse
set of ECPG code, wherein there is far more direct mapping between
client I/O strings and SQL..

Doubt that is a 'general' option so long as the project is moored to
MySQL... ECPG is a superset of an SQL standard, but AFAIK MySQL isn't
very SQL'ish. Not sure it even has embedded SQL tools.

>
>
>> There *were* bugs discussed in this area of DBMail that consumed
>
> 100% of
>
>> CPU resources, BUT, IIRC they were allegedly fixed months ago (at
>> least w/r MySQL <G>). Maybe not w/r pgsql?
>
>
> The 2 I recall were a signal handling problem in the linux kernel and
> the need to compile with -fomit-frame-pointer.
>
ACK.. Went back and checked. There were similarities in the results, but
very different cause...
>
*SNIP*

Thanks,

Bill