Mailing List Archive

Conserver logfiles
Hello,

Can anyone tell me if conserver 7.2.x or 8.0 supports logfile wrapping, for
the daemon and console log files? It looks like the files will continue to
grow until cleaned manually or no more disk is available.

Thanks,

Bill LePera, AIX/Linux Hardware Control
IBM Server Group
Poughkeepsie, NY
Re: Conserver logfiles [ In reply to ]
yep, all versions just grow. i usually set things up so that files are
moved to a subdirectory then conserver is sent a SIGUSR2 (if 7.2.5 or
above, otherwise a SIGHUP). if anyone has a good methodology for
trimming/rolling/size-restricting logfiles automatically, i'd love to
hear about it. the key is it needs to be able to do things VERY quicky
(as the server shouldn't really block waiting for a 500MB file get
copied, compressed, or whatever) and a certain amount of data, ideally,
would be retained so a reply could happen (if things roll automatically
it could be quite intrusive at a major downtime, for example).

so, those are the kinds of things i worry about. when it's controlled
externally, folks can schedule things appropriately and just opening a
bunch of files is (usually) very quick.

but, like i said, ideas would be cool!

Bryan

On Mon, Nov 24, 2003 at 11:50:02AM -0500, William P LePera wrote:
> Can anyone tell me if conserver 7.2.x or 8.0 supports logfile
> wrapping, for the daemon and console log files? It looks like the
> files will continue to grow until cleaned manually or no more disk is
> available.
Re: Conserver logfiles [ In reply to ]
On Thu, Nov 27, 2003 at 07:07:33PM -0800, Bryan Stansell wrote:
> yep, all versions just grow. i usually set things up so that files are
> moved to a subdirectory then conserver is sent a SIGUSR2 (if 7.2.5 or
> above, otherwise a SIGHUP). if anyone has a good methodology for
> trimming/rolling/size-restricting logfiles automatically, i'd love to

This is the way log rolling should happen within a server. A couple
of log rolling scripts in the contrib/ directory should be used to
take care of the rest :)

> so, those are the kinds of things i worry about. when it's controlled
> externally, folks can schedule things appropriately and just opening a
> bunch of files is (usually) very quick.

A program with similar restrictions as conserver, syslog-ng, tries to
address the needs of some folks who would like to not roll files.
Syslog-ng can use template filenames, one option in the template is
portions of the date and time.

Thus a LogfileTemplate=/var/log/conserv/$YEAR/$MONTH/$DAY/$HOST is
possible. It doesn't eliminate partions of rolling - removing old
data and compressing - but it does keep files smaller automatically.

To handle some blocking issues syslog-ng uses buffers and threads with
blocking I/O or asyncronous I/O. Thus I/O calls that take time can be
used *and* you don't loose any data. Yes, buffers can overflow, but
then you are accepting data faster than you can write to disk so you
would have lost data anyway, its just a matter of which portion of the
data.

http://www.balabit.hu/

Samba has a log filesize limit you can set, if the log file exceeds
this size (perhaps just on start or when SIGHUP is received) the log
file is moved to <logfilename>.old and a new logfile is created. The
logic is fuzzy, so the limit isn't a hard size, its more of a
suggestion of a limit that gets rolled when it makes sense to do so
(so not half way through a log message).

Apache gives you the ability to log to named pipes allowing for a
dedicated logging program to go nuts with compressing on the fly and
rolling at size, or line, or time limits.
-----------------------------------------------------------------------
__o Bradley Arlt Security Team Lead
_ \<_ arlt@cpsc.ucalgary.ca University Of Calgary
(_)/(_) Joyously Canadian Computer Science
Re: Conserver logfiles [ In reply to ]
> On Thu, Nov 27, 2003 at 07:07:33PM -0800, Bryan Stansell wrote:
>> yep, all versions just grow. i usually set things up so that files are
>> moved to a subdirectory then conserver is sent a SIGUSR2 (if 7.2.5 or
>> above, otherwise a SIGHUP). if anyone has a good methodology for
>> trimming/rolling/size-restricting logfiles automatically, i'd love to
>

Many systems have the program 'logrotate' that manages log rotation in
general. Conservers Debian package is doing just that:

/var/log/conserver/*.log {
rotate 7
size 300k
daily
compress
missingok
postrotate
/etc/init.d/conserver-server rotate
endscript
}

I used to have 'reload' instead of 'rotate' until I read Bryans last
message about sigusr2. :-)

(The 'size 300k' is for those who run logrotate more often than once a day.)

'logrotate' comes from redhat, and can be downloaded thru cvs:
cvs -z9 -d :pserver:anoncvs@sources.redhat.com:/cvs/rhl login
# password is anoncvs
cvs -z9 -d :pserver:anoncvs@sources.redhat.com:/cvs/rhl co logrotate
Upstream Authors are various RedHat people.