Mailing List Archive

> I am writing because I would like to limit the output conserver
> writes to disk to a configurable amount per second, say like a
> serial line.

So are you wanting to discard output and just write every third line
or something? Just slowing down the output isn't going to address the
problem here, which is too small space for logs.

> My application captures messages from devices that can send output
> as fast as possible over a network and the resulting log files are
> larger than the storage available.

The problem is the storage size, not the devices. How will slowing
down those devices make things work better?

> I don't think a feature to limit conserver logs this is in the
> current code, correct?

> Could you suggest what part of the code to change? Even better, has
> somebody already solved my problem?

How about using the 'logfilemax' parameter from conserver.cf (see the
man page) and then have a cronjob which copies old log files over the
network to another system with more disk space.

Also, look at the 'timestamp' option, since you probably want to turn
that off as well.


John
_______________________________________________
users mailing list
users@conserver.com
https://www.conserver.com/mailman/listinfo/users

Hi John,

Thanks for replying.

> So are you wanting to discard output and just write every third line or
> something? Just slowing down the output isn't going to address the
> problem here, which is too small space for logs.

The underlying problem is the service writing out too many errors. I am the guy in charge of fixing it.
A sustained message storm broke past a few protection mechanisms of log rotating and disk space monitoring over several days finally caused problems in other systems.
Limiting the rate could offer additional protection.

> How about using the 'logfilemax' parameter from conserver.cf (see the man
> page) and then have a cronjob which copies old log files over the network
> to another system with more disk space.

Effectively I want "logfilemax" that throws away messages in case of a sustained message storm.
I prefer throwing away messages because they tend to be repeated anyway. The system will be fixed eventually and the service restarted.
Logging can continue from where it left off.

There is a cronjob rotating log files at midnight already. I did not realise there are similar facilities in conserver.

Ronaldo

--
This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom


_______________________________________________
users mailing list
users@conserver.com
https://www.conserver.com/mailman/listinfo/users