Mailing List Archive

Can truncation of a long message cause split log files?
I am using rsyslog to transfer log files from a client (ver. 8.24.0) to a
server (ver. 5.8.10) via TCP.
I have also configured rsyslog.conf in the server to append a hostname and
date to a transferred log file name. $MaxMessageSize is unset (default).
Now I am experiencing the following strange phenomena.

a. In the folder for transferred log files, I found log files with broken
names like "ter-20190618.log" other than normal log files. Opening such a
file, it says:
> Jun 18 2019 14:17:00 ter errors will be ...
The fifth word, quite often some truncated English word, in a log file seems
to be taken as the /hostname/ in its file name.
In addition, most of these files have very small size and usually only one
line.

b. Original logs in the client and tranferred logs in the server are
different. Long lines in original logs are truncated; transferred logs can
contain up to 2044 characters per line.

c. Such a splited small log seems to have been generated when truncation
occurred, and it contains a part of truncated messages.

In this URL
<http://rsyslog-users.1305293.n2.nabble.com/Long-line-can-t-understand-forward-truncation-reason-td7594369.html>
, a similar issue was discussed even though it is not identical to mine.

My questions are as follows.

1. Can truncation of a long message cause those split log files? If so, how
can I fix this?
2. If not, what would be a cause of my probrems?
3. Will updating rsyslog on the server to ver. 8.24.0 fix / alleviate these
problems?

Any help will be much appreciated.

With best regards,
Tomoaki Hashizaki



--
Sent from: http://rsyslog-users.1305293.n2.nabble.com/
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Can truncation of a long message cause split log files? [ In reply to ]
by default, a message longer than maxmessagesize will get split into multiple
messages, and that can cause the problem that you are seeing.

v8.24 is a couple years old, we got up to 8.40 before switching version numbers
to one that indicates dates.

upgrading will not solve your problem, but will include many other fixes (not to
mention new features)

David Lang

On Mon, 17 Jun 2019, Tomoaki_Hashizaki via rsyslog wrote:

> Date: Mon, 17 Jun 2019 11:44:48 -0700 (MST)
> From: Tomoaki_Hashizaki via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: Tomoaki_Hashizaki <tomoakihashi@gmail.com>
> Subject: [rsyslog] Can truncation of a long message cause split log files?
>
> I am using rsyslog to transfer log files from a client (ver. 8.24.0) to a
> server (ver. 5.8.10) via TCP.
> I have also configured rsyslog.conf in the server to append a hostname and
> date to a transferred log file name. $MaxMessageSize is unset (default).
> Now I am experiencing the following strange phenomena.
>
> a. In the folder for transferred log files, I found log files with broken
> names like "ter-20190618.log" other than normal log files. Opening such a
> file, it says:
>> Jun 18 2019 14:17:00 ter errors will be ...
> The fifth word, quite often some truncated English word, in a log file seems
> to be taken as the /hostname/ in its file name.
> In addition, most of these files have very small size and usually only one
> line.
>
> b. Original logs in the client and tranferred logs in the server are
> different. Long lines in original logs are truncated; transferred logs can
> contain up to 2044 characters per line.
>
> c. Such a splited small log seems to have been generated when truncation
> occurred, and it contains a part of truncated messages.
>
> In this URL
> <http://rsyslog-users.1305293.n2.nabble.com/Long-line-can-t-understand-forward-truncation-reason-td7594369.html>
> , a similar issue was discussed even though it is not identical to mine.
>
> My questions are as follows.
>
> 1. Can truncation of a long message cause those split log files? If so, how
> can I fix this?
> 2. If not, what would be a cause of my probrems?
> 3. Will updating rsyslog on the server to ver. 8.24.0 fix / alleviate these
> problems?
>
> Any help will be much appreciated.
>
> With best regards,
> Tomoaki Hashizaki
>
>
>
> --
> Sent from: http://rsyslog-users.1305293.n2.nabble.com/
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Can truncation of a long message cause split log files? [ In reply to ]
Thank you for your helpful advice. I am trying to upgrade the system.

> by default, a message longer than maxmessagesize will get split
> into multiple messages,

my maxmessagesize is 8k (default value), and messages truncated between the
2044th and 2045th character in line (near 2k). Do you think that increasing
maxmessagesize will work?

Tomoaki Hashizaki




--
Sent from: http://rsyslog-users.1305293.n2.nabble.com/
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Can truncation of a long message cause split log files? [ In reply to ]
on old versions like that, I thought the maxmessagsize was 2k.

check that you have it set to 8k on the receiving side.

David Lang

On Mon, 17 Jun 2019, Tomoaki_Hashizaki via rsyslog wrote:

> Date: Mon, 17 Jun 2019 12:09:58 -0700 (MST)
> From: Tomoaki_Hashizaki via rsyslog <rsyslog@lists.adiscon.com>
> To: rsyslog@lists.adiscon.com
> Cc: Tomoaki_Hashizaki <tomoakihashi@gmail.com>
> Subject: Re: [rsyslog] Can truncation of a long message cause split log files?
>
> Thank you for your helpful advice. I am trying to upgrade the system.
>
>> by default, a message longer than maxmessagesize will get split
>> into multiple messages,
>
> my maxmessagesize is 8k (default value), and messages truncated between the
> 2044th and 2045th character in line (near 2k). Do you think that increasing
> maxmessagesize will work?
>
> Tomoaki Hashizaki
>
>
>
>
> --
> Sent from: http://rsyslog-users.1305293.n2.nabble.com/
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: Can truncation of a long message cause split log files? [ In reply to ]
You are right! As to rsyslog 5.8.10, $MaxMessageSize's default value is 2k.
https://www.rsyslog.com/doc/v5-stable/configuration/global/index.html

I set $MaxMessageSize to 8k to spot the cause of the problem and see what is
going on.

Thank you very mach, David!

Tomoaki Hashizaki



--
Sent from: http://rsyslog-users.1305293.n2.nabble.com/
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.