Mailing List Archive

imfile with existing files, splits files on server side
I tested this on both rsyslog 3.12.1 and this morning's
3.12.2 release.

I ran a couple tests with existing log files. E.g., I have
an httpd access_log which is 1.3MB and about 5,000 lines --
nowhere near as large as my real existing log, which is
136MB and over 511,000 lines.

With the smaller file, as soon as I started rsyslog on the
client side, it correctly started to show up on the server
side in:
/var/log/client1/httpd_access_log

And the hostname shows up in the prefix for the message:
Mar 13 15:12:20 client1 tag_access_log:172.16.4.133[snip]


But after a few hundred lines, it suddenly starts logging to:
/var/log/httpd_access_log

And the hostname no longer shows up in the prefix:
Mar 13 15:12:20 tag_access_log:76.8.67.2[snip]


Then, thousands of lines later, it goes back to logging to:
/var/log/client1/httpd_access_log



The other thing is, it never actually finishes the entire
5000 lines. It does 4000-something and then stops.

My gut feeling is this is related to ActionQueue settings.
So I'm going to experiment with a few different parameters.

But the part that I don't get is why it's not all being
logged to the same %HOSTNAME% directory?


I don't know what's useful to send for diagnostics -- the
original files from the client side, the split files from
the server side, the client debug, the server debug. Etc.

But I'd be happy to run any requested tests.

Meanwhile, I'll tweak ActionQueue params.

johnn
imfile with existing files, splits files on server side [ In reply to ]
Mmmhhhh...

So the 5,000 line file doesn't make it completely to the server? That's
strange... With the larger file, I'd assume that there are some flow
control issues, but with the small one...

Maybe it is related to the other problem with the hostname. Kind of
sounds like it looses sync some time in between reading the files.

I guess it is probably a good idea if you can send me the 5000 line
file, so I can run it in my lab tomorrow. Also, please confirm that the
small file actually doesn't make it to the server.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com
> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of Johnny Tan
> Sent: Thursday, March 13, 2008 8:27 PM
> To: rsyslog-users
> Subject: [rsyslog] imfile with existing files, splits files
> on server side
>
> I tested this on both rsyslog 3.12.1 and this morning's
> 3.12.2 release.
>
> I ran a couple tests with existing log files. E.g., I have
> an httpd access_log which is 1.3MB and about 5,000 lines --
> nowhere near as large as my real existing log, which is
> 136MB and over 511,000 lines.
>
> With the smaller file, as soon as I started rsyslog on the
> client side, it correctly started to show up on the server
> side in:
> /var/log/client1/httpd_access_log
>
> And the hostname shows up in the prefix for the message:
> Mar 13 15:12:20 client1 tag_access_log:172.16.4.133[snip]
>
>
> But after a few hundred lines, it suddenly starts logging to:
> /var/log/httpd_access_log
>
> And the hostname no longer shows up in the prefix:
> Mar 13 15:12:20 tag_access_log:76.8.67.2[snip]
>
>
> Then, thousands of lines later, it goes back to logging to:
> /var/log/client1/httpd_access_log
>
>
>
> The other thing is, it never actually finishes the entire
> 5000 lines. It does 4000-something and then stops.
>
> My gut feeling is this is related to ActionQueue settings.
> So I'm going to experiment with a few different parameters.
>
> But the part that I don't get is why it's not all being
> logged to the same %HOSTNAME% directory?
>
>
> I don't know what's useful to send for diagnostics -- the
> original files from the client side, the split files from
> the server side, the client debug, the server debug. Etc.
>
> But I'd be happy to run any requested tests.
>
> Meanwhile, I'll tweak ActionQueue params.
>
> johnn
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
>
imfile with existing files, splits files on server side [ In reply to ]
Rainer Gerhards wrote:
> So the 5,000 line file doesn't make it completely to the server? That's
> strange... With the larger file, I'd assume that there are some flow
> control issues, but with the small one...

Just wanted to follow-up... thanks to Rainer's tireless
poring over my excessively long logfiles and debug outputs,
this issue is finally resolved.

It seems like it was a combination of tcp flow control
issues (fixed with the new 3.12.3) and the server by default
discarding messages above severity 4.

Thanks again to Rainer for not giving up on this!

johnn