Mailing List Archive

Re: [External] : Re: MaxMessageSize makes rsyslog stop logging after a few minutes
Hi,

We can't upgrade to the very latest versions of rsyslog (for several
reasons), but we are considering upgrading to 8.2106, after this upgrade
if the problem persists would you be able to help us?

Best regards,
Ricardo Esteves.

On 3/25/22 12:43, Rainer Gerhards wrote:
> I may be wrong - it's a pretty old version. Maybe just a missing error message.
>
> El vie, 25 mar 2022 a las 12:27, Ricardo Esteves
> (<ricardo.lopes.esteves@oracle.com>) escribió:
>> That's strange, before setting it in production i tested in lab with the
>> 1.5m and sent a msg with 1m and it worked (message arrived and was not
>> truncated). Also no error's on startup of rsyslog.
>>
>> Anyway i will try with 1536k to see if that's the problem.
>>
>> Best regards,
>> Ricardo Esteves.
>>
>> On 3/25/22 12:16, Rainer Gerhards wrote:
>>> The size type parameters only support integer sizes, so it be 1m or 2m
>>> but not 1.5m - 1536k will do it. Not sure if it is related, and you
>>> should probably see a startup error in rsyslog's log.
>>>
>>> HTH
>>> Rainer
>>>
>>> El jue, 24 mar 2022 a las 11:46, Ricardo Esteves via rsyslog
>>> (<rsyslog@lists.adiscon.com>) escribió:
>>>> Hi,
>>>>
>>>> We have a setup with 10 satellite rsyslog servers that receive the logs
>>>> from all our application/infra servers and then sends all this logs to a
>>>> central rsyslog server.
>>>>
>>>> We and average of 115k requests per 10seconds arriving on our central
>>>> rsyslog server.
>>>>
>>>> One of our application owners asked us to increase the MaxMessageSize on
>>>> the application servers rsyslog and on the satellite and central rsyslog
>>>> servers in order to accommodate for some of the application big log
>>>> messages that were being truncated by rsyslog.
>>>>
>>>> The biggest log we found on the application was around 1m, so we defined
>>>> $MaxMessageSize 1.5m on the application servers rsyslog and the same on
>>>> the satellite and central rsyslog servers.
>>>>
>>>> After we activated this everything looked fine, but after 5 minutes the
>>>> central rsyslog server stopped logging and the satellite rsyslog servers
>>>> started spooling their messages to disk as they couldn't send them to
>>>> the central rsyslog server.
>>>>
>>>> The only error we could see on the central rsyslog logs was this:
>>>>
>>>> "... netstream session ... from ... will be closed due to error
>>>> [v8.24.0-52.el7 try https://urldefense.com/v3/__http://www.rsyslog.com/e/2165__;!!ACWV5N9M2RV99hQ!bMSzoezh5IocSNqVJa6XxgX07C88-HF0j_zx7jmhsnd0FzTOQNO0MwVhq4xtPBkA_wZy1mD-$ ]"
>>>>
>>>> From documentation:
>>>> rsyslog error 2165
>>>>
>>>> Error occured during socket rcv operation. This error code indicates
>>>> that there was a problem receiving from the remote host. Check network
>>>> connectivity to see if there is a physical problem. This can especially
>>>> occur if there is a partly-functioning network component.
>>>>
>>>> After restarting rsyslog on central server it starts logging again, but
>>>> after 5 minutes stops logging again.
>>>>
>>>> So for now we had to remove the MaxMessageSize parameter from the
>>>> central rsyslog.
>>>>
>>>> On the satellite rsyslog servers and application servers rsyslog the
>>>> parameter is still there and has not caused any issue.
>>>>
>>>>
>>>> Anyone has ever experienced such an issue? Any ideas?
>>>>
>>>>
>>>> Best regards,
>>>> Ricardo Esteves.
>>>> _______________________________________________
>>>> rsyslog mailing list
>>>> https://urldefense.com/v3/__https://lists.adiscon.net/mailman/listinfo/rsyslog__;!!ACWV5N9M2RV99hQ!bMSzoezh5IocSNqVJa6XxgX07C88-HF0j_zx7jmhsnd0FzTOQNO0MwVhq4xtPBkA_-m-_Hxh$
>>>> https://urldefense.com/v3/__http://www.rsyslog.com/professional-services/__;!!ACWV5N9M2RV99hQ!bMSzoezh5IocSNqVJa6XxgX07C88-HF0j_zx7jmhsnd0FzTOQNO0MwVhq4xtPBkA_19VefXG$
>>>> What's up with rsyslog? Follow https://urldefense.com/v3/__https://twitter.com/rgerhards__;!!ACWV5N9M2RV99hQ!bMSzoezh5IocSNqVJa6XxgX07C88-HF0j_zx7jmhsnd0FzTOQNO0MwVhq4xtPBkA_9bDpbC-$
>>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.

_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
Re: [External] : Re: MaxMessageSize makes rsyslog stop logging after a few minutes [ In reply to ]
Free community support (at least from me) is only for the latest
release. For paying support customers, we also look at older releases.

The reason is that in far too many cases I wasted my time
investigating long-fixed bugs. It's inefficient and honestly not fun
at all. Nothing to do in your free time.

Sorry for being direct.

Rainer

El mié, 20 abr 2022 a las 11:11, Ricardo Esteves
(<ricardo.lopes.esteves@oracle.com>) escribió:
>
> Hi,
>
> We can't upgrade to the very latest versions of rsyslog (for several
> reasons), but we are considering upgrading to 8.2106, after this upgrade
> if the problem persists would you be able to help us?
>
> Best regards,
> Ricardo Esteves.
>
> On 3/25/22 12:43, Rainer Gerhards wrote:
> > I may be wrong - it's a pretty old version. Maybe just a missing error message.
> >
> > El vie, 25 mar 2022 a las 12:27, Ricardo Esteves
> > (<ricardo.lopes.esteves@oracle.com>) escribió:
> >> That's strange, before setting it in production i tested in lab with the
> >> 1.5m and sent a msg with 1m and it worked (message arrived and was not
> >> truncated). Also no error's on startup of rsyslog.
> >>
> >> Anyway i will try with 1536k to see if that's the problem.
> >>
> >> Best regards,
> >> Ricardo Esteves.
> >>
> >> On 3/25/22 12:16, Rainer Gerhards wrote:
> >>> The size type parameters only support integer sizes, so it be 1m or 2m
> >>> but not 1.5m - 1536k will do it. Not sure if it is related, and you
> >>> should probably see a startup error in rsyslog's log.
> >>>
> >>> HTH
> >>> Rainer
> >>>
> >>> El jue, 24 mar 2022 a las 11:46, Ricardo Esteves via rsyslog
> >>> (<rsyslog@lists.adiscon.com>) escribió:
> >>>> Hi,
> >>>>
> >>>> We have a setup with 10 satellite rsyslog servers that receive the logs
> >>>> from all our application/infra servers and then sends all this logs to a
> >>>> central rsyslog server.
> >>>>
> >>>> We and average of 115k requests per 10seconds arriving on our central
> >>>> rsyslog server.
> >>>>
> >>>> One of our application owners asked us to increase the MaxMessageSize on
> >>>> the application servers rsyslog and on the satellite and central rsyslog
> >>>> servers in order to accommodate for some of the application big log
> >>>> messages that were being truncated by rsyslog.
> >>>>
> >>>> The biggest log we found on the application was around 1m, so we defined
> >>>> $MaxMessageSize 1.5m on the application servers rsyslog and the same on
> >>>> the satellite and central rsyslog servers.
> >>>>
> >>>> After we activated this everything looked fine, but after 5 minutes the
> >>>> central rsyslog server stopped logging and the satellite rsyslog servers
> >>>> started spooling their messages to disk as they couldn't send them to
> >>>> the central rsyslog server.
> >>>>
> >>>> The only error we could see on the central rsyslog logs was this:
> >>>>
> >>>> "... netstream session ... from ... will be closed due to error
> >>>> [v8.24.0-52.el7 try https://urldefense.com/v3/__http://www.rsyslog.com/e/2165__;!!ACWV5N9M2RV99hQ!bMSzoezh5IocSNqVJa6XxgX07C88-HF0j_zx7jmhsnd0FzTOQNO0MwVhq4xtPBkA_wZy1mD-$ ]"
> >>>>
> >>>> From documentation:
> >>>> rsyslog error 2165
> >>>>
> >>>> Error occured during socket rcv operation. This error code indicates
> >>>> that there was a problem receiving from the remote host. Check network
> >>>> connectivity to see if there is a physical problem. This can especially
> >>>> occur if there is a partly-functioning network component.
> >>>>
> >>>> After restarting rsyslog on central server it starts logging again, but
> >>>> after 5 minutes stops logging again.
> >>>>
> >>>> So for now we had to remove the MaxMessageSize parameter from the
> >>>> central rsyslog.
> >>>>
> >>>> On the satellite rsyslog servers and application servers rsyslog the
> >>>> parameter is still there and has not caused any issue.
> >>>>
> >>>>
> >>>> Anyone has ever experienced such an issue? Any ideas?
> >>>>
> >>>>
> >>>> Best regards,
> >>>> Ricardo Esteves.
> >>>> _______________________________________________
> >>>> rsyslog mailing list
> >>>> https://urldefense.com/v3/__https://lists.adiscon.net/mailman/listinfo/rsyslog__;!!ACWV5N9M2RV99hQ!bMSzoezh5IocSNqVJa6XxgX07C88-HF0j_zx7jmhsnd0FzTOQNO0MwVhq4xtPBkA_-m-_Hxh$
> >>>> https://urldefense.com/v3/__http://www.rsyslog.com/professional-services/__;!!ACWV5N9M2RV99hQ!bMSzoezh5IocSNqVJa6XxgX07C88-HF0j_zx7jmhsnd0FzTOQNO0MwVhq4xtPBkA_19VefXG$
> >>>> What's up with rsyslog? Follow https://urldefense.com/v3/__https://twitter.com/rgerhards__;!!ACWV5N9M2RV99hQ!bMSzoezh5IocSNqVJa6XxgX07C88-HF0j_zx7jmhsnd0FzTOQNO0MwVhq4xtPBkA_9bDpbC-$
> >>>> NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
>
_______________________________________________
rsyslog mailing list
https://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards
NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.