Mailing List Archive: Monitoring Catalyst on Apache2 & FastCGI

Monitoring Catalyst on Apache2 & FastCGI

May 5, 2014, 3:41 AM

Post #1 of 4 (2411 views)

Dear all,

I'd need some help with debugging and monitoring an apache2+fastcgi
production environment ...
... i did not find any information about that online ?! :(

what i would need is logs of the fcgi processes that show:
* the request IP, url and paramters
* how long it took to answer a request
* cpu+mem usage of each process

putting a catalyst application into Debug-mode is one option;
but thats not clever for a production environment ... and its pretty
hard to scan through all that information.

any ideas ?

the story behind....

We are running a bunch of Catalyst applications in an Apache2 and
FastCGI production environemnt.
Since a few days it happend 2 times, that one of these applications ran
really "hot";
the fcgi processes of that application used 4times more ram and constant
high CPU-usage.
the result was, that apache spanned lots of processes trying to answer
all requests;
>>> so at the end *all* requests to apache2 returned a "timeout" ..

i tried to find out what the problem was; grubbed through lots of apache
access logs without any reasonable result :(

i'd be thankful for any hints how a catalyst+apache+fastcgi environment
can be monitored ..

thanks alot,
bernhard

--
Bernhard Bauch, Webdevelopment
ZSI - Zentrum für soziale Innovation
bauch@zsi.at
Skype: berni-zsi

Re: Monitoring Catalyst on Apache2 & FastCGI [ In reply to ]

rob at intelcompute

May 5, 2014, 3:49 AM

Post #2 of 4 (2366 views)

Permalink

Are you using https://metacpan.org/pod/Catalyst::Log for your own
application-level logging?

This would be step one.

For performance testing, memory leaks, etc. I'm sure there's plenty of
options on CPAN?

On 05/05/2014 11:41 AM, Bernhard Bauch wrote:
> Dear all,
>
> I'd need some help with debugging and monitoring an apache2+fastcgi
> production environment ...
> ... i did not find any information about that online ?! :(
>
> what i would need is logs of the fcgi processes that show:
> * the request IP, url and paramters
> * how long it took to answer a request
> * cpu+mem usage of each process
>
> putting a catalyst application into Debug-mode is one option;
> but thats not clever for a production environment ... and its pretty
> hard to scan through all that information.
>
> any ideas ?
>
>
> the story behind....
>
> We are running a bunch of Catalyst applications in an Apache2 and
> FastCGI production environemnt.
> Since a few days it happend 2 times, that one of these applications ran
> really "hot";
> the fcgi processes of that application used 4times more ram and constant
> high CPU-usage.
> the result was, that apache spanned lots of processes trying to answer
> all requests;
>>>> so at the end *all* requests to apache2 returned a "timeout" ..
> i tried to find out what the problem was; grubbed through lots of apache
> access logs without any reasonable result :(
>
> i'd be thankful for any hints how a catalyst+apache+fastcgi environment
> can be monitored ..
>
> thanks alot,
> bernhard
>
>
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/

Re: Monitoring Catalyst on Apache2 & FastCGI [ In reply to ]

bauch at zsi

Jun 5, 2014, 4:39 AM

Post #3 of 4 (2322 views)

Permalink

Dear all,

i can believe that there is no monitoring tool for catalyst-application
in production.
how do you debug/monitor your catalyst applications, if suddenly your
perl/fcgi process are becoming bigger and bigger or an APP is no
responsing for some time ?
one need to fine out, which request with which parameters burdens your
apps...
of course the catalyst-logs (in debug-mode) gives you all information
about a request, but its kind of hard to read/grep through tons of
production logs ...

i guess there are some middleware plugins for starman or PSGI, but the
ones i found are specially not ment to be used in a production
environemnt...
any ideas ?

cheers,
bernhard

On 05.05.14 12:49, Robert Brown wrote:
> Are you using https://metacpan.org/pod/Catalyst::Log for your own
> application-level logging?
>
> This would be step one.
>
> For performance testing, memory leaks, etc. I'm sure there's plenty of
> options on CPAN?
>
>
>
> On 05/05/2014 11:41 AM, Bernhard Bauch wrote:
>> Dear all,
>>
>> I'd need some help with debugging and monitoring an apache2+fastcgi
>> production environment ...
>> ... i did not find any information about that online ?! :(
>>
>> what i would need is logs of the fcgi processes that show:
>> * the request IP, url and paramters
>> * how long it took to answer a request
>> * cpu+mem usage of each process
>>
>> putting a catalyst application into Debug-mode is one option;
>> but thats not clever for a production environment ... and its pretty
>> hard to scan through all that information.
>>
>> any ideas ?
>>
>>
>> the story behind....
>>
>> We are running a bunch of Catalyst applications in an Apache2 and
>> FastCGI production environemnt.
>> Since a few days it happend 2 times, that one of these applications ran
>> really "hot";
>> the fcgi processes of that application used 4times more ram and constant
>> high CPU-usage.
>> the result was, that apache spanned lots of processes trying to answer
>> all requests;
>>>>> so at the end *all* requests to apache2 returned a "timeout" ..
>> i tried to find out what the problem was; grubbed through lots of apache
>> access logs without any reasonable result :(
>>
>> i'd be thankful for any hints how a catalyst+apache+fastcgi environment
>> can be monitored ..
>>
>> thanks alot,
>> bernhard
>>
>>
>>
>>
>> _______________________________________________
>> List: Catalyst@lists.scsys.co.uk
>> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
>> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
>> Dev site: http://dev.catalyst.perl.org/
>
> !DSPAM:53676cf3302503345520506!
>
>
> _______________________________________________
> List: Catalyst@lists.scsys.co.uk
> Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
> Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
> Dev site: http://dev.catalyst.perl.org/
>
>
> !DSPAM:53676cf3302503345520506!

--
Bernhard Bauch, Webdevelopment
ZSI - Zentrum für soziale Innovation
bauch@zsi.at
Skype: berni-zsi

Re: Monitoring Catalyst on Apache2 & FastCGI [ In reply to ]

nonsolosoft at diff

Jun 5, 2014, 5:01 AM

Post #4 of 4 (2316 views)

Permalink

On 05/06/2014 13:39, Bernhard Bauch wrote:
> Dear all,
>
> i can believe that there is no monitoring tool for catalyst-application
> in production.

I've a catalyst app in production as external fastcgi since 8 years, the
only one time it had troubles was for a disk failure.

Anyway you can have application testing script, you can run them to
check if the application is still in a working state.

My opinion is:
1) use system monitor like nagios and such to verify that your system is
up and running
2) use nagios to test if your database is up and running, accepting new
connections.
3) test with nagios if your app is still answering via http
4) run application level test as many time you want (daily or weekly)
and at least each time you upgrade system or application.
Application level tests are for business logic, for features, for security.
5) improve and refine application level test each time you have a bug report

I really don't see any need of better monitoring tool.

Bye, \ferz

_______________________________________________
List: Catalyst@lists.scsys.co.uk
Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst
Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/
Dev site: http://dev.catalyst.perl.org/

Mailing List Archive

Attached Files:

Attached Files: