Mailing List Archive

On Tue, 4 Aug 2020 18:05:28 -0700
jbiskofski <jbiskofski@gmail.com> wrote:

> I had a hard time accepting this was a good configuration because for
> 20 years I had thought of webservers as big giant compiled systems
> (apache), but apparently you can now create something just as fast in
> Perl.

And a helluva lot easier to install, manage, develop on, and maintain.


--
Steven Lembark
Workhorse Computing
lembark@wrkhors.com
+1 888 359 3508

There are cases where Plack though isn't the solution and where mod_perl written well is a far better (more stable) solution.

It is good when the backend servers are slow (simple not complex app); backend requests are relatively fast, and don't use much memory.

But the warning

(1) If you have large numbers of small apps on a domain (a couple we have have over 60 admin apps under a single domain) or a large single app code base - but where many of the larger requests are hardly used; the ability to choose which perl is cached in shared memory and which is loaded when required is much simpler;
(2) Large code bases can also lead to very slow start-up times;
(3) If there are possibilities of large/slow requests - apache's dynamic nature is better and handling these and then clearing memory - issues with each Plack process keeping large amounts of memory and difficulty in culling/restarting individual Plack children; then handling load efficiently across multiple machines as the front end proxies - have difficulty handling load balancing in this case;

Note I work on a number of projects where the data is relatively large (some including many billions of rows of (closely related) entries)


-----Original Message-----
From: Steven Lembark <lembark@wrkhors.com>
Sent: 20 December 2020 15:31
To: modperl@perl.apache.org
Cc: lembark@wrkhors.com
Subject: Re: suggestions for perl as web development language [EXT]

On Tue, 4 Aug 2020 19:59:01 -0500
Mithun Bhattacharya <mithnb@gmail.com> wrote:

> The question is move off to what ? I don't see alternatives being
> shared which blows an apache+mod_perl setup out of the water.

(Sorry for being late on this...)

There are a variety of servers using Plack which can handle heavy loads and are both better documented and easier to manage than Apache. You can see a list at:

<https://urldefense.proofpoint.com/v2/url?u=https-3A__plackperl.org_&d=DwICAg&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=0pvlZHIoJPSsNhAVAVBL0NMNcYWvoQoOAVEPR_qHMJo&s=O1eJE9v7RhxcM7aaINIPmQv8R8CZSbIRwuxV2rLqHXA&e= >

One big advantage to Plack is *not* having to become a walking encyclopedia of Apache2 internals. Shoving structs around was the only way we knew in the 80's, mod_perl was just an extension of "pass a struct" and keep going. Plack provides an abstraction that at least I find simpler to program with and things like Dancer2 give you the opportunity to munge the incoming request in all sorts of ways to handle messy situations. Beyond that take a look at the servers listed on Plack's website.

--
Steven Lembark
Workhorse Computing
lembark@wrkhors.com
+1 888 359 3508


--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.

On Sun, Dec 20, 2020 at 09:33 Steven Lembark <lembark@wrkhors.com> wrote:

> On Tue, 4 Aug 2020 18:05:28 -0700
> jbiskofski <jbiskofski@gmail.com> wrote:
>
> > I had a hard time accepting this was a good configuration because for
> > 20 years I had thought of webservers as big giant compiled systems
> > (apache), but apparently you can now create something just as fast in
> > Perl.
>
> And a helluva lot easier to install, manage, develop on, and maintain.


Steven, I have met you at a couple of Perl conferences and know you know
Raku as well. One thing that has always bothered me about PSGI, Catalyst,
mod_fcgi, etc., is that I could never find a practical cookbook solution
for a simple but working dynamic and publicly available https website on a
real, modern Apache 2.4 server on a Linux bare-metal, sole user remote
server.

I would love to be able to have something similar for Raku, and there is
Cro, but it so far cannot handle auto-generation of Let's Encrypt TLS
certificates with a reverse proxy on Apache.

As an alternative, I would be happy to run with a Perl solution iif it
could handle my TLS requirement. Any suggestions will be greatly
appreciated.

BTW, I have publicly documented my current Apache setup (non-dynamic except
for limited CGI use on a couple of sites). The Apache uses nearly the
latest Apache and OpenSSL compiled from source. It includes multiple (15 or
so at last count) virtual hosts (using macros for extremely easy
deployment). It was last updated mid-2020 and I'll be looking to update it
next year. It's on Github and I'll send the reference if anyone wants it.

Blessings,

-Tom

Just curious where exactly is the challenge in this setup ? It can't be in
apache supporting real certificates - neither can it be in setting up
reverse proxy internally...

On Sun, Dec 20, 2020 at 11:19 AM Tom Browder <tom.browder@gmail.com> wrote:

>
>
> On Sun, Dec 20, 2020 at 09:33 Steven Lembark <lembark@wrkhors.com> wrote:
>
>> On Tue, 4 Aug 2020 18:05:28 -0700
>> jbiskofski <jbiskofski@gmail.com> wrote:
>>
>> > I had a hard time accepting this was a good configuration because for
>> > 20 years I had thought of webservers as big giant compiled systems
>> > (apache), but apparently you can now create something just as fast in
>> > Perl.
>>
>> And a helluva lot easier to install, manage, develop on, and maintain.
>
>
> Steven, I have met you at a couple of Perl conferences and know you know
> Raku as well. One thing that has always bothered me about PSGI, Catalyst,
> mod_fcgi, etc., is that I could never find a practical cookbook solution
> for a simple but working dynamic and publicly available https website on a
> real, modern Apache 2.4 server on a Linux bare-metal, sole user remote
> server.
>
> I would love to be able to have something similar for Raku, and there is
> Cro, but it so far cannot handle auto-generation of Let's Encrypt TLS
> certificates with a reverse proxy on Apache.
>
> As an alternative, I would be happy to run with a Perl solution iif it
> could handle my TLS requirement. Any suggestions will be greatly
> appreciated.
>
> BTW, I have publicly documented my current Apache setup (non-dynamic
> except for limited CGI use on a couple of sites). The Apache uses nearly
> the latest Apache and OpenSSL compiled from source. It includes multiple
> (15 or so at last count) virtual hosts (using macros for extremely easy
> deployment). It was last updated mid-2020 and I'll be looking to update it
> next year. It's on Github and I'll send the reference if anyone wants it.
>
> Blessings,
>
> -Tom
>

I am confused - you like threads so Perl is bad ? I am very happy forking
away and yes I work a lot with non thread safe DBI connections without any
issues.

On Sun, Dec 20, 2020 at 11:53 AM John Dunlap <John@lariat.co> wrote:

> In my opinion, no one should build new projects in Perl. The world is
> increasingly trending towards parallelism and higher numbers of cpu cores
> and Perl is poorly positioned to leverage these advancements. Many of
> Perl's dependencies are not thread safe and mod_perl forces you to use
> mpm_prefork. My organization has started moving away from Perl to Elixir
> for these reasons.
>
> On Tue, Aug 4, 2020, 3:37 AM James Smith <js5@sanger.ac.uk> wrote:
>
>> Perl is a great solution for web development.
>>
>> Others will disagree but the best way I still believe is using mod_perl -
>> but only if you use it's full power - and you probably need a special sort
>> of mind set to use - but that can be said for any language.
>>
>> From experience - it may be fractionally slower than small "standalone"
>> apps that dancer etc are good at, but it is (a) much, much more stable
>> {dancer etc does not cope well with either large requests or lots of small
>> requests}, and (b) if you have a large code base and/or a large number of
>> services then it generally uses much less compute power than the others
>> {can easily handle multiple services on a single apache instance}
>>
>> Where it really gains is the hooks into the apache process - being able
>> to add functionality easily at any stage in the request process, from path
>> translation, AAA stages, pre-processing, to post-processing and logging,
>> and also to interact with other languages at any stage - e.g. can handle
>> pre-processing & post-processing around a script written in another
>> language (e.g. PHP, Java) or produced by another webserver integrated by
>> mod_proxy.
>>
>> It isn't really a framework though like dancer or mojolicious and thus
>> has its own advantages and disadvantages.
>>
>> You would to some extent have to roll your own code to produce the pages
>> themselves although there are libraries out there to do lots of it.
>>
>> We have an in house library whose embryonic stages were written over 20
>> years ago - and has now been stable for around 12-13 years and works
>> strong...
>>
>> James
>>
>> -----Original Message-----
>> From: Wesley Peng <me@yonghua.org>
>> Sent: 04 August 2020 06:43
>> To: modperl@perl.apache.org
>> Subject: suggestions for perl as web development language [EXT]
>>
>> greetings,
>>
>> My team use all of perl, ruby, python for scripting stuff.
>> perl is stronger for system admin tasks, and data analysis etc.
>> But for web development, it seems to be not as popular as others.
>> It has less selective frameworks, and even we can't get the right people
>> to do the webdev job with perl.
>> Do you think in today we will give up perl/modperl as web development
>> language, and choose the alternatives instead?
>>
>> Thanks & Regards
>>
>>
>>
>>
>> --
>> The Wellcome Sanger Institute is operated by Genome Research
>> Limited, a charity registered in England with number 1021457 and a
>> company registered in England with number 2742969, whose registered
>> office is 215 Euston Road, London, NW1 2BE.
>
>

Agreed prefork is recommended but what is the problem with that ?

On Sun, Dec 20, 2020 at 12:47 PM John Dunlap <John@lariat.co> wrote:

> Our app segfaults at random of we use anything other than prefork.
>
> On Sun, Dec 20, 2020, 1:32 PM Mithun Bhattacharya <mithnb@gmail.com>
> wrote:
>
>> I am confused - you like threads so Perl is bad ? I am very happy forking
>> away and yes I work a lot with non thread safe DBI connections without any
>> issues.
>>
>> On Sun, Dec 20, 2020 at 11:53 AM John Dunlap <John@lariat.co> wrote:
>>
>>> In my opinion, no one should build new projects in Perl. The world is
>>> increasingly trending towards parallelism and higher numbers of cpu cores
>>> and Perl is poorly positioned to leverage these advancements. Many of
>>> Perl's dependencies are not thread safe and mod_perl forces you to use
>>> mpm_prefork. My organization has started moving away from Perl to Elixir
>>> for these reasons.
>>>
>>> On Tue, Aug 4, 2020, 3:37 AM James Smith <js5@sanger.ac.uk> wrote:
>>>
>>>> Perl is a great solution for web development.
>>>>
>>>> Others will disagree but the best way I still believe is using mod_perl
>>>> - but only if you use it's full power - and you probably need a special
>>>> sort of mind set to use - but that can be said for any language.
>>>>
>>>> From experience - it may be fractionally slower than small "standalone"
>>>> apps that dancer etc are good at, but it is (a) much, much more stable
>>>> {dancer etc does not cope well with either large requests or lots of small
>>>> requests}, and (b) if you have a large code base and/or a large number of
>>>> services then it generally uses much less compute power than the others
>>>> {can easily handle multiple services on a single apache instance}
>>>>
>>>> Where it really gains is the hooks into the apache process - being able
>>>> to add functionality easily at any stage in the request process, from path
>>>> translation, AAA stages, pre-processing, to post-processing and logging,
>>>> and also to interact with other languages at any stage - e.g. can handle
>>>> pre-processing & post-processing around a script written in another
>>>> language (e.g. PHP, Java) or produced by another webserver integrated by
>>>> mod_proxy.
>>>>
>>>> It isn't really a framework though like dancer or mojolicious and thus
>>>> has its own advantages and disadvantages.
>>>>
>>>> You would to some extent have to roll your own code to produce the
>>>> pages themselves although there are libraries out there to do lots of it.
>>>>
>>>> We have an in house library whose embryonic stages were written over 20
>>>> years ago - and has now been stable for around 12-13 years and works
>>>> strong...
>>>>
>>>> James
>>>>
>>>> -----Original Message-----
>>>> From: Wesley Peng <me@yonghua.org>
>>>> Sent: 04 August 2020 06:43
>>>> To: modperl@perl.apache.org
>>>> Subject: suggestions for perl as web development language [EXT]
>>>>
>>>> greetings,
>>>>
>>>> My team use all of perl, ruby, python for scripting stuff.
>>>> perl is stronger for system admin tasks, and data analysis etc.
>>>> But for web development, it seems to be not as popular as others.
>>>> It has less selective frameworks, and even we can't get the right
>>>> people to do the webdev job with perl.
>>>> Do you think in today we will give up perl/modperl as web development
>>>> language, and choose the alternatives instead?
>>>>
>>>> Thanks & Regards
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> The Wellcome Sanger Institute is operated by Genome Research
>>>> Limited, a charity registered in England with number 1021457 and a
>>>> company registered in England with number 2742969, whose registered
>>>> office is 215 Euston Road, London, NW1 2BE.
>>>
>>>

On Sun, Dec 20, 2020 at 11:29 Mithun Bhattacharya <mithnb@gmail.com> wrote:

> Just curious where exactly is the challenge in this setup ? It can't be in
> apache supporting real certificates - neither can it be in setting up
> reverse proxy internally...
>

The challenge to me is how exactly to code the reverse proxy on a single
instance of Apache. I have found no one who can tell me exactly how to
manage https in the http conf file between the outward facing side and
inside the reverse proxy so that the auto-tls renewal works with Let's
Encrypt, all on a single server.

I think I could cobble together a cron job to do it, but not without a lot
of trial and error, especially when I'm not sure how the proxy and proxy
pass are supposed to look.

I sure wish someone would update the old Apache Cookbook.

-Tom

Your external facing apache instance would do the SSL part and use
mod_proxy to redirect the request to another instance of apache which
implements the actual functionality. Just remember the second instance
needs to run on a different port and that it doesnt have to talk to the
outside world.

Did you check out the practical mod_perl article ?
https://docstore.mik.ua/orelly/weblinux2/modperl/ch12_07.htm

As for your Lets Encrypt certificate - autorenewal isnt a mod_perl thing
rather you do have to place a script in some sort of scheduler.
https://onepagezen.com/letsencrypt-auto-renew-certbot-apache


On Sun, Dec 20, 2020 at 1:45 PM Tom Browder <tom.browder@gmail.com> wrote:

> On Sun, Dec 20, 2020 at 11:29 Mithun Bhattacharya <mithnb@gmail.com>
> wrote:
>
>> Just curious where exactly is the challenge in this setup ? It can't be
>> in apache supporting real certificates - neither can it be in setting up
>> reverse proxy internally...
>>
>
> The challenge to me is how exactly to code the reverse proxy on a single
> instance of Apache. I have found no one who can tell me exactly how to
> manage https in the http conf file between the outward facing side and
> inside the reverse proxy so that the auto-tls renewal works with Let's
> Encrypt, all on a single server.
>
> I think I could cobble together a cron job to do it, but not without a lot
> of trial and error, especially when I'm not sure how the proxy and proxy
> pass are supposed to look.
>
> I sure wish someone would update the old Apache Cookbook.
>
> -Tom
>
>

You would have to define poor system performance - are you doing anything
cpu intensive at all ? Maybe your RAM is being the bottleneck ?

On Sun, Dec 20, 2020 at 2:28 PM John Dunlap <John@lariat.co> wrote:

> It's extremely inefficient by comparison. We host our application on beefy
> servers with 32 cores and 64G of ram and I commonly see poor system
> performance with less than 25% cpu utilization.
>
> On Sun, Dec 20, 2020, 2:22 PM Mithun Bhattacharya <mithnb@gmail.com>
> wrote:
>
>> Agreed prefork is recommended but what is the problem with that ?
>>
>> On Sun, Dec 20, 2020 at 12:47 PM John Dunlap <John@lariat.co> wrote:
>>
>>> Our app segfaults at random of we use anything other than prefork.
>>>
>>> On Sun, Dec 20, 2020, 1:32 PM Mithun Bhattacharya <mithnb@gmail.com>
>>> wrote:
>>>
>>>> I am confused - you like threads so Perl is bad ? I am very happy
>>>> forking away and yes I work a lot with non thread safe DBI connections
>>>> without any issues.
>>>>
>>>> On Sun, Dec 20, 2020 at 11:53 AM John Dunlap <John@lariat.co> wrote:
>>>>
>>>>> In my opinion, no one should build new projects in Perl. The world is
>>>>> increasingly trending towards parallelism and higher numbers of cpu cores
>>>>> and Perl is poorly positioned to leverage these advancements. Many of
>>>>> Perl's dependencies are not thread safe and mod_perl forces you to use
>>>>> mpm_prefork. My organization has started moving away from Perl to Elixir
>>>>> for these reasons.
>>>>>
>>>>> On Tue, Aug 4, 2020, 3:37 AM James Smith <js5@sanger.ac.uk> wrote:
>>>>>
>>>>>> Perl is a great solution for web development.
>>>>>>
>>>>>> Others will disagree but the best way I still believe is using
>>>>>> mod_perl - but only if you use it's full power - and you probably need a
>>>>>> special sort of mind set to use - but that can be said for any language.
>>>>>>
>>>>>> From experience - it may be fractionally slower than small
>>>>>> "standalone" apps that dancer etc are good at, but it is (a) much, much
>>>>>> more stable {dancer etc does not cope well with either large requests or
>>>>>> lots of small requests}, and (b) if you have a large code base and/or a
>>>>>> large number of services then it generally uses much less compute power
>>>>>> than the others {can easily handle multiple services on a single apache
>>>>>> instance}
>>>>>>
>>>>>> Where it really gains is the hooks into the apache process - being
>>>>>> able to add functionality easily at any stage in the request process, from
>>>>>> path translation, AAA stages, pre-processing, to post-processing and
>>>>>> logging, and also to interact with other languages at any stage - e.g. can
>>>>>> handle pre-processing & post-processing around a script written in another
>>>>>> language (e.g. PHP, Java) or produced by another webserver integrated by
>>>>>> mod_proxy.
>>>>>>
>>>>>> It isn't really a framework though like dancer or mojolicious and
>>>>>> thus has its own advantages and disadvantages.
>>>>>>
>>>>>> You would to some extent have to roll your own code to produce the
>>>>>> pages themselves although there are libraries out there to do lots of it.
>>>>>>
>>>>>> We have an in house library whose embryonic stages were written over
>>>>>> 20 years ago - and has now been stable for around 12-13 years and works
>>>>>> strong...
>>>>>>
>>>>>> James
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: Wesley Peng <me@yonghua.org>
>>>>>> Sent: 04 August 2020 06:43
>>>>>> To: modperl@perl.apache.org
>>>>>> Subject: suggestions for perl as web development language [EXT]
>>>>>>
>>>>>> greetings,
>>>>>>
>>>>>> My team use all of perl, ruby, python for scripting stuff.
>>>>>> perl is stronger for system admin tasks, and data analysis etc.
>>>>>> But for web development, it seems to be not as popular as others.
>>>>>> It has less selective frameworks, and even we can't get the right
>>>>>> people to do the webdev job with perl.
>>>>>> Do you think in today we will give up perl/modperl as web development
>>>>>> language, and choose the alternatives instead?
>>>>>>
>>>>>> Thanks & Regards
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> The Wellcome Sanger Institute is operated by Genome Research
>>>>>> Limited, a charity registered in England with number 1021457 and a
>>>>>> company registered in England with number 2742969, whose registered
>>>>>> office is 215 Euston Road, London, NW1 2BE.
>>>>>
>>>>>

Running individual functions in independent threads can't be a solution for
performance optimization - at least I have never heard such a thing, maybe
others can pitch in.

On Sun, Dec 20, 2020 at 4:15 PM John Dunlap <John@lariat.co> wrote:

> I have no doubt that our application can be optimized. We do so whenever
> we encounter poor performance and we will continue to do so. The point is
> that Perl didn't do a lot to help us in this regard. Languages like elixir
> use immutable data structures and will automatically run individual
> function calls in separate threads. Perl, by contrast, will only have a
> single thread per request.
>
> On Sun, Dec 20, 2020, 3:33 PM Mithun Bhattacharya <mithnb@gmail.com>
> wrote:
>
>> You would have to define poor system performance - are you doing anything
>> cpu intensive at all ? Maybe your RAM is being the bottleneck ?
>>
>> On Sun, Dec 20, 2020 at 2:28 PM John Dunlap <John@lariat.co> wrote:
>>
>>> It's extremely inefficient by comparison. We host our application on
>>> beefy servers with 32 cores and 64G of ram and I commonly see poor system
>>> performance with less than 25% cpu utilization.
>>>
>>> On Sun, Dec 20, 2020, 2:22 PM Mithun Bhattacharya <mithnb@gmail.com>
>>> wrote:
>>>
>>>> Agreed prefork is recommended but what is the problem with that ?
>>>>
>>>> On Sun, Dec 20, 2020 at 12:47 PM John Dunlap <John@lariat.co> wrote:
>>>>
>>>>> Our app segfaults at random of we use anything other than prefork.
>>>>>
>>>>> On Sun, Dec 20, 2020, 1:32 PM Mithun Bhattacharya <mithnb@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I am confused - you like threads so Perl is bad ? I am very happy
>>>>>> forking away and yes I work a lot with non thread safe DBI connections
>>>>>> without any issues.
>>>>>>
>>>>>> On Sun, Dec 20, 2020 at 11:53 AM John Dunlap <John@lariat.co> wrote:
>>>>>>
>>>>>>> In my opinion, no one should build new projects in Perl. The world
>>>>>>> is increasingly trending towards parallelism and higher numbers of cpu
>>>>>>> cores and Perl is poorly positioned to leverage these advancements. Many of
>>>>>>> Perl's dependencies are not thread safe and mod_perl forces you to use
>>>>>>> mpm_prefork. My organization has started moving away from Perl to Elixir
>>>>>>> for these reasons.
>>>>>>>
>>>>>>> On Tue, Aug 4, 2020, 3:37 AM James Smith <js5@sanger.ac.uk> wrote:
>>>>>>>
>>>>>>>> Perl is a great solution for web development.
>>>>>>>>
>>>>>>>> Others will disagree but the best way I still believe is using
>>>>>>>> mod_perl - but only if you use it's full power - and you probably need a
>>>>>>>> special sort of mind set to use - but that can be said for any language.
>>>>>>>>
>>>>>>>> From experience - it may be fractionally slower than small
>>>>>>>> "standalone" apps that dancer etc are good at, but it is (a) much, much
>>>>>>>> more stable {dancer etc does not cope well with either large requests or
>>>>>>>> lots of small requests}, and (b) if you have a large code base and/or a
>>>>>>>> large number of services then it generally uses much less compute power
>>>>>>>> than the others {can easily handle multiple services on a single apache
>>>>>>>> instance}
>>>>>>>>
>>>>>>>> Where it really gains is the hooks into the apache process - being
>>>>>>>> able to add functionality easily at any stage in the request process, from
>>>>>>>> path translation, AAA stages, pre-processing, to post-processing and
>>>>>>>> logging, and also to interact with other languages at any stage - e.g. can
>>>>>>>> handle pre-processing & post-processing around a script written in another
>>>>>>>> language (e.g. PHP, Java) or produced by another webserver integrated by
>>>>>>>> mod_proxy.
>>>>>>>>
>>>>>>>> It isn't really a framework though like dancer or mojolicious and
>>>>>>>> thus has its own advantages and disadvantages.
>>>>>>>>
>>>>>>>> You would to some extent have to roll your own code to produce the
>>>>>>>> pages themselves although there are libraries out there to do lots of it.
>>>>>>>>
>>>>>>>> We have an in house library whose embryonic stages were written
>>>>>>>> over 20 years ago - and has now been stable for around 12-13 years and
>>>>>>>> works strong...
>>>>>>>>
>>>>>>>> James
>>>>>>>>
>>>>>>>> -----Original Message-----
>>>>>>>> From: Wesley Peng <me@yonghua.org>
>>>>>>>> Sent: 04 August 2020 06:43
>>>>>>>> To: modperl@perl.apache.org
>>>>>>>> Subject: suggestions for perl as web development language [EXT]
>>>>>>>>
>>>>>>>> greetings,
>>>>>>>>
>>>>>>>> My team use all of perl, ruby, python for scripting stuff.
>>>>>>>> perl is stronger for system admin tasks, and data analysis etc.
>>>>>>>> But for web development, it seems to be not as popular as others.
>>>>>>>> It has less selective frameworks, and even we can't get the right
>>>>>>>> people to do the webdev job with perl.
>>>>>>>> Do you think in today we will give up perl/modperl as web
>>>>>>>> development language, and choose the alternatives instead?
>>>>>>>>
>>>>>>>> Thanks & Regards
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> The Wellcome Sanger Institute is operated by Genome Research
>>>>>>>> Limited, a charity registered in England with number 1021457 and a
>>>>>>>> company registered in England with number 2742969, whose
>>>>>>>> registered
>>>>>>>> office is 215 Euston Road, London, NW1 2BE.
>>>>>>>
>>>>>>>

On Sun, 20 Dec 2020 14:03:35 -0600
Mithun Bhattacharya <mithnb@gmail.com> wrote:
>
> As for your Lets Encrypt certificate - autorenewal isnt a mod_perl thing
> rather you do have to place a script in some sort of scheduler.

I use Lets Encrypt on Debian, there is no need to place a script; certificates are being automatically renewed every ~60 days

--
https://compta.libremen.com
Logiciel libre de comptabilit? g?n?rale en partie double

[.don't know why I'm seeing only parts of this thread, I did not get the original messages for some reason]

> >> On Sun, Dec 20, 2020 at 2:28 PM John Dunlap <John@lariat.co> wrote:
> >>
> >>> It's extremely inefficient by comparison. We host our application on
> >>> beefy servers with 32 cores and 64G of ram and I commonly see poor system
> >>> performance with less than 25% cpu utilization.
> >>>

Hu? I'm hosting web apps on a lowly Kimsufi host (2 cores, 1,86GHz; 8 euros/month); each app makes dozens of database queries, the latest one makes 150 different queries in .05 seconds. I never see more than 0.15 load average with 10 users on line.

Granted, I have low traffic and small databases, but still.

(incidentally, I had a look at your home page, which took a good 5 seconds to load)

Bien ? vous, Vincent Veyron

--
https://compta.libremen.com
Logiciel libre de comptabilit? g?n?rale en partie double

[You forgot to cc the list ]

On Sun, 20 Dec 2020 23:16:03 -0500
John Dunlap <John@lariat.co> wrote:

> We run 20 customers on a single box and our database has approximately 500
> tables. We run hundreds or thousands of queries per second.
>

500 tables is a lot more than what I typically handle. I'm sure it complicates things.

But see this post by James Smith in a recent thread :

http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/ajax/%3Cef383804cf394c53b48258531891d12b%40sanger.ac.uk%3E

Easier to read in this archive :

http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/browser

I also remember a post by a chinese guy who handled the same order of database size, in which he wrote that he had compared several frameworks and mod_perl was the fastest; but that was something like 10 years ago, and I can't find it anymore.

So I'm not sure how mod_perl could handle that kind of load and be horribly inefficient?

(I forgot to say in my previous post that over 50% of the time used by my script is spent on the _one_ query out of 120 that writes a smallish session hash to disk)

--
Bien ? vous, Vincent Veyron

https://compta.libremen.com
Logiciel libre de comptabilit? g?n?rale en partie double

Making a wild guess here - most RDBMS won't like it if you make thousands
of queries per second across 500 tables every second. Can this be done -
yes but most setup's aren't tuned to be able to handle such a scenario.

If I was doing something like this I can imagine quite a few places which
would fall apart in my current code which would have nothing to do with
either mod_perl or forking/threading. Again I have absolutely no idea what
has been implemented by John so it is quite possible it is a mod_perl and
forking issue but that can not be generalized for everyone.

On Mon, Dec 21, 2020 at 1:27 AM Vincent Veyron <vv.lists@wanadoo.fr> wrote:

>
> [You forgot to cc the list ]
>
> On Sun, 20 Dec 2020 23:16:03 -0500
> John Dunlap <John@lariat.co> wrote:
>
> > We run 20 customers on a single box and our database has approximately
> 500
> > tables. We run hundreds or thousands of queries per second.
> >
>
> 500 tables is a lot more than what I typically handle. I'm sure it
> complicates things.
>
> But see this post by James Smith in a recent thread :
>
>
> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/ajax/%3Cef383804cf394c53b48258531891d12b%40sanger.ac.uk%3E
>
> Easier to read in this archive :
>
> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/browser
>
> I also remember a post by a chinese guy who handled the same order of
> database size, in which he wrote that he had compared several frameworks
> and mod_perl was the fastest; but that was something like 10 years ago, and
> I can't find it anymore.
>
> So I'm not sure how mod_perl could handle that kind of load and be
> horribly inefficient?
>
> (I forgot to say in my previous post that over 50% of the time used by my
> script is spent on the _one_ query out of 120 that writes a smallish
> session hash to disk)
>
> --
> Bien à vous, Vincent Veyron
>
> https://compta.libremen.com
> Logiciel libre de comptabilité générale en partie double
>

Didn’t see the earlier response – John if you are seeing 25% cpu utilization that indicates that something is wrong with architecture of the solution rather than the language.

It would suggest that you have bottlenecks elsewhere – network, memory, database, disk. We have seen that the sweet spot of well designed servers is somewhere between 4 and 8 CPUs and 8-16G RAM, after that the performance gain is not so good – not because of the language – but because of all the other constraints on the system - wanting to databases/disk/network etc. We have HPC compute clusters at work – and to really make use of the large CPU/memory machines (last time I checked it was around 200 cores and 1Tb RAM) not just the coding but the underlying algorithm has to suit parallelisation, and care has to be taken to avoid writing to disk at any point in the operations.

From: Mithun Bhattacharya <mithnb@gmail.com>
Sent: 20 December 2020 22:29
To: mod_perl list <modperl@perl.apache.org>
Subject: Re: suggestions for perl as web development language [EXT]

Running individual functions in independent threads can't be a solution for performance optimization - at least I have never heard such a thing, maybe others can pitch in.

On Sun, Dec 20, 2020 at 4:15 PM John Dunlap <John@lariat.co<mailto:John@lariat.co>> wrote:
I have no doubt that our application can be optimized. We do so whenever we encounter poor performance and we will continue to do so. The point is that Perl didn't do a lot to help us in this regard. Languages like elixir use immutable data structures and will automatically run individual function calls in separate threads. Perl, by contrast, will only have a single thread per request.

On Sun, Dec 20, 2020, 3:33 PM Mithun Bhattacharya <mithnb@gmail.com<mailto:mithnb@gmail.com>> wrote:
You would have to define poor system performance - are you doing anything cpu intensive at all ? Maybe your RAM is being the bottleneck ?

On Sun, Dec 20, 2020 at 2:28 PM John Dunlap <John@lariat.co<mailto:John@lariat.co>> wrote:
It's extremely inefficient by comparison. We host our application on beefy servers with 32 cores and 64G of ram and I commonly see poor system performance with less than 25% cpu utilization.

On Sun, Dec 20, 2020, 2:22 PM Mithun Bhattacharya <mithnb@gmail.com<mailto:mithnb@gmail.com>> wrote:
Agreed prefork is recommended but what is the problem with that ?

On Sun, Dec 20, 2020 at 12:47 PM John Dunlap <John@lariat.co<mailto:John@lariat.co>> wrote:
Our app segfaults at random of we use anything other than prefork.

On Sun, Dec 20, 2020, 1:32 PM Mithun Bhattacharya <mithnb@gmail.com<mailto:mithnb@gmail.com>> wrote:
I am confused - you like threads so Perl is bad ? I am very happy forking away and yes I work a lot with non thread safe DBI connections without any issues.

On Sun, Dec 20, 2020 at 11:53 AM John Dunlap <John@lariat.co<mailto:John@lariat.co>> wrote:
In my opinion, no one should build new projects in Perl. The world is increasingly trending towards parallelism and higher numbers of cpu cores and Perl is poorly positioned to leverage these advancements. Many of Perl's dependencies are not thread safe and mod_perl forces you to use mpm_prefork. My organization has started moving away from Perl to Elixir for these reasons.

On Tue, Aug 4, 2020, 3:37 AM James Smith <js5@sanger.ac.uk<mailto:js5@sanger.ac.uk>> wrote:
Perl is a great solution for web development.

Others will disagree but the best way I still believe is using mod_perl - but only if you use it's full power - and you probably need a special sort of mind set to use - but that can be said for any language.

From experience - it may be fractionally slower than small "standalone" apps that dancer etc are good at, but it is (a) much, much more stable {dancer etc does not cope well with either large requests or lots of small requests}, and (b) if you have a large code base and/or a large number of services then it generally uses much less compute power than the others {can easily handle multiple services on a single apache instance}

Where it really gains is the hooks into the apache process - being able to add functionality easily at any stage in the request process, from path translation, AAA stages, pre-processing, to post-processing and logging, and also to interact with other languages at any stage - e.g. can handle pre-processing & post-processing around a script written in another language (e.g. PHP, Java) or produced by another webserver integrated by mod_proxy.

It isn't really a framework though like dancer or mojolicious and thus has its own advantages and disadvantages.

You would to some extent have to roll your own code to produce the pages themselves although there are libraries out there to do lots of it.

We have an in house library whose embryonic stages were written over 20 years ago - and has now been stable for around 12-13 years and works strong...

James

-----Original Message-----
From: Wesley Peng <me@yonghua.org<mailto:me@yonghua.org>>
Sent: 04 August 2020 06:43
To: modperl@perl.apache.org<mailto:modperl@perl.apache.org>
Subject: suggestions for perl as web development language [EXT]

greetings,

My team use all of perl, ruby, python for scripting stuff.
perl is stronger for system admin tasks, and data analysis etc.
But for web development, it seems to be not as popular as others.
It has less selective frameworks, and even we can't get the right people to do the webdev job with perl.
Do you think in today we will give up perl/modperl as web development language, and choose the alternatives instead?

Thanks & Regards




--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.



--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.

> (I forgot to say in my previous post that over 50% of the time used by my script is spent on the _one_ query out of 120 that writes a smallish session hash to disk)

The first rule of session hashes is don't use session hashes, but the 2nd rule of session hashes is don't write them to disk - that is really inefficient. Look at using something like MySQL, memcached, redis, ... to store them instead - whatever you do - just avoid writing to disk!

-----Original Message-----
From: Vincent Veyron <vv.lists@wanadoo.fr>
Sent: 21 December 2020 07:27
To: John Dunlap <John@lariat.co>
Cc: modperl@perl.apache.org
Subject: Re: suggestions for perl as web development language [EXT]


[You forgot to cc the list ]

On Sun, 20 Dec 2020 23:16:03 -0500
John Dunlap <John@lariat.co> wrote:

> We run 20 customers on a single box and our database has approximately
> 500 tables. We run hundreds or thousands of queries per second.
>

500 tables is a lot more than what I typically handle. I'm sure it complicates things.

But see this post by James Smith in a recent thread :

https://urldefense.proofpoint.com/v2/url?u=http-3A__mail-2Darchives.apache.org_mod-5Fmbox_perl-2Dmodperl_202008.mbox_ajax_-253Cef383804cf394c53b48258531891d12b-2540sanger.ac.uk-253E&d=DwIFAw&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=wOEBUDMfU7L3Uq4Q7pzEIfQ0Qfh1jaa-dmtnINBi3sM&s=fztwZzN5yK5qT6hb4WlQYTaEBqRmSv7Pj7v_o6WmyfM&e=

Easier to read in this archive :

https://urldefense.proofpoint.com/v2/url?u=http-3A__mail-2Darchives.apache.org_mod-5Fmbox_perl-2Dmodperl_202008.mbox_browser&d=DwIFAw&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=wOEBUDMfU7L3Uq4Q7pzEIfQ0Qfh1jaa-dmtnINBi3sM&s=3saqpwQfa8fOf9eiynEa-w3JsT-tenD-pyUc3vfFhwc&e=

I also remember a post by a chinese guy who handled the same order of database size, in which he wrote that he had compared several frameworks and mod_perl was the fastest; but that was something like 10 years ago, and I can't find it anymore.

So I'm not sure how mod_perl could handle that kind of load and be horribly inefficient?


--
Bien ? vous, Vincent Veyron

https://urldefense.proofpoint.com/v2/url?u=https-3A__compta.libremen.com&d=DwIFAw&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=wOEBUDMfU7L3Uq4Q7pzEIfQ0Qfh1jaa-dmtnINBi3sM&s=yvWfWmnBK99th1DSC5sQcjzdWwA6QZRGXpO2R5H1414&e=
Logiciel libre de comptabilit? g?n?rale en partie double


--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.

mod_perl is horribly inefficient because prefork is inefficient and because
each request is single threaded. In addition to this, mod_perl also cannot
provide websockets which are essential in a modern application.

On Mon, Dec 21, 2020 at 1:26 AM Vincent Veyron <vv.lists@wanadoo.fr> wrote:

>
> [You forgot to cc the list ]
>
> On Sun, 20 Dec 2020 23:16:03 -0500
> John Dunlap <John@lariat.co> wrote:
>
> > We run 20 customers on a single box and our database has approximately
> 500
> > tables. We run hundreds or thousands of queries per second.
> >
>
> 500 tables is a lot more than what I typically handle. I'm sure it
> complicates things.
>
> But see this post by James Smith in a recent thread :
>
>
> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/ajax/%3Cef383804cf394c53b48258531891d12b%40sanger.ac.uk%3E
>
> Easier to read in this archive :
>
> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/browser
>
> I also remember a post by a chinese guy who handled the same order of
> database size, in which he wrote that he had compared several frameworks
> and mod_perl was the fastest; but that was something like 10 years ago, and
> I can't find it anymore.
>
> So I'm not sure how mod_perl could handle that kind of load and be
> horribly inefficient?
>
> (I forgot to say in my previous post that over 50% of the time used by my
> script is spent on the _one_ query out of 120 that writes a smallish
> session hash to disk)
>
> --
> Bien à vous, Vincent Veyron
>
> https://compta.libremen.com
> Logiciel libre de comptabilité générale en partie double
>


--
John Dunlap
*CTO | Lariat *

*Direct:*
*john@lariat.co <john@lariat.co>*

*Customer Service:*
877.268.6667
support@lariat.co

Forking is not inefficient unless you work on Windows. Threads are more
complicated to code for - thread safe coding is a thing.

Agreed web sockets are lacking but they are not essential in all modern
application.

I think we have discussed this topic enough and nothing new is being shared
on the topic hence this will be the last reply on this conversation.

On Tue, Dec 22, 2020, 7:35 AM John Dunlap <John@lariat.co> wrote:

> mod_perl is horribly inefficient because prefork is inefficient and
> because each request is single threaded. In addition to this, mod_perl also
> cannot provide websockets which are essential in a modern application.
>
> On Mon, Dec 21, 2020 at 1:26 AM Vincent Veyron <vv.lists@wanadoo.fr>
> wrote:
>
>>
>> [You forgot to cc the list ]
>>
>> On Sun, 20 Dec 2020 23:16:03 -0500
>> John Dunlap <John@lariat.co> wrote:
>>
>> > We run 20 customers on a single box and our database has approximately
>> 500
>> > tables. We run hundreds or thousands of queries per second.
>> >
>>
>> 500 tables is a lot more than what I typically handle. I'm sure it
>> complicates things.
>>
>> But see this post by James Smith in a recent thread :
>>
>>
>> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/ajax/%3Cef383804cf394c53b48258531891d12b%40sanger.ac.uk%3E
>>
>> Easier to read in this archive :
>>
>> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/browser
>>
>> I also remember a post by a chinese guy who handled the same order of
>> database size, in which he wrote that he had compared several frameworks
>> and mod_perl was the fastest; but that was something like 10 years ago, and
>> I can't find it anymore.
>>
>> So I'm not sure how mod_perl could handle that kind of load and be
>> horribly inefficient?
>>
>> (I forgot to say in my previous post that over 50% of the time used by my
>> script is spent on the _one_ query out of 120 that writes a smallish
>> session hash to disk)
>>
>> --
>> Bien à vous, Vincent Veyron
>>
>> https://compta.libremen.com
>> Logiciel libre de comptabilité générale en partie double
>>
>
>
> --
> John Dunlap
> *CTO | Lariat *
>
> *Direct:*
> *john@lariat.co <john@lariat.co>*
>
> *Customer Service:*
> 877.268.6667
> support@lariat.co
>

There are not many applications which really benefit from multiple threads in web server environments unless you have very low load – as they are only efficient as they can use multiple cores, so you need stupidly speced machines to manage this load properly, and if you are using external resources can often be blocking anyway.

Yes mod_perl doesn’t do web-sockets – but usually that isn’t an issue unless you have the need to push when people update content and you want it immediately available. There are others issues to consider with web-sockets e.g. port usage to keep the connections open. If you don’t need that immediate response – polling can achieve the desired effect – and Apache can even tell the client how long you need to wait before you send another connection.


From: John Dunlap <John@lariat.co>
Sent: 22 December 2020 13:35
To: Vincent Veyron <vv.lists@wanadoo.fr>
Cc: mod_perl list <modperl@perl.apache.org>
Subject: Re: suggestions for perl as web development language [EXT]

mod_perl is horribly inefficient because prefork is inefficient and because each request is single threaded. In addition to this, mod_perl also cannot provide websockets which are essential in a modern application.

On Mon, Dec 21, 2020 at 1:26 AM Vincent Veyron <vv.lists@wanadoo.fr<mailto:vv.lists@wanadoo.fr>> wrote:

[You forgot to cc the list ]

On Sun, 20 Dec 2020 23:16:03 -0500
John Dunlap <John@lariat.co<mailto:John@lariat.co>> wrote:

> We run 20 customers on a single box and our database has approximately 500
> tables. We run hundreds or thousands of queries per second.
>

500 tables is a lot more than what I typically handle. I'm sure it complicates things.

But see this post by James Smith in a recent thread :

http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/ajax/%3Cef383804cf394c53b48258531891d12b%40sanger.ac.uk%3E [mail-archives.apache.org]<https://urldefense.proofpoint.com/v2/url?u=http-3A__mail-2Darchives.apache.org_mod-5Fmbox_perl-2Dmodperl_202008.mbox_ajax_-253Cef383804cf394c53b48258531891d12b-2540sanger.ac.uk-253E&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=nK2GCkR9GuTuv3TffZBjRZgIDya7tetj5oHLLDBsn18&e=>

Easier to read in this archive :

http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/browser [mail-archives.apache.org]<https://urldefense.proofpoint.com/v2/url?u=http-3A__mail-2Darchives.apache.org_mod-5Fmbox_perl-2Dmodperl_202008.mbox_browser&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=Om-DkyClcEdOiq5HBizz-ydRhOziZbAP-AL_sR0eXAE&e=>

I also remember a post by a chinese guy who handled the same order of database size, in which he wrote that he had compared several frameworks and mod_perl was the fastest; but that was something like 10 years ago, and I can't find it anymore.

So I'm not sure how mod_perl could handle that kind of load and be horribly inefficient?

(I forgot to say in my previous post that over 50% of the time used by my script is spent on the _one_ query out of 120 that writes a smallish session hash to disk)

--
Bien à vous, Vincent Veyron

https://compta.libremen.com [compta.libremen.com]<https://urldefense.proofpoint.com/v2/url?u=https-3A__compta.libremen.com&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=mT-OlBL98gDcEsBMBidxank1GbEXq2zX6zPGOmpwobU&e=>
Logiciel libre de comptabilité générale en partie double


--
John Dunlap
CTO | Lariat

Direct:
john@lariat.co<mailto:john@lariat.co>

Customer Service:
877.268.6667
support@lariat.co<mailto:support@lariat.co>
[cid:image001.png@01D6D86F.18E6D9C0]



--
The Wellcome Sanger Institute is operated by Genome Research
Limited, a charity registered in England with number 1021457 and a
company registered in England with number 2742969, whose registered
office is 215 Euston Road, London, NW1 2BE.

We have hundreds of users polling our servers every few seconds just
waiting for events. In the near future, I'm going to have to do a refactor
to avoid users polling two different endpoints for different kinds of
events. The vast majority of the request in our access logs are polling
requests and they put significant load on our systems that I would prefer
to avoid. Additionally, web sockets aren't just for pushing content to the
browser. If you use them in the opposite direction, they improve
performance because you can avoid opening/closing a new connection and
because you can avoid the overhead of the HTTP protocol resulting in
shorter request times.

On Tue, Dec 22, 2020 at 8:32 AM James Smith <js5@sanger.ac.uk> wrote:

> There are not many applications which really benefit from multiple threads
> in web server environments unless you have very low load – as they are only
> efficient as they can use multiple cores, so you need stupidly speced
> machines to manage this load properly, and if you are using external
> resources can often be blocking anyway.
>
> Yes mod_perl doesn’t do web-sockets – but usually that isn’t an issue
> unless you have the need to push when people update content and you want it
> immediately available. There are others issues to consider with web-sockets
> e.g. port usage to keep the connections open. If you don’t need that
> immediate response – polling can achieve the desired effect – and Apache
> can even tell the client how long you need to wait before you send another
> connection.
>
>
>
>
>
> *From:* John Dunlap <John@lariat.co>
> *Sent:* 22 December 2020 13:35
> *To:* Vincent Veyron <vv.lists@wanadoo.fr>
> *Cc:* mod_perl list <modperl@perl.apache.org>
> *Subject:* Re: suggestions for perl as web development language [EXT]
>
>
>
> mod_perl is horribly inefficient because prefork is inefficient and
> because each request is single threaded. In addition to this, mod_perl also
> cannot provide websockets which are essential in a modern application.
>
>
>
> On Mon, Dec 21, 2020 at 1:26 AM Vincent Veyron <vv.lists@wanadoo.fr>
> wrote:
>
>
> [You forgot to cc the list ]
>
> On Sun, 20 Dec 2020 23:16:03 -0500
> John Dunlap <John@lariat.co> wrote:
>
> > We run 20 customers on a single box and our database has approximately
> 500
> > tables. We run hundreds or thousands of queries per second.
> >
>
> 500 tables is a lot more than what I typically handle. I'm sure it
> complicates things.
>
> But see this post by James Smith in a recent thread :
>
> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/ajax/%3Cef383804cf394c53b48258531891d12b%40sanger.ac.uk%3E
> [mail-archives.apache.org]
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__mail-2Darchives.apache.org_mod-5Fmbox_perl-2Dmodperl_202008.mbox_ajax_-253Cef383804cf394c53b48258531891d12b-2540sanger.ac.uk-253E&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=nK2GCkR9GuTuv3TffZBjRZgIDya7tetj5oHLLDBsn18&e=>
>
> Easier to read in this archive :
>
> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/browser
> [mail-archives.apache.org]
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__mail-2Darchives.apache.org_mod-5Fmbox_perl-2Dmodperl_202008.mbox_browser&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=Om-DkyClcEdOiq5HBizz-ydRhOziZbAP-AL_sR0eXAE&e=>
>
> I also remember a post by a chinese guy who handled the same order of
> database size, in which he wrote that he had compared several frameworks
> and mod_perl was the fastest; but that was something like 10 years ago, and
> I can't find it anymore.
>
> So I'm not sure how mod_perl could handle that kind of load and be
> horribly inefficient?
>
> (I forgot to say in my previous post that over 50% of the time used by my
> script is spent on the _one_ query out of 120 that writes a smallish
> session hash to disk)
>
> --
> Bien à vous, Vincent Veyron
>
> https://compta.libremen.com [compta.libremen.com]
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__compta.libremen.com&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=mT-OlBL98gDcEsBMBidxank1GbEXq2zX6zPGOmpwobU&e=>
> Logiciel libre de comptabilité générale en partie double
>
>
>
> --
>
> John Dunlap
>
> CTO | Lariat
>
>
>
> *Direct:*
>
> john@lariat.co
>
>
> *Customer Service:*
>
> 877.268.6667
>
> support@lariat.co
>
> -- The Wellcome Sanger Institute is operated by Genome Research Limited, a
> charity registered in England with number 1021457 and a company registered
> in England with number 2742969, whose registered office is 215 Euston Road,
> London, NW1 2BE.
>


--
John Dunlap
*CTO | Lariat *

*Direct:*
*john@lariat.co <john@lariat.co>*

*Customer Service:*
877.268.6667
support@lariat.co

Sounds like a classic use case for Kafka - your service should publish when
done and your clients should subscribe to their relevant topics. This is
not going to be scalable even with websockets.

On Tue, Dec 22, 2020 at 9:06 AM John Dunlap <John@lariat.co> wrote:

> We have hundreds of users polling our servers every few seconds just
> waiting for events. In the near future, I'm going to have to do a refactor
> to avoid users polling two different endpoints for different kinds of
> events. The vast majority of the request in our access logs are polling
> requests and they put significant load on our systems that I would prefer
> to avoid. Additionally, web sockets aren't just for pushing content to the
> browser. If you use them in the opposite direction, they improve
> performance because you can avoid opening/closing a new connection and
> because you can avoid the overhead of the HTTP protocol resulting in
> shorter request times.
>
> On Tue, Dec 22, 2020 at 8:32 AM James Smith <js5@sanger.ac.uk> wrote:
>
>> There are not many applications which really benefit from multiple
>> threads in web server environments unless you have very low load – as they
>> are only efficient as they can use multiple cores, so you need stupidly
>> speced machines to manage this load properly, and if you are using external
>> resources can often be blocking anyway.
>>
>> Yes mod_perl doesn’t do web-sockets – but usually that isn’t an issue
>> unless you have the need to push when people update content and you want it
>> immediately available. There are others issues to consider with web-sockets
>> e.g. port usage to keep the connections open. If you don’t need that
>> immediate response – polling can achieve the desired effect – and Apache
>> can even tell the client how long you need to wait before you send another
>> connection.
>>
>>
>>
>>
>>
>> *From:* John Dunlap <John@lariat.co>
>> *Sent:* 22 December 2020 13:35
>> *To:* Vincent Veyron <vv.lists@wanadoo.fr>
>> *Cc:* mod_perl list <modperl@perl.apache.org>
>> *Subject:* Re: suggestions for perl as web development language [EXT]
>>
>>
>>
>> mod_perl is horribly inefficient because prefork is inefficient and
>> because each request is single threaded. In addition to this, mod_perl also
>> cannot provide websockets which are essential in a modern application.
>>
>>
>>
>> On Mon, Dec 21, 2020 at 1:26 AM Vincent Veyron <vv.lists@wanadoo.fr>
>> wrote:
>>
>>
>> [You forgot to cc the list ]
>>
>> On Sun, 20 Dec 2020 23:16:03 -0500
>> John Dunlap <John@lariat.co> wrote:
>>
>> > We run 20 customers on a single box and our database has approximately
>> 500
>> > tables. We run hundreds or thousands of queries per second.
>> >
>>
>> 500 tables is a lot more than what I typically handle. I'm sure it
>> complicates things.
>>
>> But see this post by James Smith in a recent thread :
>>
>> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/ajax/%3Cef383804cf394c53b48258531891d12b%40sanger.ac.uk%3E
>> [mail-archives.apache.org]
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__mail-2Darchives.apache.org_mod-5Fmbox_perl-2Dmodperl_202008.mbox_ajax_-253Cef383804cf394c53b48258531891d12b-2540sanger.ac.uk-253E&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=nK2GCkR9GuTuv3TffZBjRZgIDya7tetj5oHLLDBsn18&e=>
>>
>> Easier to read in this archive :
>>
>> http://mail-archives.apache.org/mod_mbox/perl-modperl/202008.mbox/browser
>> [mail-archives.apache.org]
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__mail-2Darchives.apache.org_mod-5Fmbox_perl-2Dmodperl_202008.mbox_browser&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=Om-DkyClcEdOiq5HBizz-ydRhOziZbAP-AL_sR0eXAE&e=>
>>
>> I also remember a post by a chinese guy who handled the same order of
>> database size, in which he wrote that he had compared several frameworks
>> and mod_perl was the fastest; but that was something like 10 years ago, and
>> I can't find it anymore.
>>
>> So I'm not sure how mod_perl could handle that kind of load and be
>> horribly inefficient?
>>
>> (I forgot to say in my previous post that over 50% of the time used by my
>> script is spent on the _one_ query out of 120 that writes a smallish
>> session hash to disk)
>>
>> --
>> Bien à vous, Vincent Veyron
>>
>> https://compta.libremen.com [compta.libremen.com]
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__compta.libremen.com&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=d_ZhFPlvaXw1KbAmeExguXy1fSbpEKzIuAPZMOJ9h78&s=mT-OlBL98gDcEsBMBidxank1GbEXq2zX6zPGOmpwobU&e=>
>> Logiciel libre de comptabilité générale en partie double
>>
>>
>>
>> --
>>
>> John Dunlap
>>
>> CTO | Lariat
>>
>>
>>
>> *Direct:*
>>
>> john@lariat.co
>>
>>
>> *Customer Service:*
>>
>> 877.268.6667
>>
>> support@lariat.co
>>
>> -- The Wellcome Sanger Institute is operated by Genome Research Limited,
>> a charity registered in England with number 1021457 and a company
>> registered in England with number 2742969, whose registered office is 215
>> Euston Road, London, NW1 2BE.
>>
>
>
> --
> John Dunlap
> *CTO | Lariat *
>
> *Direct:*
> *john@lariat.co <john@lariat.co>*
>
> *Customer Service:*
> 877.268.6667
> support@lariat.co
>

On Sun, Dec 20, 2020 at 2:03 PM Mithun Bhattacharya <mithnb@gmail.com> wrote:
>
> Your external facing apache instance would do the SSL part and use mod_proxy to redirect the request to another instance of apache which implements the actual functionality. Just remember the second instance needs to run on a different port and that it doesnt have to talk to the outside world.

That looks good. Is there any concern about security in the backend?
Who should own the process, the apache process owner I assume.

> Did you check out the practical mod_perl article ? https://docstore.mik.ua/orelly/weblinux2/modperl/ch12_07.htm

I did, good reference. Thanks!

It looks to me like I can run Raku's Cro http server process on the
back end instead of using mod_perl.

Happy New Year, all!

-Tom

Choice of Programming language is totally up to you :)

As for security SSL shouldnt be your only protection and all
authorization/authentication will have to be implemented in the internal
service.

On Wed, Dec 30, 2020 at 7:19 AM Tom Browder <tom.browder@gmail.com> wrote:

> On Sun, Dec 20, 2020 at 2:03 PM Mithun Bhattacharya <mithnb@gmail.com>
> wrote:
> >
> > Your external facing apache instance would do the SSL part and use
> mod_proxy to redirect the request to another instance of apache which
> implements the actual functionality. Just remember the second instance
> needs to run on a different port and that it doesnt have to talk to the
> outside world.
>
> That looks good. Is there any concern about security in the backend?
> Who should own the process, the apache process owner I assume.
>
> > Did you check out the practical mod_perl article ?
> https://docstore.mik.ua/orelly/weblinux2/modperl/ch12_07.htm
>
> I did, good reference. Thanks!
>
> It looks to me like I can run Raku's Cro http server process on the
> back end instead of using mod_perl.
>
> Happy New Year, all!
>
> -Tom
>