Mailing List Archive: Thread-safe & vars scope clarification

Thread-safe & vars scope clarification

Apr 4, 2016, 1:44 PM

Post #1 of 6 (2492 views)

Hello,

I have some questions regarding mod_perl, threads, thread-safe functions, special vars scope...

From https://perl.apache.org/docs/2.0/user/coding/coding.html#toc_Thread_environment_Issues :
"if you chdir() in one thread, all other thread now see the current working directory of that thread that chdir()'ed to that directory"
Do we speak about threads from threaded MPM ?
(so the specific Apache process which owns the thread running this Perl script, and all its (the process) threads, will chdir()).

From the same page :
"Other usually problematic (non thread-safe) functions include readdir(), srand(), etc."
Why readdir() would not be thread-safe ? I can't see why. Perhaps readdir() chdir() to read a directory content ?
How could we then read dir content being thread-safe ?

From https://perl.apache.org/docs/general/perl_reference/perl_reference.html#toc_The_Scope_of_the_Special_Perl_Variables :
"In the mod_perl situation where the child server doesn't usually exit, if in one of your scripts you modify a global variable it will be changed for the rest of the process' life and will affect all the scripts executed by the same process."
So in a threaded MPM, if one of the threads modifies one of the special Perl variables, it will instantly affect all other threads of this process ?

What about variables like $? ? (return code of system() calls)
And ${^CHILD_ERROR_NATIVE} ? (return code too)
Does their value stay local to the thread ? / Does each thread have these variables with their own proper value ?
Or is it unsafe to read them because they could be modified by another thread ?

Thank you very much for your clarification,

Best regards,

Ben

Re: Thread-safe & vars scope clarification [ In reply to ]

pharkins at gmail

Apr 4, 2016, 1:49 PM

Post #2 of 6 (2464 views)

Permalink

Hi Ben.

Before you get too far into the details of using threads, can I ask why
you're considering it? The memory footprint and performance of using forked
processes with Perl is generally going to be better than that of threads,
due to copy-on-write.

- Perrin

On Mon, Apr 4, 2016 at 4:44 PM, Ben RUBSON <ben.rubson@gmail.com> wrote:

> Hello,
>
> I have some questions regarding mod_perl, threads, thread-safe functions,
> special vars scope...
>
> From
> https://perl.apache.org/docs/2.0/user/coding/coding.html#toc_Thread_environment_Issues
> :
> "if you chdir() in one thread, all other thread now see the current
> working directory of that thread that chdir()'ed to that directory"
> Do we speak about threads from threaded MPM ?
> (so the specific Apache process which owns the thread running this Perl
> script, and all its (the process) threads, will chdir()).
>
> From the same page :
> "Other usually problematic (non thread-safe) functions include readdir(),
> srand(), etc."
> Why readdir() would not be thread-safe ? I can't see why. Perhaps
> readdir() chdir() to read a directory content ?
> How could we then read dir content being thread-safe ?
>
> From
> https://perl.apache.org/docs/general/perl_reference/perl_reference.html#toc_The_Scope_of_the_Special_Perl_Variables
> :
> "In the mod_perl situation where the child server doesn't usually exit, if
> in one of your scripts you modify a global variable it will be changed for
> the rest of the process' life and will affect all the scripts executed by
> the same process."
> So in a threaded MPM, if one of the threads modifies one of the special
> Perl variables, it will instantly affect all other threads of this process ?
>
> What about variables like $? ? (return code of system() calls)
> And ${^CHILD_ERROR_NATIVE} ? (return code too)
> Does their value stay local to the thread ? / Does each thread have these
> variables with their own proper value ?
> Or is it unsafe to read them because they could be modified by another
> thread ?
>
> Thank you very much for your clarification,
>
> Best regards,
>
> Ben
>
>

Re: Thread-safe & vars scope clarification [ In reply to ]

ben.rubson at gmail

Apr 4, 2016, 2:03 PM

Post #3 of 6 (2465 views)

Permalink

Hi Perrin,

Thank you for your answer.

Well, for the moment I only use the prefork (non-threaded) MPM.
But I would have liked to test performance of the threaded MPM, so I would have liked my code to be thread-ready.

According to this :
https://perl.apache.org/docs/2.0/user/performance/mpm.html#toc_Memory_Requirements_in_Threaded_MPM
Sounds like threads could have an interest, but according to your message, should not be a general case.

Thank you,

Ben

> Le 4 avr. 2016 Ã 22:49, Perrin Harkins <pharkins@gmail.com> a Ã©crit :
>
> Hi Ben.
>
> Before you get too far into the details of using threads, can I ask why you're considering it? The memory footprint and performance of using forked processes with Perl is generally going to be better than that of threads, due to copy-on-write.
>
> - Perrin
>
> On Mon, Apr 4, 2016 at 4:44 PM, Ben RUBSON <ben.rubson@gmail.com <mailto:ben.rubson@gmail.com>> wrote:
> Hello,
>
> I have some questions regarding mod_perl, threads, thread-safe functions, special vars scope...
>
> From https://perl.apache.org/docs/2.0/user/coding/coding.html#toc_Thread_environment_Issues <https://perl.apache.org/docs/2.0/user/coding/coding.html#toc_Thread_environment_Issues> :
> "if you chdir() in one thread, all other thread now see the current working directory of that thread that chdir()'ed to that directory"
> Do we speak about threads from threaded MPM ?
> (so the specific Apache process which owns the thread running this Perl script, and all its (the process) threads, will chdir()).
>
> From the same page :
> "Other usually problematic (non thread-safe) functions include readdir(), srand(), etc."
> Why readdir() would not be thread-safe ? I can't see why. Perhaps readdir() chdir() to read a directory content ?
> How could we then read dir content being thread-safe ?
>
> From https://perl.apache.org/docs/general/perl_reference/perl_reference.html#toc_The_Scope_of_the_Special_Perl_Variables <https://perl.apache.org/docs/general/perl_reference/perl_reference.html#toc_The_Scope_of_the_Special_Perl_Variables> :
> "In the mod_perl situation where the child server doesn't usually exit, if in one of your scripts you modify a global variable it will be changed for the rest of the process' life and will affect all the scripts executed by the same process."
> So in a threaded MPM, if one of the threads modifies one of the special Perl variables, it will instantly affect all other threads of this process ?
>
> What about variables like $? ? (return code of system() calls)
> And ${^CHILD_ERROR_NATIVE} ? (return code too)
> Does their value stay local to the thread ? / Does each thread have these variables with their own proper value ?
> Or is it unsafe to read them because they could be modified by another thread ?
>
> Thank you very much for your clarification,
>
> Best regards,
>
> Ben
>
>

Re: Thread-safe & vars scope clarification [ In reply to ]

pharkins at gmail

Apr 4, 2016, 3:20 PM

Post #4 of 6 (2459 views)

Permalink

If you want to try it yourself and report back, I'm sure we'd all be
interested in what you find out. The main thing I'm aware of is that
copy-on-write works very well for preforking and threads usually can't
match it, but maybe your application is different.

Regarding your thread-safety questions, chdir() is global to the process,
so not thread-safe. I think the issue with readdir() is that open directory
handles don't get copied when threads are spawned, but maybe there's more
to it. The special Perl vars are not a problem with threads, since each
thread is a separate interpreter.

You can read about the limitations of threads here:
http://perldoc.perl.org/threads.html#BUGS-AND-LIMITATIONS

- Perrin

On Mon, Apr 4, 2016 at 5:03 PM, Ben RUBSON <ben.rubson@gmail.com> wrote:

> Hi Perrin,
>
> Thank you for your answer.
>
> Well, for the moment I only use the prefork (non-threaded) MPM.
> But I would have liked to test performance of the threaded MPM, so I would
> have liked my code to be thread-ready.
>
> According to this :
>
> https://perl.apache.org/docs/2.0/user/performance/mpm.html#toc_Memory_Requirements_in_Threaded_MPM
> Sounds like threads could have an interest, but according to your message,
> should not be a general case.
>
> Thank you,
>
> Ben
>
> Le 4 avr. 2016 Ã 22:49, Perrin Harkins <pharkins@gmail.com> a Ã©crit :
>
> Hi Ben.
>
> Before you get too far into the details of using threads, can I ask why
> you're considering it? The memory footprint and performance of using forked
> processes with Perl is generally going to be better than that of threads,
> due to copy-on-write.
>
> - Perrin
>
> On Mon, Apr 4, 2016 at 4:44 PM, Ben RUBSON <ben.rubson@gmail.com> wrote:
>
>> Hello,
>>
>> I have some questions regarding mod_perl, threads, thread-safe functions,
>> special vars scope...
>>
>> From
>> https://perl.apache.org/docs/2.0/user/coding/coding.html#toc_Thread_environment_Issues
>> :
>> "if you chdir() in one thread, all other thread now see the current
>> working directory of that thread that chdir()'ed to that directory"
>> Do we speak about threads from threaded MPM ?
>> (so the specific Apache process which owns the thread running this Perl
>> script, and all its (the process) threads, will chdir()).
>>
>> From the same page :
>> "Other usually problematic (non thread-safe) functions include readdir(),
>> srand(), etc."
>> Why readdir() would not be thread-safe ? I can't see why. Perhaps
>> readdir() chdir() to read a directory content ?
>> How could we then read dir content being thread-safe ?
>>
>> From
>> https://perl.apache.org/docs/general/perl_reference/perl_reference.html#toc_The_Scope_of_the_Special_Perl_Variables
>> :
>> "In the mod_perl situation where the child server doesn't usually exit,
>> if in one of your scripts you modify a global variable it will be changed
>> for the rest of the process' life and will affect all the scripts executed
>> by the same process."
>> So in a threaded MPM, if one of the threads modifies one of the special
>> Perl variables, it will instantly affect all other threads of this process ?
>>
>> What about variables like $? ? (return code of system() calls)
>> And ${^CHILD_ERROR_NATIVE} ? (return code too)
>> Does their value stay local to the thread ? / Does each thread have these
>> variables with their own proper value ?
>> Or is it unsafe to read them because they could be modified by another
>> thread ?
>>
>> Thank you very much for your clarification,
>>
>> Best regards,
>>
>> Ben
>>
>>
>
>

Re: Thread-safe & vars scope clarification [ In reply to ]

lembark at wrkhors

Apr 5, 2016, 6:11 AM

Post #5 of 6 (2446 views)

Permalink

> If you want to try it yourself and report back, I'm sure we'd all be
> interested in what you find out. The main thing I'm aware of is that
> copy-on-write works very well for preforking and threads usually can't
> match it, but maybe your application is different.
>
> Regarding your thread-safety questions, chdir() is global to the process,
> so not thread-safe. I think the issue with readdir() is that open directory
> handles don't get copied when threads are spawned, but maybe there's more
> to it. The special Perl vars are not a problem with threads, since each
> thread is a separate interpreter.
>
> You can read about the limitations of threads here:
> http://perldoc.perl.org/threads.html#BUGS-AND-LIMITATIONS

File handles, directory handles, cwd, uid, euid, gid, egid, signal
mask (%SIG): anything defined in the process struct is shared among
all threads. For example, the BSD proc sturuct looks like:

<http://people.freebsd.org/~meganm/data/tutorials/ddwg/ddwg63.html>

which gives a rough idea of what is in common.

The only thing you can do with threads that forks make difficult is
having multiple streams of execution working on a common data
structure in parallel with about-to-block threads returning control
before they actually do block. Nice for a web server or database, but
you probably aren't writing an RDBMS in Perl :-)

By the time your computing problem has percolated up to the level of
Perl (vs. C) you are probably better off dealing with the work using
forks (at least on *NIX) in order to avoid all of the locking, memory,
and pool-manglement issues.

--
Steven Lembark 3646 Flora Pl
Workhorse Computing St Louis, MO 63110
lembark@wrkhors.com +1 888 359 3508

Re: Thread-safe & vars scope clarification [ In reply to ]

ben.rubson at gmail

Apr 7, 2016, 7:39 AM

Post #6 of 6 (2437 views)

Permalink

>> If you want to try it yourself and report back, I'm sure we'd all be
>> interested in what you find out. The main thing I'm aware of is that
>> copy-on-write works very well for preforking and threads usually can't
>> match it, but maybe your application is different.
>>
>> Regarding your thread-safety questions, chdir() is global to the process,
>> so not thread-safe. I think the issue with readdir() is that open directory
>> handles don't get copied when threads are spawned, but maybe there's more
>> to it. The special Perl vars are not a problem with threads, since each
>> thread is a separate interpreter.
>>
>> You can read about the limitations of threads here:
>> http://perldoc.perl.org/threads.html#BUGS-AND-LIMITATIONS
>
> File handles, directory handles, cwd, uid, euid, gid, egid, signal
> mask (%SIG): anything defined in the process struct is shared among
> all threads. For example, the BSD proc sturuct looks like:
>
> <http://people.freebsd.org/~meganm/data/tutorials/ddwg/ddwg63.html>
>
> which gives a rough idea of what is in common.
>
> The only thing you can do with threads that forks make difficult is
> having multiple streams of execution working on a common data
> structure in parallel with about-to-block threads returning control
> before they actually do block. Nice for a web server or database, but
> you probably aren't writing an RDBMS in Perl :-)
>
> By the time your computing problem has percolated up to the level of
> Perl (vs. C) you are probably better off dealing with the work using
> forks (at least on *NIX) in order to avoid all of the locking, memory,
> and pool-manglement issues.

Perrin, Steven, sorry for my late answer and thank you very much for your clarification and useful links.

So no I'm not writing a RDBMS :) but a filesystem API which of course uses readdir().
So according to your answers, threads sound dangerous in this situation.

This link was also very useful to understand how Apache threads work with mod_perl :
https://perl.apache.org/docs/2.0/user/intro/overview.html#Threads_Support

The difficulty (from my point of view) seems to have the answer to "is my code thread-safe ?".
I did not found an exhaustive list of unsafe functions, and documentation does not help much :
"Other usually problematic functions include readdir(), srand(), etc."

In addition, seems to hava exceptions :
Here http://perldoc.perl.org/threads.html#BUGS-AND-LIMITATION, paragraph "Open directory handles" :
Seems that opendir() (so readdir() ?) could be thread-safe if system has fchdir support.
But how to be sure...

I think I will stay with the classic forked MPM for security reasons :)

Thank you !

Ben