Mailing List Archive

On 16.05.2016 16:38, Vincent Veyron wrote:
> Hi,
>
> I use the code shown here :
>
> http://pastebin.com/6YL9FWGX
>
> in a mod_perl handler.
>
> At line 57, $args->{_token_id} ||= ... generates a random token that is used to uniquely identify the request (users may reload the URL in two different windows), and added to the request's arguments.
>
> Out of five different servers, the code works fine on four machines, and a different token is generated every time the page is loaded or re-loaded. On one server however, a previous token is being re-used. The servers are not identical, but all run the same up-to-date Debian Jessie, using the same distribution packages of Apache and mod_perl
>
> Below is what is printed in the error log of the faulty system by Data::Dumper; as you can see, on the third iteration, the first token is reused.
>
> Before : $VAR1 = {
> 'id_entry' => '16007',
> 'mois' => 0,
> 'open_journal' => 'Fournisseurs'
> };
> After : $VAR1 = {
> '_token_id' => 'Gh63Y2J9YYaNuReIsI8JAEj9oCY39oiy', <------------------------------------- token 1
> 'id_entry' => '16007',
> 'mois' => 0,
> 'open_journal' => 'Fournisseurs'
> };
> Before : $VAR1 = {
> 'id_entry' => '16007',
> 'open_journal' => 'Fournisseurs',
> 'mois' => 0
> };
> After : $VAR1 = {
> 'open_journal' => 'Fournisseurs',
> 'mois' => 0,
> 'id_entry' => '16007',
> '_token_id' => 'EHL2mm1LvmFIgPeYEFYO8dKt71lWiwkP' <------------------------------------- token 2
> };
> Before : $VAR1 = {
> 'mois' => 0,
> 'open_journal' => 'Fournisseurs',
> 'id_entry' => '16257'
> };
> After : $VAR1 = {
> 'open_journal' => 'Fournisseurs',
> 'mois' => 0,
> 'id_entry' => '16257',
> '_token_id' => 'Gh63Y2J9YYaNuReIsI8JAEj9oCY39oiy' <------------------------------------- token 1 again
> };
>
> What can I do to find the cause of this misbehaviour?
>

Hi.

I have not really analysed the instruction by which you create the token, to check if
there was not some flaw there which would lead to generate a less-than-random token.
But I have a suspicion and a theory nevertheless :

1) What is the Apache MPM used on the 5 machines (prefork, worker, etc.) ?
You can find out by entering "apache2ctl -l" on each machine.

2) the rand() function, when called repeatedly, provides a *pseudo*-random series of
return values.
Example : run this little program, *repeatedly*, on any one of the servers :

#!/usr/bin/perl
# test_rand.pl : simple test for rand()
use strict;
use warnings;
srand(12345);
for my $it (1..10) {
print "it # $it : " . sprintf("%05d",int(rand(99999))) . "\n";
}

3) what does this show ?
That when you start with the same "seed" (srand(12345)), even in totally independent
processes running one after the other, you obtain exactly the same sequence of
pseudo-random numbers when you then call rand() repeatedly.

4) you are running Apache + mod_perl. If this is a "prefork" configuration, it means that
there will be a "main Apache process" (which actually receives the browser requests) , and
a number of Apache "children" started in parallel, each with its own persistent perl
interpreter. When a request comes into the main Apache, it looks in some table, to find a
"child" process which isn't currently doing anything, and if found, it passes the request
to that one for processing.
This is largely unpredictable : depending on how many requests come in over a period of
time, and the number of started Apache children processes, and the time it takes to
process one request, one or the other child process can be free or not, to handle the
latest request.
It is thus possible that the same child process will handle a series of requests in a row,
or that each child will be called in turn, or whatever combination.
(And also, according to how I understand your application, Apache will receive a
succession of requests, some with already a token, and some not. So rand() will be called
or not in any child, depending on how requests exactly come in.

5) now look at the description of rand() and srand() here :
http://perldoc.perl.org/functions/rand.html

It says that the first time rand() is called by any process, if srand() has not been
called before, it is called automatically wihout argument, to return some seed for the
first rand() call.
I do not know exactly what srand() returns in such a case, but if it happened in each of
these "Apache children processes + perl", to return initially the same seed, then each of
these Apache processes, subsequently, would return the same series of successive "random"
numbers when rand() is called.

So now put all of that together, and I believe that it is almost guaranteed that you will
regularly get the same keys, on all your servers, not just the one you cite as different.

Also, if I may say so, the function that you are using to generate your "random" unique
session-id :

join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32 ;

looks at first sight to me like quite inefficient and probably likely to generate the same
string regularly, even if it does not look that way.
(The only variable there is rand(), and it can only return values between 0 and 62).
I haven't done the math, but it just looks like an awful lot of function calls, int/string
conversions, array contruction and retrievals etc., to get a 32-char string that is
finally not so random.
Would it not be better to call rand() only a few times, with a much larger argument, so
that it returns a much wider range of values, and then map these values to letters for
example using a hex sprintf, and then concatenate these hex strings ?
(Or better, use one of the CPAN modules specially designed to do similar things,
efficiently and really randomly)

> On May 16, 2016, at 10:15 AM, André Warnier (tomcat) <aw@ice-sa.com> wrote:
>
>
> join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32 ;
>
> looks at first sight to me like quite inefficient and probably likely to generate the same string regularly, even if it does not look that way.
> (The only variable there is rand(), and it can only return values between 0 and 62).

The function is meant to map a random element from the 62-element-long array (0..9,"a".."z","A".."Z”) (hence a rand() call to generate a number from 0 and 62), 32 times, and join them into a string.

Although I think that should really be rand(9+26*2) to properly generate array indices for the entire array and no more. With a number between 0 and 62 (63 numbers) and a 62-element array, you’ll be retrieving nulls from the array 1/62 calls, but all that means is that the string is one char shorter for each time '62’ comes up...

So long as rand is properly seeded, you should not get repeats, at least not frequently enough to ever notice, I’d think.

This is textbook Perl, as in I’m pretty sure it’s out of one of Larry Wall’s books; I use it to generate random strings for cookies.

If it’s properly seeded in the original code, it should either work or not work on all five servers. Not working on one out of the five makes me think maybe there’s some sort of weird caching issue.

--
Bruce Johnson
University of Arizona
College of Pharmacy
Information Technology Group

Institutions do not have opinions, merely customs

On 16.05.2016 20:03, Bruce Johnson wrote:
>
>> On May 16, 2016, at 10:15 AM, André Warnier (tomcat) <aw@ice-sa.com> wrote:
>>
>>
>> join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32 ;
>>
>> looks at first sight to me like quite inefficient and probably likely to generate the same string regularly, even if it does not look that way.
>> (The only variable there is rand(), and it can only return values between 0 and 62).
>
> The function is meant to map a random element from the 62-element-long array (0..9,"a".."z","A".."Z”) (hence a rand() call to generate a number from 0 and 62), 32 times, and join them into a string.
>
> Although I think that should really be rand(9+26*2) to properly generate array indices for the entire array and no more. With a number between 0 and 62 (63 numbers) and a 62-element array, you’ll be retrieving nulls from the array 1/62 calls, but all that means is that the string is one char shorter for each time '62’ comes up...
>
> So long as rand is properly seeded, you should not get repeats, at least not frequently enough to ever notice, I’d think.
>
> This is textbook Perl, as in I’m pretty sure it’s out of one of Larry Wall’s books; I use it to generate random strings for cookies.
>

Maybe the textbook example was meant to show the power of the syntax, but not necessarily
the efficiency of it.
I interpret this as doing the following (if we ignore any optimisation which the perl
compiler may make by itself) :

quote perldoc
map EXPR,LIST

Evaluates the BLOCK or EXPR for each element of LIST (locally setting $_ to each element)
and returns the list value composed of the results of each such evaluation. In scalar
context, returns the total number of elements so generated. Evaluates BLOCK or EXPR in
list context, so each element of LIST may produce zero, one, or more elements in the
returned value.
unquote

LIST in this case is 1..32 = 32 elements.
EXPR is (0..9,"a".."z","A".."Z") which is another list (say LIST-2), of 62 elements.
Thus, we are 32 times evaluating (building) a list LIST-2 of 62 elements.
(Always the same one, but does perl know this and optimise it away ?)
Out of which LIST-2 then (at each of the 32 iterations) we extract just 1 "random"
element, by virtue of the [rand(62)] index into the list LIST-2.
I am not sure immediately of the effect of the leading +(LIST-2), but it may have the
effect of forcing some internal conversion of the elements of (LIST-2).
At the end of the 32 iterations of
map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32
we have an array of 32 single-character elements.
Which we then join into a 32-character string by means of the join().

I may be wrong, but at least intuitively, this does not seem to be an optimal way to
obtain a 32-char long random key.

Mathematically, I think such a string provides for 62 ** 32 combinations, which is ..
2,0859248397665137523388883849312 e+93
according to my pocket calculator.
That may be more than the total number of atoms in the known Universe.
Which I have also intuitively a tendency to consider a bit excessive, for generating a
simple time-limited session key for a webapp.

Note also that the fact that there are theoretically that many possible combinations, does
not mean that all of them are equally likely.
For that to be the case, rand() would have to return a series of really random responses,
which we know it doesn't.

> If it’s properly seeded in the original code, it should either work or not work on all five servers. Not working on one out of the five makes me think maybe there’s some sort of weird caching issue.
>

I agree, that the seed is the main issue.
The code should insure that for each server process (+ perl interpreter), srand() is
called once, whence that process starts (or the first time the webapp is run), and with a
different seed for each process.
(A BEGIN block with a seed based on the time (hi-res) may be a solution.)

But the fact that it does not evidently work on one of the servers only, may be due to
many factors, such as the rate at which already-keyed and not-already-keyed requests are
hitting each server, their order, how many server processes there are to respond to the
requests, and how long it takes to respond to one request.
So it may also be that "it does not work" on all servers, but that it has only been so far
noticed on the one.
If all server processes on one server happen to use the same seed, then it depends
on which server process handles the next unkeyed request. If it happens to be, for that
server process, the n-th unkeyed request that it handles (and other server processes have
already handled more than n requests before), then the key generated by this server
process will duplicate one which has already been issued before, no matter how many
different keys are theoretically possible.
And whether this causes problems or not (or gets noticed), may only depend on whether the
session which previously used the same key, has already concluded or not.

> On May 16, 2016, at 1:26 PM, André Warnier <aw@ice-sa.com> wrote:
>
> I may be wrong, but at least intuitively, this does not seem to be an optimal way to obtain a 32-char long random key.
>

It’s about 30% slower to run

join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32;

versus

$r= join "", map @a[rand(10+26*2)], 1..32;

With "@a =map +(0..9,"a".."z","A".."Z”);” declared only once, based on some Q&D testing.

(I had to boost the iterations to 2.5e^6 times for each mode to reach 47 v 36 seconds on my desktop here, so in practice running once per page change on the server it’s not going to be noticeable either way)

I agree that 32 bytes are probably excessive, but on the other hand it also makes it more unlikely even a poor random number generator would generate two identical strings in such close proximity, let alone do it regularly.


>> If it’s properly seeded in the original code, it should either work or not work on all five servers. Not working on one out of the five makes me think maybe there’s some sort of weird caching issue.
>>
>
> I agree, that the seed is the main issue.
> The code should insure that for each server process (+ perl interpreter), srand() is called once, whence that process starts (or the first time the webapp is run), and with a different seed for each process.

This is automatic when rand() is called without srand() being invoked. <http://perldoc.perl.org/functions/srand.html> so it’s already being done when the code is run on each server + new interpreter. The only way to get a repeatable random sequence is to explicitly call srand() with a parameter.

The perldoc page on rand() explicitly says that it’s not cryptographically safe, but there’s a very large gap between "cryptographically safe” and "quickly and readily repeats randomly generated 32-byte strings”. It’s good enough for this purpose.

I really do not think that the source of the random token is the issue here, but the server (or something else, a load balancer, something like that?) caching the request parameters for some reason.

Possibly the server is not recognizing it as a new request, and thus not generating the new token.

I don’t think it would be likely for Vincent to ever see this once, let alone have it rise to the issue of a problem if it were strictly about non-randomness of the rand() function.

--
Bruce Johnson
University of Arizona
College of Pharmacy
Information Technology Group

Institutions do not have opinions, merely customs

On Mon, 16 May 2016 22:45:14 +0000
"Bruce Johnson" <johnson@Pharmacy.Arizona.EDU> wrote:

>
> I don’t think it would be likely for Vincent to ever see this once, let alone have it rise to the issue of a problem if it were strictly about non-randomness of the rand() function.
>

Indeed : I also use the function to generate cookies; the strings are stored in a database table with a primary key on the token ; running 'ab -c 4 -n 4000 "http://my/url/here"' 3 times yields 12 000 unique keys on a functional server.

The machine where the error showed up can't get two different strings in a row. It's a backup server, so I'm the only one using it for test purposes, which is why I did not notice the error before; it may have been present for a long time.

Below is what is installed on each machine, the first one (arsene) being the faulty one; it is a rather old Dell desktop, the others are dedicated servers in datacenters, such as kimsufi.com, online.net. Everything is on bare metal, and incidentally not as uniform as I thought :-(
reload-perl is installed but disabled in apache2.conf

perl -v on the faulty machine (arsene) :
This is perl 5, version 20, subversion 2 (v5.20.2) built for i586-linux-gnu-thread-multi-64int
perl -v on the good ones :
This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi


arsene (cpuinfo : Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz)
ii apache2 2.4.10-10+deb8u4 i386 Apache HTTP Server
ii apache2-bin 2.4.10-10+deb8u4 i386 Apache HTTP Server (modules and other binary files)
ii apache2-data 2.4.10-10+deb8u4 all Apache HTTP Server (common files)
ii apache2-mpm-prefork 2.4.10-10+deb8u4 i386 transitional prefork MPM package for apache2
ii apache2-utils 2.4.10-10+deb8u4 i386 Apache HTTP Server (utility programs for web servers)
ii apache2.2-common 2.4.10-10+deb8u4 i386 Transitional package for apache2
ii libapache2-mod-apreq2 2.13-4+b1 i386 generic Apache request library - Apache module
ii libapache2-mod-perl2 2.0.9~1624218-2+deb8u1 i386 Integration of perl with the Apache2 web server
ii libapache2-reload-perl 0.12-3 all module for reloading Perl modules when changed on disk
ii libapache2-request-perl 2.13-4+b1 i386 generic Apache request library - Perl modules

kimsufi (cpuinfo : Intel(R) Atom(TM) CPU N2800 @ 1.86GHz)
ii apache2 2.4.10-10+deb8u4 amd64 Apache HTTP Server
ii apache2-bin 2.4.10-10+deb8u4 amd64 Apache HTTP Server (modules and other binary files)
ii apache2-data 2.4.10-10+deb8u4 all Apache HTTP Server (common files)
ii apache2-mpm-prefork 2.4.10-10+deb8u4 amd64 transitional prefork MPM package for apache2
ii apache2-utils 2.4.10-10+deb8u4 amd64 Apache HTTP Server (utility programs for web servers)
ii apache2.2-common 2.4.10-10+deb8u4 amd64 Transitional package for apache2
ii libapache2-mod-apreq2 2.13-4+b1 amd64 generic Apache request library - Apache module
ii libapache2-mod-perl2 2.0.9~1624218-2+deb8u1 amd64 Integration of perl with the Apache2 web server
ii libapache2-reload-perl 0.12-3 all module for reloading Perl modules when changed on disk
ii libapache2-request-perl 2.13-4+b1 amd64 generic Apache request library - Perl modules

online (cpuinfo : VIA Nano processor U2250 (1.6GHz Capable))
ii apache2 2.4.10-10+deb8u4 amd64 Apache HTTP Server
ii apache2-bin 2.4.10-10+deb8u4 amd64 Apache HTTP Server (modules and other binary files)
ii apache2-data 2.4.10-10+deb8u4 all Apache HTTP Server (common files)
ii apache2-mpm-prefork 2.4.10-10+deb8u4 amd64 transitional prefork MPM package for apache2
ii apache2-utils 2.4.10-10+deb8u4 amd64 Apache HTTP Server (utility programs for web servers)
ii libapache2-mod-apreq2 2.13-4+b1 amd64 generic Apache request library - Apache module
ii libapache2-mod-perl2 2.0.9~1624218-2+deb8u1 amd64 Integration of perl with the Apache2 web server
ii libapache2-request-perl 2.13-4+b1 amd64 generic Apache request library - Perl modules

ovh (cpuinfo : Intel(R) Celeron(R) CPU 220 @ 1.20GHz)
ii apache2 2.4.10-10+deb8u4 amd64 Apache HTTP Server
ii apache2-bin 2.4.10-10+deb8u4 amd64 Apache HTTP Server (modules and other binary files)
ii apache2-data 2.4.10-10+deb8u4 all Apache HTTP Server (common files)
ii apache2-utils 2.4.10-10+deb8u4 amd64 Apache HTTP Server (utility programs for web servers)
ii libapache2-mod-apreq2 2.13-4+b1 amd64 generic Apache request library - Apache module
ii libapache2-mod-perl2 2.0.9~1624218-2+deb8u1 amd64 Integration of perl with the Apache2 web server
ii libapache2-reload-perl 0.12-3 all module for reloading Perl modules when changed on disk
ii libapache2-request-perl 2.13-4+b1 amd64 generic Apache request library - Perl modules

--
Bien à vous, Vincent Veyron

https://marica.fr/
Gestion des sinistres assurance et des dossiers juridiques

On 17.05.2016 08:28, Vincent Veyron wrote:
> On Mon, 16 May 2016 22:45:14 +0000
> "Bruce Johnson" <johnson@Pharmacy.Arizona.EDU> wrote:
>
>>
>> I don’t think it would be likely for Vincent to ever see this once, let alone have it rise to the issue of a problem if it were strictly about non-randomness of the rand() function.
>>
>
> Indeed : I also use the function to generate cookies; the strings are stored in a database table with a primary key on the token ; running 'ab -c 4 -n 4000 "http://my/url/here"' 3 times yields 12 000 unique keys on a functional server.
>
> The machine where the error showed up can't get two different strings in a row. It's a backup server, so I'm the only one using it for test purposes, which is why I did not notice the error before; it may have been present for a long time.
>
> Below is what is installed on each machine, the first one (arsene) being the faulty one; it is a rather old Dell desktop, the others are dedicated servers in datacenters, such as kimsufi.com, online.net. Everything is on bare metal, and incidentally not as uniform as I thought :-(
> reload-perl is installed but disabled in apache2.conf
>
> perl -v on the faulty machine (arsene) :
> This is perl 5, version 20, subversion 2 (v5.20.2) built for i586-linux-gnu-thread-multi-64int
> perl -v on the good ones :
> This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux-gnu-thread-multi
>
>
> arsene (cpuinfo : Intel(R) Core(TM)2 Duo CPU E7500 @ 2.93GHz)
> ii apache2 2.4.10-10+deb8u4 i386 Apache HTTP Server
> ii apache2-bin 2.4.10-10+deb8u4 i386 Apache HTTP Server (modules and other binary files)
> ii apache2-data 2.4.10-10+deb8u4 all Apache HTTP Server (common files)
> ii apache2-mpm-prefork 2.4.10-10+deb8u4 i386 transitional prefork MPM package for apache2
> ii apache2-utils 2.4.10-10+deb8u4 i386 Apache HTTP Server (utility programs for web servers)
> ii apache2.2-common 2.4.10-10+deb8u4 i386 Transitional package for apache2
> ii libapache2-mod-apreq2 2.13-4+b1 i386 generic Apache request library - Apache module
> ii libapache2-mod-perl2 2.0.9~1624218-2+deb8u1 i386 Integration of perl with the Apache2 web server
> ii libapache2-reload-perl 0.12-3 all module for reloading Perl modules when changed on disk
> ii libapache2-request-perl 2.13-4+b1 i386 generic Apache request library - Perl modules
>
> kimsufi (cpuinfo : Intel(R) Atom(TM) CPU N2800 @ 1.86GHz)
> ii apache2 2.4.10-10+deb8u4 amd64 Apache HTTP Server
> ii apache2-bin 2.4.10-10+deb8u4 amd64 Apache HTTP Server (modules and other binary files)
> ii apache2-data 2.4.10-10+deb8u4 all Apache HTTP Server (common files)
> ii apache2-mpm-prefork 2.4.10-10+deb8u4 amd64 transitional prefork MPM package for apache2
> ii apache2-utils 2.4.10-10+deb8u4 amd64 Apache HTTP Server (utility programs for web servers)
> ii apache2.2-common 2.4.10-10+deb8u4 amd64 Transitional package for apache2
> ii libapache2-mod-apreq2 2.13-4+b1 amd64 generic Apache request library - Apache module
> ii libapache2-mod-perl2 2.0.9~1624218-2+deb8u1 amd64 Integration of perl with the Apache2 web server
> ii libapache2-reload-perl 0.12-3 all module for reloading Perl modules when changed on disk
> ii libapache2-request-perl 2.13-4+b1 amd64 generic Apache request library - Perl modules
>
> online (cpuinfo : VIA Nano processor U2250 (1.6GHz Capable))
> ii apache2 2.4.10-10+deb8u4 amd64 Apache HTTP Server
> ii apache2-bin 2.4.10-10+deb8u4 amd64 Apache HTTP Server (modules and other binary files)
> ii apache2-data 2.4.10-10+deb8u4 all Apache HTTP Server (common files)
> ii apache2-mpm-prefork 2.4.10-10+deb8u4 amd64 transitional prefork MPM package for apache2
> ii apache2-utils 2.4.10-10+deb8u4 amd64 Apache HTTP Server (utility programs for web servers)
> ii libapache2-mod-apreq2 2.13-4+b1 amd64 generic Apache request library - Apache module
> ii libapache2-mod-perl2 2.0.9~1624218-2+deb8u1 amd64 Integration of perl with the Apache2 web server
> ii libapache2-request-perl 2.13-4+b1 amd64 generic Apache request library - Perl modules
>
> ovh (cpuinfo : Intel(R) Celeron(R) CPU 220 @ 1.20GHz)
> ii apache2 2.4.10-10+deb8u4 amd64 Apache HTTP Server
> ii apache2-bin 2.4.10-10+deb8u4 amd64 Apache HTTP Server (modules and other binary files)
> ii apache2-data 2.4.10-10+deb8u4 all Apache HTTP Server (common files)
> ii apache2-utils 2.4.10-10+deb8u4 amd64 Apache HTTP Server (utility programs for web servers)
> ii libapache2-mod-apreq2 2.13-4+b1 amd64 generic Apache request library - Apache module
> ii libapache2-mod-perl2 2.0.9~1624218-2+deb8u1 amd64 Integration of perl with the Apache2 web server
> ii libapache2-reload-perl 0.12-3 all module for reloading Perl modules when changed on disk
> ii libapache2-request-perl 2.13-4+b1 amd64 generic Apache request library - Perl modules
>

I don't see above any signifiant difference in configuration between the servers, apart
from the fact that the "faulty" server runs a 64-bit version of perl.

But I still wonder if this is the issue :

http://perldoc.perl.org/functions/srand.html
says :

"If srand() is not called explicitly, it is called implicitly without a parameter at the
first use of the rand operator. However, there are a few situations where programs are
likely to want to call srand. One is for generating predictable results, generally for
testing or debugging. There, you use srand($seed), with the same $seed each time. Another
case is that you may want to call srand() after a fork() to avoid child processes sharing
the same seed value as the parent (and consequently each other)."

This would tend to imply that the "real" seed used initially by srand() (such as when it
is called implicitly at the first call to rand()), is something inside the perl
interpreter itself. In a pre-fork configuration, the main Apache process starts first,
and loads a perl interpreter. Then this main Apache process is forked into nn Apache
children processes. Which means that the initial copies of the perl interpreter, in each
child, are also identical.
The main Apache process never runs any request (it just passes them to a child, which does
run the request), so its copy of the perl interpreter remains "pristine", as it was before
the first fork.
If at some point there are not enough children alive, the main process forks again to
create a new child.
So basically all children, present and future, start with the same perl.
And thus with the same whatever-it-is that is the seed of the first srand() (implicitly
called by the first rand()).
So, unless they all do explicitly call srand() once with each a different seed, each child
does generate exactly the same sequence of responses to successive rand() calls.
In other words, on any given machine, the Nth un-keyed request hitting one particular
Apache child, will generate the same key as the Nth un-keyed request hitting any other
child (assuming rand() is not used anywhere else).
(By "un-keyed request", I mean one which triggers the key-generating instruction).
(and N including perhaps also the requests which trigger similar calls to rand to generate
a cookie, if they call rand() the same number of times (32)).

Except that..
The machine where the problem shows up so quickly, is running a 64-bit perl, while the
others are running 32-bit perls.

Now I also found this :
http://rabexc.org/posts/randomizing-should-be-easy-right-oh

I am not sure that I really understand this all the way down, but would this not be a
suspect in a case where the behaviour seems different between one 64-bit machine, and a
bunch of 32-bit ones ?

This being said, it still looks to me as if the current code is flawed on *all* machines,
and *will* repeat keys quite often. It just depends again on the exact sequence of
requests hitting a specific Apache, and the other parameters I mentioned before.
I still believe that the fact that it does not *seem* to happen, is just due to the
inherent randomness of these other factors on the production machines.

On Tue, 17 May 2016 10:16:43 +0200
André Warnier <aw@ice-sa.com> wrote:
>
> I don't see above any signifiant difference in configuration between the servers, apart
> from the fact that the "faulty" server runs a 64-bit version of perl.

Sorry : slightly digressive rant about the fact that every time I compare my configs, I find some subtle differences. Should be getting into config management tools, but that takes time too.

>
> Now I also found this :
> http://rabexc.org/posts/randomizing-should-be-easy-right-oh
>
> I am not sure that I really understand this all the way down, but would this not be a
> suspect in a case where the behaviour seems different between one 64-bit machine, and a
> bunch of 32-bit ones ?

Nope; same results on both types when running the script

>
> This being said, it still looks to me as if the current code is flawed on *all* machines,
> and *will* repeat keys quite often. It just depends again on the exact sequence of
> requests hitting a specific Apache, and the other parameters I mentioned before.
> I still believe that the fact that it does not *seem* to happen, is just due to the
> inherent randomness of these other factors on the production machines.
>

Well, I already posted a test with ab and 12 000 requests, so not sure about the 'quite often' part?

This is on the faulty one :

xxxx@arsene:~$ perl -le '%h=();for (1..10_000_000) {my $session_id = join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32;$h{$session_id}=1};$v=keys %h; print $v'
10000000


--
Bien à vous, Vincent Veyron

https://libremen.com
Logiciels de gestion, libres

Your test are on the command line, not inside apache and thus more or less worthless.

As André clearly showed the issue is almost certainly with the way srand is being called
inside apache

Add a call to srand with something suitably random from time pid and maybe /dev/random
INSIDE apache before generating your ‘random’ code

The issue will almost certainly go away


--
Clive Eisen
GPG: 75056DD0






> On 17 May 2016, at 13:11, Vincent Veyron <vv.lists@wanadoo.fr> wrote:
>
> On Tue, 17 May 2016 10:16:43 +0200
> André Warnier <aw@ice-sa.com> wrote:
>>
>> I don't see above any signifiant difference in configuration between the servers, apart
>> from the fact that the "faulty" server runs a 64-bit version of perl.
>
> Sorry : slightly digressive rant about the fact that every time I compare my configs, I find some subtle differences. Should be getting into config management tools, but that takes time too.
>
>>
>> Now I also found this :
>> http://rabexc.org/posts/randomizing-should-be-easy-right-oh
>>
>> I am not sure that I really understand this all the way down, but would this not be a
>> suspect in a case where the behaviour seems different between one 64-bit machine, and a
>> bunch of 32-bit ones ?
>
> Nope; same results on both types when running the script
>
>>
>> This being said, it still looks to me as if the current code is flawed on *all* machines,
>> and *will* repeat keys quite often. It just depends again on the exact sequence of
>> requests hitting a specific Apache, and the other parameters I mentioned before.
>> I still believe that the fact that it does not *seem* to happen, is just due to the
>> inherent randomness of these other factors on the production machines.
>>
>
> Well, I already posted a test with ab and 12 000 requests, so not sure about the 'quite often' part?
>
> This is on the faulty one :
>
> xxxx@arsene:~$ perl -le '%h=();for (1..10_000_000) {my $session_id = join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32;$h{$session_id}=1};$v=keys %h; print $v'
> 10000000
>
>
> --
> Bien à vous, Vincent Veyron
>
> https://libremen.com
> Logiciels de gestion, libres

On 17.05.2016 14:11, Vincent Veyron wrote:
> On Tue, 17 May 2016 10:16:43 +0200
> André Warnier <aw@ice-sa.com> wrote:
>>
>> I don't see above any signifiant difference in configuration between the servers, apart
>> from the fact that the "faulty" server runs a 64-bit version of perl.
>
> Sorry : slightly digressive rant about the fact that every time I compare my configs, I find some subtle differences. Should be getting into config management tools, but that takes time too.
>
>>
>> Now I also found this :
>> http://rabexc.org/posts/randomizing-should-be-easy-right-oh
>>
>> I am not sure that I really understand this all the way down, but would this not be a
>> suspect in a case where the behaviour seems different between one 64-bit machine, and a
>> bunch of 32-bit ones ?
>
> Nope; same results on both types when running the script
>
>>
>> This being said, it still looks to me as if the current code is flawed on *all* machines,
>> and *will* repeat keys quite often. It just depends again on the exact sequence of
>> requests hitting a specific Apache, and the other parameters I mentioned before.
>> I still believe that the fact that it does not *seem* to happen, is just due to the
>> inherent randomness of these other factors on the production machines.
>>
>
> Well, I already posted a test with ab and 12 000 requests, so not sure about the 'quite often' part?
>
> This is on the faulty one :
>
> xxxx@arsene:~$ perl -le '%h=();for (1..10_000_000) {my $session_id = join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32;$h{$session_id}=1};$v=keys %h; print $v'
> 10000000
>
>

Yes, but this is *one* process. Each independent process, if you consider the keys, will
get a succession of different answers from rand(), and thus generate different keys.
But if n different processes were all starting with the same initial seed, they would all
generate the *same* sequence of rand() responses, and the same sequence of keys.
And that is what I am saying : each of your Apache pre-fork children is a separate
process, but they all always start with the same random seed.
So they will all, ultimately, generate the same sequence of keys (but not necessarily at
the same time).

Let's say that there are initially 5 Apache children, and that Apache never starts more
than 5.
Now you start bombarding the server with hundreds of requests, all of them triggering the
key-generation mechanism.
And let's say that it takes your module 1 s. to respond to a request (just to make things
simpler below).

T0 :
Request #1 comes in.
The main Apache looks for a free child, and finds child #1.
It passes request #1 to child #1.
This child will be busy until T0 + 1s.

T0 + 0.1s :
Request #2 comes in.
The main Apache looks for a free child.
Child #1 is still busy, so it finds child #2.
It passes request #2 to child #2.
This child will be busy until T0 + 2s.

and so on..
(child 5 is now busy until T0 + 5 s.)

Request #6, at T0 + 0.6s) :
Now all 5 children are busy, and Apache has to wait with the request, until
one child becomes free (*).
In this very simplified case, it will be child #1, at T0 + 1s.

At T0 + 1s, child #1 becomes free again. So child #1 now gets request #6, which for him is
only the *second* request that it processes.
So it generates *its* key #2 (which globally is the generated key #6).

In this very simplified example, the first 5 keys generated globally by Apache will be
identical, because each child starts with the same seed, and they are all called neatly in
a regular sequence.
And then the next 5 keys will be identical, because for each child it is now the second
request.
And so on.

But in a real situation :
- not all requests come in so neatly at regular intervals
(so for example child #1 may become free, before child #5 is even called once).
- not all requests take the same time to serve (other things happen on the server)
- not all requests generate a key (so if child #4 is called but does not call rand(), it
does not count; or if it calls rand() only 5 times instead of 32, that screws up the whole
sequence, and it will now start generating keys that are different from all the others)
- the number of children will vary over time. New ones will be created as needed,
some older ones will die and be replaced by a brand-new one. Each time that happens, the
new child will start with key #1 again, because it jus got a brand-new perl. While at the
same time, there may still be an older child alive, for which the next key is already
number 5000 in its own sequence.
- etc..
And this "disorder" will tend to be larger, the more loaded is that server.
So over any given period of time, each child will tend to be at a different stage in his
rand() calls. And the risk of having the same key being returned to 2 clients at about the
same time, is relatively low.
But if the keys are stored somewhere in a persistent way, you are increasing the risk
greatly, because key #13 generated by a new child today, may conflict with the key #13
generated by another child yesterday.

(*) or start an additional child

On 16 May 2016 at 20:03, Bruce Johnson <johnson@pharmacy.arizona.edu> wrote:
>
>> On May 16, 2016, at 10:15 AM, André Warnier (tomcat) <aw@ice-sa.com> wrote:
>>
>>
>> join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32 ;
>>
>> looks at first sight to me like quite inefficient and probably likely to generate the same string regularly, even if it does not look that way.
>> (The only variable there is rand(), and it can only return values between 0 and 62).
>
> The function is meant to map a random element from the 62-element-long array (0..9,"a".."z","A".."Z”) (hence a rand() call to generate a number from 0 and 62), 32 times, and join them into a string.
>
> Although I think that should really be rand(9+26*2) to properly generate array indices for the entire array and no more. With a number between 0 and 62 (63 numbers) and a 62-element array, you’ll be retrieving nulls from the array 1/62 calls, but all that means is that the string is one char shorter for each time '62’ comes up...
>
> So long as rand is properly seeded, you should not get repeats, at least not frequently enough to ever notice, I’d think.
>
> This is textbook Perl, as in I’m pretty sure it’s out of one of Larry Wall’s books; I use it to generate random strings for cookies.
>
> If it’s properly seeded in the original code, it should either work or not work on all five servers. Not working on one out of the five makes me think maybe there’s some sort of weird caching issue.

Or for some reason one of the servers goes through a code path where
it calls srand/rand prefork.

An unfortunate side effect of the rules of srand in perl is that if
you fork without calling rand each child process will have their own
seed. if you rand before fork then all the children will have their
own seed.

Thus you need to ensure you call srand() explicitly post fork in your webserver.

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

On 17 May 2016 at 20:23, demerphq <demerphq@gmail.com> wrote:
> On 16 May 2016 at 20:03, Bruce Johnson <johnson@pharmacy.arizona.edu> wrote:
>>
>>> On May 16, 2016, at 10:15 AM, André Warnier (tomcat) <aw@ice-sa.com> wrote:
>>>
>>>
>>> join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32 ;
>>>
>>> looks at first sight to me like quite inefficient and probably likely to generate the same string regularly, even if it does not look that way.
>>> (The only variable there is rand(), and it can only return values between 0 and 62).
>>
>> The function is meant to map a random element from the 62-element-long array (0..9,"a".."z","A".."Z”) (hence a rand() call to generate a number from 0 and 62), 32 times, and join them into a string.
>>
>> Although I think that should really be rand(9+26*2) to properly generate array indices for the entire array and no more. With a number between 0 and 62 (63 numbers) and a 62-element array, you’ll be retrieving nulls from the array 1/62 calls, but all that means is that the string is one char shorter for each time '62’ comes up...
>>
>> So long as rand is properly seeded, you should not get repeats, at least not frequently enough to ever notice, I’d think.
>>
>> This is textbook Perl, as in I’m pretty sure it’s out of one of Larry Wall’s books; I use it to generate random strings for cookies.
>>
>> If it’s properly seeded in the original code, it should either work or not work on all five servers. Not working on one out of the five makes me think maybe there’s some sort of weird caching issue.
>
> Or for some reason one of the servers goes through a code path where
> it calls srand/rand prefork.
>
> An unfortunate side effect of the rules of srand in perl is that if
> you fork without calling rand each child process will have their own
> seed. if you rand before fork then all the children will have their
> own seed.

Sorry, that should read "if you rand before fork then all the children
will SHARE THE SAME SEED".

I personally consider this a bug in Perl, but I doubt it will get fixed.

Yves


--
perl -Mre=debug -e "/just|another|perl|hacker/"

On 17 May 2016 at 15:10, André Warnier <aw@ice-sa.com> wrote:
> On 17.05.2016 14:11, Vincent Veyron wrote:
>>
>> On Tue, 17 May 2016 10:16:43 +0200
>> André Warnier <aw@ice-sa.com> wrote:
>>>
>>>
>>> I don't see above any signifiant difference in configuration between the
>>> servers, apart
>>> from the fact that the "faulty" server runs a 64-bit version of perl.
>>
>>
>> Sorry : slightly digressive rant about the fact that every time I compare
>> my configs, I find some subtle differences. Should be getting into config
>> management tools, but that takes time too.
>>
>>>
>>> Now I also found this :
>>> http://rabexc.org/posts/randomizing-should-be-easy-right-oh
>>>
>>> I am not sure that I really understand this all the way down, but would
>>> this not be a
>>> suspect in a case where the behaviour seems different between one 64-bit
>>> machine, and a
>>> bunch of 32-bit ones ?
>>
>>
>> Nope; same results on both types when running the script
>>
>>>
>>> This being said, it still looks to me as if the current code is flawed on
>>> *all* machines,
>>> and *will* repeat keys quite often. It just depends again on the exact
>>> sequence of
>>> requests hitting a specific Apache, and the other parameters I mentioned
>>> before.
>>> I still believe that the fact that it does not *seem* to happen, is just
>>> due to the
>>> inherent randomness of these other factors on the production machines.
>>>
>>
>> Well, I already posted a test with ab and 12 000 requests, so not sure
>> about the 'quite often' part?
>>
>> This is on the faulty one :
>>
>> xxxx@arsene:~$ perl -le '%h=();for (1..10_000_000) {my $session_id = join
>> "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)],
>> 1..32;$h{$session_id}=1};$v=keys %h; print $v'
>> 10000000
>>
>>
[SNIP]
> And this "disorder" will tend to be larger, the more loaded is that server.
> So over any given period of time, each child will tend to be at a different
> stage in his rand() calls. And the risk of having the same key being
> returned to 2 clients at about the same time, is relatively low.
> But if the keys are stored somewhere in a persistent way, you are increasing
> the risk greatly, because key #13 generated by a new child today, may
> conflict with the key #13 generated by another child yesterday.
>
> (*) or start an additional child
>

a very simple solution to this problem is to put the time in the string eg:

my $token= pack "LC*", time, map int rand 256, 1..$number_of_bytes_in_token;

then just hex encode or base64 encode the string.

Doing it this way also probably massively faster (done right) than the
approaches i have seen mentioned in this thread. pack is *fast*.

Note the chance that two drand48 states are the same at the same time,
is really really low if sranded properly. Essentially zero.

Yves

Yves

--
perl -Mre=debug -e "/just|another|perl|hacker/"

On 17.05.2016 20:26, demerphq wrote:
> On 17 May 2016 at 20:23, demerphq <demerphq@gmail.com> wrote:
>> On 16 May 2016 at 20:03, Bruce Johnson <johnson@pharmacy.arizona.edu> wrote:
>>>
>>>> On May 16, 2016, at 10:15 AM, André Warnier (tomcat) <aw@ice-sa.com> wrote:
>>>>
>>>>
>>>> join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32 ;
>>>>
>>>> looks at first sight to me like quite inefficient and probably likely to generate the same string regularly, even if it does not look that way.
>>>> (The only variable there is rand(), and it can only return values between 0 and 62).
>>>
>>> The function is meant to map a random element from the 62-element-long array (0..9,"a".."z","A".."Z”) (hence a rand() call to generate a number from 0 and 62), 32 times, and join them into a string.
>>>
>>> Although I think that should really be rand(9+26*2) to properly generate array indices for the entire array and no more. With a number between 0 and 62 (63 numbers) and a 62-element array, you’ll be retrieving nulls from the array 1/62 calls, but all that means is that the string is one char shorter for each time '62’ comes up...
>>>
>>> So long as rand is properly seeded, you should not get repeats, at least not frequently enough to ever notice, I’d think.
>>>
>>> This is textbook Perl, as in I’m pretty sure it’s out of one of Larry Wall’s books; I use it to generate random strings for cookies.
>>>
>>> If it’s properly seeded in the original code, it should either work or not work on all five servers. Not working on one out of the five makes me think maybe there’s some sort of weird caching issue.
>>
>> Or for some reason one of the servers goes through a code path where
>> it calls srand/rand prefork.
>>
>> An unfortunate side effect of the rules of srand in perl is that if
>> you fork without calling rand each child process will have their own
>> seed. if you rand before fork then all the children will have their
>> own seed.
>
> Sorry, that should read "if you rand before fork then all the children
> will SHARE THE SAME SEED".
>
> I personally consider this a bug in Perl, but I doubt it will get fixed.
>

I think that the children will share the same seed in any case. (That sounds kind of
biblical..).
The point is, all children have initially a copy of the same pristine perl interpreter,
and the initial grand-father seed (to the initial srand()) is in there somewhere.
Only if each child starts by an srand($my-own-unique-seed), will they subsequently get a
different sequence of rand() responses.

On 17 May 2016 at 20:37, André Warnier <aw@ice-sa.com> wrote:
> On 17.05.2016 20:26, demerphq wrote:
>>
>> On 17 May 2016 at 20:23, demerphq <demerphq@gmail.com> wrote:
>>>
>>> On 16 May 2016 at 20:03, Bruce Johnson <johnson@pharmacy.arizona.edu>
>>> wrote:
>>>>
>>>>
>>>>> On May 16, 2016, at 10:15 AM, André Warnier (tomcat) <aw@ice-sa.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>> join "", map +(0..9,"a".."z","A".."Z")[rand(10+26*2)], 1..32 ;
>>>>>
>>>>> looks at first sight to me like quite inefficient and probably likely
>>>>> to generate the same string regularly, even if it does not look that way.
>>>>> (The only variable there is rand(), and it can only return values
>>>>> between 0 and 62).
>>>>
>>>>
>>>> The function is meant to map a random element from the 62-element-long
>>>> array (0..9,"a".."z","A".."Z”) (hence a rand() call to generate a number
>>>> from 0 and 62), 32 times, and join them into a string.
>>>>
>>>> Although I think that should really be rand(9+26*2) to properly generate
>>>> array indices for the entire array and no more. With a number between 0 and
>>>> 62 (63 numbers) and a 62-element array, you’ll be retrieving nulls from the
>>>> array 1/62 calls, but all that means is that the string is one char shorter
>>>> for each time '62’ comes up...
>>>>
>>>> So long as rand is properly seeded, you should not get repeats, at least
>>>> not frequently enough to ever notice, I’d think.
>>>>
>>>> This is textbook Perl, as in I’m pretty sure it’s out of one of Larry
>>>> Wall’s books; I use it to generate random strings for cookies.
>>>>
>>>> If it’s properly seeded in the original code, it should either work or
>>>> not work on all five servers. Not working on one out of the five makes me
>>>> think maybe there’s some sort of weird caching issue.
>>>
>>>
>>> Or for some reason one of the servers goes through a code path where
>>> it calls srand/rand prefork.
>>>
>>> An unfortunate side effect of the rules of srand in perl is that if
>>> you fork without calling rand each child process will have their own
>>> seed. if you rand before fork then all the children will have their
>>> own seed.
>>
>>
>> Sorry, that should read "if you rand before fork then all the children
>> will SHARE THE SAME SEED".
>>
>> I personally consider this a bug in Perl, but I doubt it will get fixed.
>>
>
> I think that the children will share the same seed in any case. (That sounds
> kind of biblical..).
> The point is, all children have initially a copy of the same pristine perl
> interpreter, and the initial grand-father seed (to the initial srand()) is
> in there somewhere.
> Only if each child starts by an srand($my-own-unique-seed), will they
> subsequently get a different sequence of rand() responses.

No.

Perl has a flag internally that says if srand has been called prior to
use of rand.

If you fork before you call (s)rand then each child process will have
their own copy of the flag, which will be false, and thus will cause
srand() to be called in the subprocess properly.

This is easily verified without needing to know the source (which I do):

ml905401:~ yorton$ perl -le'fork; print rand'

0.96367916867985

0.404761314931424

ml905401:~ yorton$ perl -le'rand; fork; print rand'

0.716089713943319

0.716089713943319


--
perl -Mre=debug -e "/just|another|perl|hacker/"

On Tue, 17 May 2016 20:41:28 +0200
demerphq <demerphq@gmail.com> wrote:
>
> If you fork before you call (s)rand then each child process will have
> their own copy of the flag, which will be false, and thus will cause
> srand() to be called in the subprocess properly.
>

So now I'm lost : I understand this as meaning that one should _not_ call srand in mod_perl, since each child process is forked by Apache?

In any case, I tried to use srand per André's suggestion and your other post:

my $seed = time ^ $$ ;
srand($seed);
my $token = pack "LC*", time, map int rand 256, 1..32 ;
my $session_id = encode_base64($token);

It does not solve the problem : the same key is regenerated several times, I suppose because time is in seconds, and the machine serves 40 requests/second, so the seed gets reused?

I tried using /dev/random, but this dies after two requests

my $seed;
open(RAND, "/dev/random") or die "no /dev/random?\n";
read (RAND, $seed, 4);
close(RAND);
srand(unpack("L", $seed));

Many posts on the web suggest using modules such as Math::Random. If I have to install additional modules, I'd prefer to use the Debian packaging system, for maintenance reasons. Is libcrypt-random-seed-perl a possibility?

Still don't understand why the problem does not show up in the other machines, no matter what I throw at them.


--
Bien à vous, Vincent Veyron

https://compta.libremen.com
Logiciel de comptabilité générale, libre

On 18.05.2016 01:23, Vincent Veyron wrote:
> On Tue, 17 May 2016 20:41:28 +0200
> demerphq <demerphq@gmail.com> wrote:
>>
>> If you fork before you call (s)rand then each child process will have
>> their own copy of the flag, which will be false, and thus will cause
>> srand() to be called in the subprocess properly.
>>
>
> So now I'm lost : I understand this as meaning that one should _not_ call srand in mod_perl, since each child process is forked by Apache?
>
> In any case, I tried to use srand per André's suggestion and your other post:
>
> my $seed = time ^ $$ ;
> srand($seed);

No, this will re-seed it every time, which is also not the point.
The explicit call to srand(), in any given child, should happen only once.
Maybe you can try a BEGIN block. If I remember correctly, under mod_perl (if you run under
"registry" to keep the compiled cgi-bin in memory), a BEGIN block should be run only once,
when the script is first compiled (by this child).

Now, "demerphg" says that this is all not valid, according to the perl source - which he
consulted and I did not - so it is only my own best guess.
But it is not very expensive to try.

> my $token = pack "LC*", time, map int rand 256, 1..32 ;
> my $session_id = encode_base64($token);
>
> It does not solve the problem : the same key is regenerated several times, I suppose because time is in seconds, and the machine serves 40 requests/second, so the seed gets reused?
>
> I tried using /dev/random, but this dies after two requests
>
> my $seed;
> open(RAND, "/dev/random") or die "no /dev/random?\n";
> read (RAND, $seed, 4);
> close(RAND);
> srand(unpack("L", $seed));
>
> Many posts on the web suggest using modules such as Math::Random. If I have to install additional modules, I'd prefer to use the Debian packaging system, for maintenance reasons. Is libcrypt-random-seed-perl a possibility?
>
> Still don't understand why the problem does not show up in the other machines, no matter what I throw at them.
>
>

On Wed, 18 May 2016 09:12:17 +0200
André Warnier <aw@ice-sa.com> wrote:

> Maybe you can try a BEGIN block. If I remember correctly, under mod_perl (if you run under
> "registry" to keep the compiled cgi-bin in memory), a BEGIN block should be run only once,
> when the script is first compiled (by this child).

Tried that, no dice. And I checked the code on an eeepc, it generates thousands of unique keys no problem, it's just that one server :-(

Well, giving up for now. Thank you for your help.


--
Bien à vous, Vincent Veyron

https://libremen.com
Logiciels de gestion, libres

Is it possible that its not a problem of the rand, but a problem of the ||=
?

Try debugging the value before you set it. It could be a scoping issue
where the old value stays in the variable and then ||= doesn't overwrite it.


*Lathan Bidwell*


On Wed, May 18, 2016 at 9:08 AM, Vincent Veyron <vv.lists@wanadoo.fr> wrote:

> On Wed, 18 May 2016 09:12:17 +0200
> André Warnier <aw@ice-sa.com> wrote:
>
> > Maybe you can try a BEGIN block. If I remember correctly, under mod_perl
> (if you run under
> > "registry" to keep the compiled cgi-bin in memory), a BEGIN block should
> be run only once,
> > when the script is first compiled (by this child).
>
> Tried that, no dice. And I checked the code on an eeepc, it generates
> thousands of unique keys no problem, it's just that one server :-(
>
> Well, giving up for now. Thank you for your help.
>
>
> --
> Bien à vous, Vincent Veyron
>
> https://libremen.com
> Logiciels de gestion, libres
>

On Wed, 18 May 2016 09:35:45 -0400
Lathan Bidwell <lathan@andrews.edu> wrote:

> Is it possible that its not a problem of the rand, but a problem of the ||=
> ?
>
> Try debugging the value before you set it. It could be a scoping issue
> where the old value stays in the variable and then ||= doesn't overwrite it.
>

Hi Lathan,

That's not it I'm afraid : below is this output in error_log before and after ||=, you can see that the value is undefined first, then set to a previous value after a few iterations

Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : RzFMMRznJ9fRaHgVU27AQdrN4WgMr7cH at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : 45GJYXZvPkMw55Zfkxr2H3EnN7S19Wzw at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : xW0r3dUFFDfeTxzQnftxhMm2PyoX12Bc at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : 8Os5Dyfmpch7DWOdZ0yvn1JN7aVrT9q3 at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : VpCyv6GmeQwPqnD84nYBVGJeS93g64of at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : 7YoiW2abgrnno8nivS8VYoEXKKh9K2KI at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : nNDYxt5EocI8gjXwq7Lc5FczPdfcNKPn at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : RzFMMRznJ9fRaHgVU27AQdrN4WgMr7cH at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : 45GJYXZvPkMw55Zfkxr2H3EnN7S19Wzw at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : xW0r3dUFFDfeTxzQnftxhMm2PyoX12Bc at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : 8Os5Dyfmpch7DWOdZ0yvn1JN7aVrT9q3 at /home/lib/Compta/Base/Handler/entry.pm line 150.
Use of uninitialized value in concatenation (.) or string at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : at /home/lib/Compta/Base/Handler/entry.pm line 146.
Token : VpCyv6GmeQwPqnD84nYBVGJeS93g64of at /home/lib/Compta/Base/Handler/entry.pm line 150.




--
Bien à vous, Vincent Veyron

https://compta.libremen.com
Logiciel de comptabilité générale, libre

On Mon, 16 May 2016 16:38:18 +0200
Vincent Veyron <vv.lists@wanadoo.fr> wrote:

> Out of five different servers, the code works fine on four machines, and a different token is generated every time the page is loaded or re-loaded. On one server however, a previous token is being re-used

Hi,

The faulty server was installed with 32-bit binaries on a 64-bit processor.

I did a clean re-install using 64-bit binaries (debian stable AMD64), and the problem went away.

ab -n 20000 -c 5 "<my test url>"

now generates 20000 unique keys.

Speed also increased from 40 to 70 pages/second

--
Bien à vous, Vincent Veyron

https://compta.libremen.com
Logiciel libre de comptabilité générale

On 26.05.2016 16:01, Vincent Veyron wrote:
> On Mon, 16 May 2016 16:38:18 +0200
> Vincent Veyron <vv.lists@wanadoo.fr> wrote:
>
>> Out of five different servers, the code works fine on four machines, and a different token is generated every time the page is loaded or re-loaded. On one server however, a previous token is being re-used
>
> Hi,
>
> The faulty server was installed with 32-bit binaries on a 64-bit processor.
>
> I did a clean re-install using 64-bit binaries (debian stable AMD64), and the problem went away.
>
> ab -n 20000 -c 5 "<my test url>"
>
> now generates 20000 unique keys.
>
> Speed also increased from 40 to 70 pages/second
>

Thanks for reporting this.

The above strongly hints at some flaw in the srand() of perl, when called by a 32-bit
perl, on a 64-bit OS/machine.
In the course of this discussion, I remember citing some article found on the web, which
was talking about something along those lines (such as the fact that only the lower or
upper 32 bits were being used or so).
Maybe it is worth passing this info along to the perl (language) developers, at www.perl.org ?

On Fri, 27 May 2016 09:29:36 +0200
André Warnier <aw@ice-sa.com> wrote:

>
> The above strongly hints at some flaw in the srand() of perl, when called by a 32-bit
> perl, on a 64-bit OS/machine.
> Maybe it is worth passing this info along to the perl (language) developers, at www.perl.org ?
>

Sorry for the delay getting back to you : since I wiped and re-installed the machine, I created a virtual machine with a 32bit image, so that took a little time. And, guess what? I can't reproduce the problem in the VM.

Also, it only appeared in mod_perl, not perl.

The bug may have to do with a cpan upgrade I did on the machine, that generated quite a bit of errors, and messed my system somehow.

--
Bien à vous, Vincent Veyron

https://compta.libremen.com
Logiciel libre de comptabilité générale

On 31.05.2016 18:40, Vincent Veyron wrote:
> On Fri, 27 May 2016 09:29:36 +0200
> André Warnier <aw@ice-sa.com> wrote:
>
>>
>> The above strongly hints at some flaw in the srand() of perl, when called by a 32-bit
>> perl, on a 64-bit OS/machine.
>> Maybe it is worth passing this info along to the perl (language) developers, at www.perl.org ?
>>
>
> Sorry for the delay getting back to you : since I wiped and re-installed the machine, I created a virtual machine with a 32bit image, so that took a little time. And, guess what? I can't reproduce the problem in the VM.
>
> Also, it only appeared in mod_perl, not perl.
>
> The bug may have to do with a cpan upgrade I did on the machine, that generated quite a bit of errors, and messed my system somehow.
>

Thanks for the feedback.
But now you have a 32-bit machine + OS, and a 32-bit perl, right ?
The problem before happened on a 64-bit machine, with a 64-bit OS, but a 32-bit perl, no ?

On 27 May 2016 at 09:29, André Warnier <aw@ice-sa.com> wrote:
> On 26.05.2016 16:01, Vincent Veyron wrote:
>>
>> On Mon, 16 May 2016 16:38:18 +0200
>> Vincent Veyron <vv.lists@wanadoo.fr> wrote:
>>
>>> Out of five different servers, the code works fine on four machines, and
>>> a different token is generated every time the page is loaded or re-loaded.
>>> On one server however, a previous token is being re-used
>>
>>
>> Hi,
>>
>> The faulty server was installed with 32-bit binaries on a 64-bit
>> processor.
>>
>> I did a clean re-install using 64-bit binaries (debian stable AMD64), and
>> the problem went away.
>>
>> ab -n 20000 -c 5 "<my test url>"
>>
>> now generates 20000 unique keys.
>>
>> Speed also increased from 40 to 70 pages/second
>>
>
> Thanks for reporting this.
>
> The above strongly hints at some flaw in the srand() of perl, when called by
> a 32-bit perl, on a 64-bit OS/machine.
> In the course of this discussion, I remember citing some article found on
> the web, which was talking about something along those lines (such as the
> fact that only the lower or upper 32 bits were being used or so).
> Maybe it is worth passing this info along to the perl (language) developers,
> at www.perl.org ?

If it can be replicated on a command line then yes.

But you will want to send the bug report to perlbug@perl.org

FWIW, if your perl is sufficiently late we use the BSD implementation
of DRAND48, so we definitely would like to know if you find an issue.

Yves (who is a perl5porter)


--
perl -Mre=debug -e "/just|another|perl|hacker/"