Mailing List Archive

problems with %udat persistance
Dear all,

I use embperl's sessions management with mySQL under Suse Linux. I am
experiencing non deterministic behaviour from embperl's session management
as %udat does not always store or retrieve data appropriately. I use IE5 and
I have it configured to check for newer versions of stored pages on every
visit to the page. I retrieve some page successfully, then I try to fetch it
again and it fails. The rate of failure varies. I really do not know whether
it is indeed embperl that is causing the failure or, if it might be the
Apache web server or mySQL that is causing it.

I operate the web server (Apache) on a LAN (ethernet).

It is worth mentioning that the graphics on my pages do not them either
always load, although it rarely happens. That could indicate that embperl's
session management is working and that the browser or web server fails to
send data successfully between themselves.

Furthermore, I always access %udat 'directly' on every page to enable the
session management to detect change.

I have the feeling that there is some time out occurring somewhere in the
process. Whether it is emberl, mySQL, Apache, or even the browser, that is
causing the problem I do not know.

Does anyone have a clue?

Please see my configuration files below.

regards
ragnar

------------- startup.pl --------------
#!/usr/bin/perl

BEGIN {
use Apache ();
use lib Apache->server_root_relative('lib/perl');
unshift (@INC, "/caseTracking/packages");
}

use Apache::Registry();
use Apache::Status;
use Apache::DBI;
use Apache::AuthDBI;
use DBI;
use Apache::Constants();
use CGI qw(-compile :all);
use CGI::Carp();
use LWP();
use Apache::Session();
use Apache::TicketAccess();
use Apache::SubTicketAccess();
use Storable;

$ENV{TRACE_LEVEL}=3;
$ENV{TRACE_FILE_NAME}="/caseTracking/logs/dbTraceLog";
$ENV{EMBPERL_SESSION_CLASSES} = "DBIStore SysVSemaphoreLocker";
$ENV{EMBPERL_SESSION_ARGS} = "DataSource=dbi:mysql:session UserName=clm";



$Apache::DBI::DEBUG=2;
$Apache::Session::SysVSemaphoreLocker::nsems = 32;

# -- Access directives for 'protected' area --
Apache::SubTicketAccess->configure('protected', {
TicketDB => 'dbi:Pg(AutoCommit=>0):dbname=transactiondb',
TicketDBUser => 'clm',
TicketUserTable => 'users:username:passwd',
TicketTable => 'tickets:ticket_hash:expires',
TicketSecretTable => 'ticketsecrets:sec_data:sec_version',
TicketDomain => '.conquestnetwork.com',
TicketPath => '/protected',
TicketSecure => 0,
TicketLoginForm => '/loginform',
TicketLoginScript => '/login',
TicketExpires => 30,
TicketLogoutURI => '/logout.html',
TicketDefaultPage => '/home.epl',
});

1;




------------- perl.conf ---------------

PerlRequire conf/startup.pl
PerlFreshRestart On

# SetEnv DBI_DSN "Pg"

<location /hello/world>
SetHandler perl-script
PerlHandler Apache::Hello
</location>

alias /perl/ /caseTracking/www/perl/

<location /perl>
SetHandler perl-script
PerlHandler Apache::Registry
PerlSendHeader On
Options +ExecCGI
</location>

SetEnv EMBPERL_DEBUG 270352
SetEnv EMBPERL_VIRTLOG /embperl/log
SetEnv EMBPERL_COOKIE_NAME CONQUEST_UID
#SetEnv EMBPERL_DEBUG 3170492
# SetEnv EMBPERL_COOKIE_EXPIRES 01/01/2100
# SetEnv EMBPERL_OPTIONS 262144

<location /embperl/log>
SetHandler perl-script
PerlHandler HTML::Embperl
Options execcgi
</location>

<files *.epl>
SetHandler perl-script
PerlHandler HTML::Embperl
Options +ExecCGI
</files>

AddType text/html .epl



# Access directives
<location /protected>
PerlAccessHandler Apache::SubTicketAccess->authenticate
PerlSetVar TicketRealm protected
</location>

<location /loginform>
SetHandler perl-script
PerlHandler Apache::SubTicketAccess->login_form
PerlSetVar TicketRealm protected
</location>

<location /login>
SetHandler perl-script
PerlHandler Apache::SubTicketAccess->login
PerlSetVar TicketRealm protected
</location>

<location /protected/logout>
SetHandler perl-script
PerlHandler Apache::SubTicketAccess->logout
PerlSetVar TicketRealm protected
</location>

# Enable Status support
<Location /server-status>
setHandler server-status
order deny,allow
deny from all
allow from localhost, 192.168.0.44
allow from 192.168.0.150
</Location>

ExtendedStatus on
RE: problems with %udat persistance [ In reply to ]
>
> I use embperl's sessions management with mySQL under Suse Linux. I am
> experiencing non deterministic behaviour from embperl's session management
> as %udat does not always store or retrieve data appropriately. I
> use IE5

As Vladimir told us, early version of IE 5 have problems with cookie. So it
could be that's an IE 5 problem. Could you try to upgrade your IE 5 or test
it with Netscape?

You can also enable the dbgHeadersIn flag in EMBPERL_DEBUG and take a look
at the embperl logfile. You should see the cookie with the session id for
every request in the http headers your browser send. Are the session id
correctly send by the browser?

Gerald

-------------------------------------------------------------
Gerald Richter ecos electronic communication services gmbh
Internetconnect * Webserver/-design/-datenbanken * Consulting

Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz
E-Mail: richter@ecos.de Voice: +49 6133 925151
WWW: http://www.ecos.de Fax: +49 6133 925152
-------------------------------------------------------------
RE: problems with %udat persistance [ In reply to ]
Dear Gerald,

Thanks for your quick response.

Cookies are being received and sent by the browser correctly. I am using a
cookie based login facility that works without any problems. Furthermore,
when reading the embperl debug file I see that the embperl session cookie id
is received by embperl from the browser, always.

I have detected that when embperl receives the session id that changes do
not get updated in the session database. Could the problem have anything to
do with the $Apache::Session::SysVSemaphoreLocker::nsems variable or the
mechanism related with it?

When I fetch a page and %udat is not correctly updated, or even not
containing any older data, I have to wait for some time before successfully
re-fetching the page with %udat responding correctly as opposed to
re-fetching the page instantly on failure. However, this is not 'always' the
case. Could it be that the SysVSemaphoreLocker mechanism is blocking access
to the row containing the data for the session id in question?

I suspect that the problem does not lie with embperl nor the browser. I
think the problem might be related with the persistence mechanism that
embperl uses. I always successfully retrieve database resources through DBI
(PostgreSQL) within my pages so I have a high degree of confidence with DBI.
Another point worth mentioning is that the session database is mySQL while
all other database resources through out the system are located in
PostgreSQL. For instance, the cookie based login mechanism I am using stores
it's cookie data in PostgreSQL and it never fails. I find it hard to believe
that mySQL is causing the problem as it seems to be used by many as the
session database. That leaves us with the SysVSemaphoreLocker mechanism as
the prime suspect, or?

Am I completely on the wrong track?

I am puzzled.

regards,
ragnar



-----Original Message-----
From: Gerald Richter [mailto:richter@ecos.de]
Sent: 10 May 2000 18:57
To: Ragnar Hakonarson; embperl@perl.apache.org
Cc: steini@conquestnetwork.com
Subject: RE: problems with %udat persistance


>
> I use embperl's sessions management with mySQL under Suse Linux. I am
> experiencing non deterministic behaviour from embperl's session management
> as %udat does not always store or retrieve data appropriately. I
> use IE5

As Vladimir told us, early version of IE 5 have problems with cookie. So it
could be that's an IE 5 problem. Could you try to upgrade your IE 5 or test
it with Netscape?

You can also enable the dbgHeadersIn flag in EMBPERL_DEBUG and take a look
at the embperl logfile. You should see the cookie with the session id for
every request in the http headers your browser send. Are the session id
correctly send by the browser?

Gerald

-------------------------------------------------------------
Gerald Richter ecos electronic communication services gmbh
Internetconnect * Webserver/-design/-datenbanken * Consulting

Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz
E-Mail: richter@ecos.de Voice: +49 6133 925151
WWW: http://www.ecos.de Fax: +49 6133 925152
-------------------------------------------------------------
RE: problems with %udat persistance [ In reply to ]
> That leaves us with the SysVSemaphoreLocker mechanism as
> the prime suspect, or?
>

I also use mysql as session database on a Linux system and it works without
problems..., but from what you describe it really looks like a Locker
problem, so I would first try to use NullLocker instead of
SysVSemaphoreLocker. Has your system many page requests or is it just a test
system?

> Am I completely on the wrong track?
>

No, surely not

Gerald
-------------------------------------------------------------
Gerald Richter ecos electronic communication services gmbh
Internetconnect * Webserver/-design/-datenbanken * Consulting

Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz
E-Mail: richter@ecos.de Voice: +49 6133 925151
WWW: http://www.ecos.de Fax: +49 6133 925152
-------------------------------------------------------------
RE: problems with %udat persistance [ In reply to ]
Hi again,

I just changed from SysVSemaphoreLocker to NULLLocker. I'll inform you how
this changed affected %udat when it has been in use for some time.

The system is currently under development and will be used in business.
There will be a considerable load on the system once it is launched.

I believe your concern behind your question relating to the number of page
requests that the system is under is that with no locking mechanism and a
high degree of cuncurrent requests might not be the best of combinations.

Thanks once again for your support.

regards
ragnar

-----Original Message-----
From: Gerald Richter [mailto:richter@ecos.de]
Sent: 10 May 2000 20:53
To: Ragnar Hakonarson; embperl@perl.apache.org
Subject: RE: problems with %udat persistance


> That leaves us with the SysVSemaphoreLocker mechanism as
> the prime suspect, or?
>

I also use mysql as session database on a Linux system and it works without
problems..., but from what you describe it really looks like a Locker
problem, so I would first try to use NullLocker instead of
SysVSemaphoreLocker. Has your system many page requests or is it just a test
system?

> Am I completely on the wrong track?
>

No, surely not

Gerald
-------------------------------------------------------------
Gerald Richter ecos electronic communication services gmbh
Internetconnect * Webserver/-design/-datenbanken * Consulting

Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz
E-Mail: richter@ecos.de Voice: +49 6133 925151
WWW: http://www.ecos.de Fax: +49 6133 925152
-------------------------------------------------------------
RE: problems with %udat persistance [ In reply to ]
Hi, I would just like to tune in here. I observe same occasional
problems that you describe. I'm using also the samy config, Postgres
6.5.3, Apache::Session 1.02 with Cookies, SysVSemaphoreLocker and
Embperl 1.2.1 on Linux 2.2.10. What helps me is to restart the apache
process.

Maybe we should forward this discussion to Jeffrey Baker
<jeffrey@kathyandjeffrey.net> who is the author of Apache::Session
and/or the modperl list.


Dirk
RE: problems with %udat persistance [ In reply to ]
>
>
> Hi, I would just like to tune in here. I observe same occasional
> problems that you describe. I'm using also the samy config, Postgres
> 6.5.3, Apache::Session 1.02 with Cookies, SysVSemaphoreLocker and
> Embperl 1.2.1 on Linux 2.2.10. What helps me is to restart the apache
> process.
>

This could by a locker problem, because locks will be released when you
restart apache. Could you try the NullLocker? If the problem persist we have
to search in another direction, if the problem went away by using the
NullLocker, then it's more likey an Embperl problem (that Embperl doesn't
relaese the session under some circumstances)

> Maybe we should forward this discussion to Jeffrey Baker
> <jeffrey@kathyandjeffrey.net> who is the author of Apache::Session
> and/or the modperl list.
>

First I would like to get the result from the above test.

Gerald


-------------------------------------------------------------
Gerald Richter ecos electronic communication services gmbh
Internetconnect * Webserver/-design/-datenbanken * Consulting

Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz
E-Mail: richter@ecos.de Voice: +49 6133 925151
WWW: http://www.ecos.de Fax: +49 6133 925152
-------------------------------------------------------------
RE: problems with %udat persistance [ In reply to ]
Gerald Richter writes:
> >
> >
> > Hi, I would just like to tune in here. I observe same occasional
> > problems that you describe. I'm using also the samy config, Postgres
> > 6.5.3, Apache::Session 1.02 with Cookies, SysVSemaphoreLocker and
> > Embperl 1.2.1 on Linux 2.2.10. What helps me is to restart the apache
> > process.
> >
>
> This could by a locker problem, because locks will be released when you
> restart apache. Could you try the NullLocker? If the problem persist we have
> to search in another direction, if the problem went away by using the
> NullLocker, then it's more likey an Embperl problem (that Embperl doesn't
> relaese the session under some circumstances)

Ok, I have set it to NullLocker now and report what happens. What do I
loose if I'm using NullLocker instead of SysVSemaphoreLocker? Btw. I'm
not using %udat but Apache::Session (1.02) directly.

Dirk
RE: problems with %udat persistance [ In reply to ]
> Ok, I have set it to NullLocker now and report what happens. What do I
> loose if I'm using NullLocker instead of SysVSemaphoreLocker?

You will run into problems if more then one Apache child tries to update the
same session at the same time. That normaly should not happen within Embperl
%udat, because you have one session per user/browser and the same browser
should not request two embperl pages at the same time. (in contrast to
%mdat)

> Btw. I'm
> not using %udat but Apache::Session (1.02) directly.
>

Then you should check if there could be any place where you don't release
the Session object.

Gerald
RE: problems with %udat persistance [ In reply to ]
Hello,

I have now been running the system with the NullLocker for the last couple
of days and it has greatly improved the %udat persistence problem. However,
the problem still occurs.

The characteristics of the problem has however somewhat changed after I
started using the NullLocker. The problem does not occur nearly as often as
before but when it occurs I often have to restart the apache server to get
any data from %udat while when using the SysVSemaphoreLocker I often just
had to wait some time and then refresh the page.

Anyhow, the improvement was considerable when moving to the NullLocker but,
as Gerald said, %mdat will most likely be affected,at least in my case, as
there it is very likely that the same page will be fetched simultaneously.

Any suggestion on where to go from here?

Dirk, I look forward to hear how the NullLocker affected your system.

Ragnar

-----Original Message-----
From: Dirk Lutzebaeck [mailto:lutzeb@aeccom.com]
Sent: 12 May 2000 08:46
To: Gerald Richter
Cc: Ragnar Hakonarson; embperl@perl.apache.org
Subject: RE: problems with %udat persistance


Gerald Richter writes:
> >
> >
> > Hi, I would just like to tune in here. I observe same occasional
> > problems that you describe. I'm using also the samy config, Postgres
> > 6.5.3, Apache::Session 1.02 with Cookies, SysVSemaphoreLocker and
> > Embperl 1.2.1 on Linux 2.2.10. What helps me is to restart the apache
> > process.
> >
>
> This could by a locker problem, because locks will be released when you
> restart apache. Could you try the NullLocker? If the problem persist we
have
> to search in another direction, if the problem went away by using the
> NullLocker, then it's more likey an Embperl problem (that Embperl doesn't
> relaese the session under some circumstances)

Ok, I have set it to NullLocker now and report what happens. What do I
loose if I'm using NullLocker instead of SysVSemaphoreLocker? Btw. I'm
not using %udat but Apache::Session (1.02) directly.

Dirk

---------------------------------------------------------------------
To unsubscribe, e-mail: embperl-unsubscribe@perl.apache.org
For additional commands, e-mail: embperl-help@perl.apache.org
RE: problems with %udat persistance [ In reply to ]
Hello,

>
> Any suggestion on where to go from here?
>

I would be very important if you could isolate the situation where the
problem occurs. What are you doing before? Is it possible that it starts
after an error has occured? Does it occur when you refresh a page very fast?
(You may try to use the ab from the Apache distribution to put some high
load on your pages). Does it occur only after the request of specific pages?

Gerald


-------------------------------------------------------------
Gerald Richter ecos electronic communication services gmbh
Internetconnect * Webserver/-design/-datenbanken * Consulting

Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz
E-Mail: richter@ecos.de Voice: +49 6133 925151
WWW: http://www.ecos.de Fax: +49 6133 925152
-------------------------------------------------------------
RE: problems with %udat persistance [ In reply to ]
Gerald Richter writes:

> Then you should check if there could be any place where you don't release
> the Session object.

I thought that Apache::Session releases that for me. Typically I have
a global variable which stores the session and the globals are
destroyed by Embperl I think which should force the sync of the
Session object.

The only problem I had to solve where the session was never released
(only with IE5!) was in the situation of a redirect. There I needed to
force the sync:


[# my page.html (just a sketch) #]

[.-
$session = CS::Session->getSession;

$session->{'x'} = 1;

$session->make_synced;

$r = Apache->request;

$r->header_out("Location" => "/somewhere.html");

$r->status(REDIRECT);

Apache::exit();
-]


Could it be that the session is still not released in this case and
this would influence all subsequent accesses for this session?

Dirk
RE: problems with %udat persistance [ In reply to ]
> I thought that Apache::Session releases that for me. Typically I have
> a global variable which stores the session and the globals are
> destroyed by Embperl I think which should force the sync of the
> Session object.
>

Yes

> The only problem I had to solve where the session was never released
> (only with IE5!) was in the situation of a redirect. There I needed to
> force the sync:
>
>
> [# my page.html (just a sketch) #]
>
> [.-
> $session = CS::Session->getSession;
>
> $session->{'x'} = 1;
>
> $session->make_synced;
>
> $r = Apache->request;
>
> $r->header_out("Location" => "/somewhere.html");
>
> $r->status(REDIRECT);
>
> Apache::exit();
> -]
>
>
> Could it be that the session is still not released in this case and
> this would influence all subsequent accesses for this session?
>

If the session object is DESTROYED than the lock is released, but if the not
the lock will persist. The make_synced only marks the session object as sync
but will not do anythingelse.

You need to make sure that no references to the session object exists after
the end of the request. (globals inside your Embperl page will be cleaned by
Embperl, but your are responsible for all other globals)

Gerald
RE: problems with %udat persistance [ In reply to ]
Gerald Richter writes:
> > Ok, I have set it to NullLocker now and report what happens. What do I
> > loose if I'm using NullLocker instead of SysVSemaphoreLocker?
>
> You will run into problems if more then one Apache child tries to update the
> same session at the same time. That normaly should not happen within Embperl
> %udat, because you have one session per user/browser and the same browser
> should not request two embperl pages at the same time. (in contrast to
> %mdat)
>
> > Btw. I'm
> > not using %udat but Apache::Session (1.02) directly.
> >
>
> Then you should check if there could be any place where you don't release
> the Session object.

I have run NullLocker now for a while and it looks like the problem
with the session data still persists. It happens not as much as with
SysVSemaphoreLocker but is still observable. Maybe I have found one
more point of interest here:

Gerald suggested to check that the session object is really destroyed
which means that it should be saved. I checked this directly in
Apache::Session with a warning: I could observe no occasion where it
wouldn't be destroyed. Also when the case happened that session data
seems not to be changed I checked the session data field of the
session table in the data base. The interesting thing seems to be that
the data has changed in the database but somehow would not be
retrieved from the next page which still gets that old values.

Next thing is to switch from Apache::Session 1.02 to 1.03

Dirk
RE: problems with %udat persistance [ In reply to ]
Dirk,

> I have run NullLocker now for a while and it looks like the problem
> with the session data still persists. It happens not as much as with
> SysVSemaphoreLocker but is still observable. Maybe I have found one
> more point of interest here:
>
> Gerald suggested to check that the session object is really destroyed
> which means that it should be saved. I checked this directly in
> Apache::Session with a warning: I could observe no occasion where it
> wouldn't be destroyed. Also when the case happened that session data
> seems not to be changed I checked the session data field of the
> session table in the data base. The interesting thing seems to be that
> the data has changed in the database but somehow would not be
> retrieved from the next page which still gets that old values.
>

Have you tried to explicitly turn off caching like Ceasr suggested. (I
append the mail again at the end)

> Next thing is to switch from Apache::Session 1.02 to 1.03
>

There is no change from 1.02 to 1.03 (only unused code has changed), so this
will not change anything.

Gerald


-----Original Message-----
From: Cesar Bonadio [mailto:bonadio@picture.com.br]
Sent: Thursday, May 18, 2000 5:51 PM
To: richter@ecos.de
Subject: udat persistence issues possible solution


Dear Gerald

I saw all the persistence messages in the list and we are in a project
and we had
the same problems we tried several optoins but the only one that help
was
set NullLocker in the DBIStore, but even with that the problem occured
less frequently

we found that if you set the page to do not cache that solved our
problems
[-
$http_headers_out{'pragma'} = "no-cache";
$http_headers_out{'expires'} = "0";

we have been with this configuration and NullLocker for 2 days and no
problems so far.
this is tests system but soon will go to production.

Hope that helps

Best Regards

Cesar Bonadio



--
----------------------------------------------------------
Cesar A. Bonadio Picture Internet
<bonadio@picture.com.br> Rua Sud Mennucci 331
Tel/Fax:011 5084-0028 Sao Paulo -SP-Brasil-04017-080