Mailing List Archive

Michael Mauws wrote:
> For better or for worse, I can live with the situation I've just
> described. However, the problem I'm having is that it's not just the
> "update request" that hangs. Rather, it's the whole Zope that hangs
> until this one request has been satisfied.

Are you _sure_ your other requests aren't relying on something locked by
this process?

I'd suggest a couple of things:

- there's a product out there called DeadlockDebugger for Zope which can
help you find out where the other requests are getting stuck

- find some of those new fangled Oracle tools (you know, the ones you
pay inordinate amounts of money for an that are riddled with critical
security vulnerabilities ;-) and see if you can figure out what's going
on from the oracle end of things

good luck!

Chris

--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

Michael Mauws wrote at 2006-5-1 19:02 -0600:
> ...
>However, the problem I'm having is that it's not just the
>"update request" that hangs. Rather, it's the whole Zope that hangs
>until this one request has been satisfied.

Even Zope requests that have nothing to do with Oracle?

This would indicate that "DCOracle2" forgets to release
the GIL at some place.

We are using "DCOracle2" as well and have not yet met such
a problem.

--
Dieter
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

Please always stay on the list!

Michael Mauws wrote at 2006-5-2 14:39 -0600:
> ...
>In light of the above, I don't think it's a GIL problem (without
>having a clue what a GIL actually is)

This is the "Global Interpreter Lock". It prevents more than
a single thread to run Python code at the same time.

If C code does not access Python data, it may release the GIL
(and than run in parallel with Python code). If it forgets
to release the GIL, then no Python code can run.

>or a DCOracle2 problem. Instead,
>I think it may be related to Zope's transaction algorithm

This is quite unlikely.

While Zope cannot concurrently commit more than a single
transaction (committing is synchronized on a per storage
level) it can run multiple
requests while some requests wait on database locks.

However, should Oracle delay a commit (due to lock contention),
than Zope may not be able to perform further commits.

Note, that this would only affect requests that either modify
the ZODB or an external database.

>I'm not sure how threading works in ZOpe, but it
>appears that my Zope is dealing with page requests sequentially rather
>than in parallel.

That is unusual...

>As I indicated before, some have suggested that this
>is solved with ZEO.

They err.

>As well, I found one post that seemed to be
>suggesting the problem had to do with running Zope behind Apache.

It, too, errs.

>Do
>either of these explanations make any sense?

No.

--
Dieter
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

Thanks yet again, Dieter, particularly for putting to rest a few of
the "solutions" I found while poking around. As you might expect, I've
been continuing to explore this problem and, at the moment, I'm
zeroing in on rollback segments. If this is indeed the problem, it
would fit well with what you're saying....


...michael


On 5/3/06, Dieter Maurer <dieter@handshake.de> wrote:
> Please always stay on the list!
>
> Michael Mauws wrote at 2006-5-2 14:39 -0600:
> > ...
> >In light of the above, I don't think it's a GIL problem (without
> >having a clue what a GIL actually is)
>
> This is the "Global Interpreter Lock". It prevents more than
> a single thread to run Python code at the same time.
>
> If C code does not access Python data, it may release the GIL
> (and than run in parallel with Python code). If it forgets
> to release the GIL, then no Python code can run.
>
> >or a DCOracle2 problem. Instead,
> >I think it may be related to Zope's transaction algorithm
>
> This is quite unlikely.
>
> While Zope cannot concurrently commit more than a single
> transaction (committing is synchronized on a per storage
> level) it can run multiple
> requests while some requests wait on database locks.
>
> However, should Oracle delay a commit (due to lock contention),
> than Zope may not be able to perform further commits.
>
> Note, that this would only affect requests that either modify
> the ZODB or an external database.
>
> >I'm not sure how threading works in ZOpe, but it
> >appears that my Zope is dealing with page requests sequentially rather
> >than in parallel.
>
> That is unusual...
>
> >As I indicated before, some have suggested that this
> >is solved with ZEO.
>
> They err.
>
> >As well, I found one post that seemed to be
> >suggesting the problem had to do with running Zope behind Apache.
>
> It, too, errs.
>
> >Do
> >either of these explanations make any sense?
>
> No.
>
> --
> Dieter
>
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

Michael Mauws wrote:

> Thanks yet again, Dieter, particularly for putting to rest a few of
> the "solutions" I found while poking around. As you might expect, I've
> been continuing to explore this problem and, at the moment, I'm
> zeroing in on rollback segments. If this is indeed the problem, it
> would fit well with what you're saying....

Hi!

Rarely, but I have a similiar problems with Zope and Oracle - I mean
Zope stops responding. You can easily check what is the cause with
DeadlockDebugger, and I think you should definitely start your
debugging from this point.
In my case all Zope threads are hung in cursor.execute() line in DCOracle2
query (AFAIR) function - they're waiting for database to finish some
queries.
Usually the problem is with long lasting queries and nervous users that are
clicking refresh buttons again and again.
Unfortunatelly sometimes the problem is different - queries are 'locked' in
Oracle. As db administrator says one query has locked a table and didn't
removed a lock, and all other queries are waiting for this. If db admin
executes commit/rollback to finish transactions everything starts working.
Restarting Zope is solution too. So far I don't know exactly why these
locks appear. They are rare - sometimes once a week or two and
thus difficult to track down.

I think it is a problem with Oracle procedures, but db admins say it
may be that Zope is not commiting some transactions (is this possible?).
I've added some code to DCOracle2 to log queries, and turned on debug
mode to be able to see what hapened with transactions in Zope, but now
I'm waiting for the problem to appear.

Maybe your problem is something similiar.

--
Maciej Wisniowski
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

I'm pretty confident that the db component (the C layer) doesn't have
any GIL problems. However, the DA component (the Zope layer) *may*
have all sorts of unusual failure modes.

Debugging transaction concurrency problems is always tricky, but keep
in mind there are Python layer abstractions that get and release
locks as well.

I'm speculating based on the history of your discussion thread that
you're running out of Zope db connections. The way it works in Zope
is somewhat unusual; there are usually NOT enough object database
connections if you run more than four threads (unless someone fixed
this way back when, the default was 7). Threads in Zope wait in line
for a zodb connection to become available. Under normal
circumstances they don't have to wait, but in yours I think they are
waiting. Are you trying to run with more than four threads? If so
you might want to tweak the pool_size parameter up in the DB class of
lib/python/ZODB/DB.py. There are negative consequences of doing this
(ie each DB connection cache gobbles memory) but on the upside you
might be able to avoid stalling out Zope.

This won't provide you relief the way you might hope; if the problem
is that threads are entering a long running transaction and waiting
on Oracle to complete, all you're doing is adding a few more feet to
the rope before Zope hangs itself up again.

DCOracle2 does support event tracing, so you could turn on the trace
log to see what the module is doing when Zope is becoming
nonresponsive. It probably will say that it's waiting in a fetch or
execute statement, so that won't be all too terribly helpful but it
MIGHT indicate that there's a problem with handles (some versions of
DCOracle2 leaked a handle -- make sure you have the latest code from
the CVS repository).

Also, the C layer isn't capable of coping with some result types very
well (if at all) ie using Oracle in object mode or returning table
results from stored procedures, etc.

Also, the PYTHON layer code that currently exists as the DA *does*
leak connections if Oracle closes the connection. This is something
Chris Withers tried to fix, but I think he was never happy with the
solution and so it remains on a branch.

I haven't done any DCO2 development in a little over two years; I
hate to abandon the code but I don't run any current Oracles at home
and its a fairly intensive time commitment to just set up a
development environment. Oracle never gave me any freebies either, /
sniff -- not that I expected any mind you, but the best way to stay
current with them costs $$ that I'm not likely to fork over any time
soon.


On May 3, 2006, at 3:36 PM, Michael Mauws wrote:

> Thanks yet again, Dieter, particularly for putting to rest a few of
> the "solutions" I found while poking around. As you might expect, I've
> been continuing to explore this problem and, at the moment, I'm
> zeroing in on rollback segments. If this is indeed the problem, it
> would fit well with what you're saying....
>
>
> ...michael
>
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db