Mailing List Archive

On 6 Oct 2005, at 14:00, rdl3@compaqnet.fr wrote:
> So I went to the "undo" part of the ZMI, and I saw that zope stored
> in the
> database every, EVERY transaction, that means even the simple fact of
> VIEWING a page. I began to understand why it increased so
> dramatically.
>
> So my questions are:
>
> 1) is it normal ?
> 2) is it possible to tell zope NOT to store some things in the
> database
> (for instance in zope.conf-I've looked but not too closely and found
> nothing)

1) It is not normal and usually caused by badly written software that
you run on top of Zope.

2) No.

jens

_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

[rdl3@compaqnet.fr]
> After a successful Database packing (thanks to Dieter!), I compared the
> old and new sizes of my databases. From 26 GB, it was 320 Mb after
> packing.
>
> I thought there was too much difference between both. I watched closely at
> the progression of my new database, and realised it had gained 100 MB in
> just a day !
>
> So I went to the "undo" part of the ZMI, and I saw that zope stored in the
> database every, EVERY transaction, that means even the simple fact of
> VIEWING a page.

A transaction adds info to the database if and only if a persistent
object was modified during the transaction -- and it's the _purpose_
of a database to keep track of changes to persistent objects, so
that's necessary.

Viewing a page alone should not be enough to modify the state of any
persistent object. More must be going on than you currently know
about. You can use fsdump.py to find out which objects are changing:

http://zope.org/Wikis/ZODB/FileStorageBackup

> I began to understand why it increased so dramatically.
>
> So my questions are:
>
> 1) is it normal ?

If and only if you're running applications that do excessive
modification of persistent objects ;-)

> 2) is it possible to tell zope NOT to store some things in the database

No, and it doesn't make sense (as above, it's the database's purpose
to keep track of changes; if "too many" changes are happening, the
cure is to find the software responsible for making those changes, and
alter it to stop making so many changes).

> ...
> PS : I use zope 2.7.2
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

> 2) is it possible to tell zope NOT to store some things in the database
> (for instance in zope.conf-I've looked but not too closely and found
> nothing)
Why don't you run a cron job that packs the database, let's say each
month? You could also do a backup before packing; just in case you want
to undo something later.

Something that would be nice, would be to be able to store transactions
in a separate file before packing, but I don't think it is posible in
Zope. You may have to write something to do this.

Regards,
Josef

_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

ok thanks,
The software responsible for this problem is Nuxeo CPS (Collaborative
Portal Server), which is based upon CMF (I think CMF itself is well-tested
enough not to be accused of being responsible) and is very similar to the
maybe best-known Plone.
(By the way does anyone would know if this happens also with plone ?)

Anyway, I'm gonna ask on CPS mailing lists to see if this is a (hopefully
corrected in new realeases) bug or if this is not well known.

Thanks all again for your help...

_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

On 7 Oct 2005, at 08:27, rdl3@compaqnet.fr wrote:

> ok thanks,
> The software responsible for this problem is Nuxeo CPS (Collaborative
> Portal Server), which is based upon CMF (I think CMF itself is well-
> tested
> enough not to be accused of being responsible) and is very similar
> to the
> maybe best-known Plone.
> (By the way does anyone would know if this happens also with plone ?)
>
> Anyway, I'm gonna ask on CPS mailing lists to see if this is a
> (hopefully
> corrected in new realeases) bug or if this is not well known.

You need to do a little homework first. Try to find out *what* is
writing to the database, what objects get changed. The Undo log might
have some clues, otherwise use a ZODB analyzing tool like Tim suggested.

jens

P.S.: The CMF does *not* write to the database when you view content,
that's for sure.

_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

Josef Meile wrote:
>> 2) is it possible to tell zope NOT to store some things in the database
>> (for instance in zope.conf-I've looked but not too closely and found
>> nothing)
>
> Why don't you run a cron job that packs the database, let's say each
> month? You could also do a backup before packing; just in case you want
> to undo something later.

Urm, in this case, I'd suggest the guy should really find out what's
causing those writes, they're likely impacting heavily on performance,
if nothing else...

> Something that would be nice, would be to be able to store transactions
> in a separate file before packing, but I don't think it is posible in
> Zope. You may have to write something to do this.

Huh? not sure what you mean... if you keep the .old file created during
packing, that has all the transactions in it that would have been packed
away...

cheers,

Chris

PS: Any chance this thread could move to a more appropriate list, oh, I
dunno, say zodb-dev@zope.org or zope@zope.org?

--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk

_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

>> Something that would be nice, would be to be able to store transactions
>> in a separate file before packing, but I don't think it is posible in
>> Zope. You may have to write something to do this.
>
>
> Huh? not sure what you mean... if you keep the .old file created during
> packing, that has all the transactions in it that would have been packed
> away...
I was just thinking about only keeping a log file with the transactions
without the data and let's say apply it to a packed Data.fs. It was just
a nice-to-have idea, nothing else. But I think for the OP would be
enough to keep the Data.fs.old as you said.

_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

> Josef Meile wrote:
>
> Urm, in this case, I'd suggest the guy should really find out what's
> causing those writes, they're likely impacting heavily on performance,
> if nothing else...

Yes that's what I'm doing now... I have an impression that it affects also
RAM
...
>

>
> PS: Any chance this thread could move to a more appropriate list, oh, I
> dunno, say zodb-dev@zope.org or zope@zope.org?
>

Oh I'm sorry, I thought this list was appropriate...
What is the original purpose of this list ?
(and which one could be the most appropriate for the problems I focus on,
according to you ?)

Thanks a lot
Renaud

_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

rdl3@compaqnet.fr wrote:
> Oh I'm sorry, I thought this list was appropriate...
> What is the original purpose of this list ?

Relational Database usage within Zope.

> (and which one could be the most appropriate for the problems I focus on,
> according to you ?)

Either of the ones I mentioned before...

Chris

--
Simplistix - Content Management, Zope & Python Consulting
- http://www.simplistix.co.uk

_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db