Mailing List Archive

On 4/14/06, Cynthia Kiser <cnk@caltech.edu> wrote:
> Hope you get an answer to this - but I think you are probably
> asking the wrong list

Thanks for responding. Well, I choose this list because I thought
(without extensive Zope knowledge) a possible solution would be to use
an external DB, like MySQL for example, on a separate server for all
client data and only use the ZoDB for site/programming related data.
My thought was then that I could point both the production server and
the test server at that DB server. I take it this isn't easy to
achieve ?

> and you have found the fatal flaw with Zope:
> it is really hard to separate programming/configuration from the site
> data. Those two things are intertwined in a single binary file, the
> ZODB and things that are easy to do from the admin interface (2 second
> changes) take a long time to script.

It was my understanding though that you can use an external DB in
conjunction with the ZoDB, wouldn't it be possible to put my
client-entered data (form responses mostly) on this external DB ?

Christopher Appell
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

On 4/13/06, jpenny@ykksnap-america.com <jpenny@ykksnap-america.com> wrote:
> Well, this is one reason that I strongly prefer keeping my code
> on the file system. The availability of change-control software
> is the other. I happen to like git.

Well, we are probably going to do this for files (some forms have
associated document files), but I don't see how this would be overly
possible for form data, am I off base ? Oh, I would love to implement
change control software but I figure let's ask for emergency changes
first and then we can look at luxury options.

> I do development on my machine, and as much testing as possible. Then on the production machine I do a cg-update my_machine_name to pull the code from my machine into production.
>
> So, how do you get code in files? The two best I have found are the FileSystemSite from infrae, and the DirectoryView portion of zope CMF. Unfortunately, either require some reformatting of method data, particularly the headers. So some retyping and retesting is required. I am using FileSystemSite, and any new code goes there. Working code stays in the ZODB until I have time or need to fix it.

I'm not sure if this would work for me as 90% of the data I'm worried
about getting squashed is form data entered by the site's users rather
than code the developer is working with.

> Another option, which I have never used, is zsyncer,
> http://sourceforge.net/projects/zsyncer

Thanks, I'll have a look at that. But I guess really what I'm hoping
to find out is how feasible is it to dhunk out the user-entered data
and put it in an exteranl DB so it is safe from being stepped on
during code updates ?

Christopher Appell
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On 14 Apr 2006, at 17:35, Christophe Appell wrote:
> My thought was then that I could point both the production server and
> the test server at that DB server. I take it this isn't easy to
> achieve ?

There is no problem setting that up, but it is plain *bad bad
practice*! Do not do this unless you want to risk corrupting
production data by experimenting and developing on the development
system.


>> and you have found the fatal flaw with Zope:
>> it is really hard to separate programming/configuration from the site
>> data. Those two things are intertwined in a single binary file, the
>> ZODB and things that are easy to do from the admin interface (2
>> second
>> changes) take a long time to script.

I'm not sure where that opinion comes from. It seems you never
considered any development paradigm Zope offers beyond the barebones
through-the-web style which is less and less used by Zope developers.



> It was my understanding though that you can use an external DB in
> conjunction with the ZoDB, wouldn't it be possible to put my
> client-entered data (form responses mostly) on this external DB ?

Yes.

jens

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (Darwin)

iD8DBQFEP8MZRAx5nvEhZLIRAvQCAJ49ELo81F+pBG79TFAHOGd3xEsnrACeO91H
PgfoAbYrIIXp7EVnTGnA/R8=
=ceJ3
-----END PGP SIGNATURE-----
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

On 4/14/06, Jens Vagelpohl <jens@dataflake.org> wrote:
> > My thought was then that I could point both the production server and
> > the test server at that DB server.
> There is no problem setting that up, but it is plain *bad bad
> practice*! Do not do this unless you want to risk corrupting
> production data by experimenting and developing on the development
> system.

I think I wasn't being clear, the idea would of course be to mirror
the production database for development and testing and only point to
the live DB when the build was approved. The problem I'm having right
now is that the developer who's working on the project, when I can
convince him to work off line, will pull a snapshot of the instance
and develop on it, then pushing it back up when done with the coding.
Of course this steps on any new data that has been added to the ZoDB
since he took his snapshot.

And before you ask, no I can't replace him or command him (volunteer
project). All I can do is push really hard to create an environment
that makes it easy for him to do the right thing, which is what I'm
attempting to do.

> > It was my understanding though that you can use an external DB in
> > conjunction with the ZoDB, wouldn't it be possible to put my
> > client-entered data (form responses mostly) on this external DB ?
> Yes.

Can you give me an idea of the level of effort to accomplish this
partial data abstraction ? I imagine it's not so straight forward
since he has pushed back pretty hard to my initial requests but I
figured it's worth asking about as he may simply not know how to go
about it.

Christopher Appell
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

--On 14. April 2006 17:55:15 +0200 Christophe Appell <butchfag@gmail.com>
wrote:
>
>> > It was my understanding though that you can use an external DB in
>> > conjunction with the ZoDB, wouldn't it be possible to put my
>> > client-entered data (form responses mostly) on this external DB ?
>> Yes.
>
> Can you give me an idea of the level of effort to accomplish this
> partial data abstraction ? I imagine it's not so straight forward
> since he has pushed back pretty hard to my initial requests but I
> figured it's worth asking about as he may simply not know how to go
> about it.
>

This depends really on your individual use case. Sometimes you have an
exisiting RDBMS and want to store additional content in the ZODB and
sometimes it is vice versa. The "tricky" part is to keep track which
objects in the ZODB belong to which row(s) in your RDBMS. For such a hybrid
solution I stored the path of the objects in the ZODB as new column in the
primary table of the RDBMS (this is ok as long as you don't move objects
around)...or you might store the primary key as attribute of your objects..
don't try to move data from a RDBMS into Zope if your data depends very
much on the relational mode...the ZODB and RDBMSs have their pros and
cons..just
use them for the purpose they were designed to.

-aj

--
ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany
Web: www.zopyx.com - Email: info@zopyx.com - Phone +49 - 7071 - 793376
E-Publishing, Python, Zope & Plone development, Consulting

On 4/14/06, Andreas Jung <lists@zopyx.com> wrote:
> > Can you give me an idea of the level of effort to accomplish this
> > partial data abstraction ?
> This depends really on your individual use case. Sometimes you have an
> exisiting RDBMS and want to store additional content in the ZODB and
> sometimes it is vice versa.

In this case _everything_ is currently in the ZoDB and I want to be
able to protect some of the data. I am in no way attempting to impose
a database choice, there is no reason in particular in this case that
pushes us to use an RDBMS. In fact another hypothesis I had thought
about was setting up a ZEO server to abstract the data, perhaps this
is a better route ?

Christopher Appell
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

--On 14. April 2006 18:19:38 +0200 Christophe Appell <butchfag@gmail.com>
wrote:

> On 4/14/06, Andreas Jung <lists@zopyx.com> wrote:
>> > Can you give me an idea of the level of effort to accomplish this
>> > partial data abstraction ?
>> This depends really on your individual use case. Sometimes you have an
>> exisiting RDBMS and want to store additional content in the ZODB and
>> sometimes it is vice versa.
>
> In this case _everything_ is currently in the ZoDB and I want to be
> able to protect some of the data.

"protect"?

> I am in no way attempting to impose
> a database choice, there is no reason in particular in this case that
> pushes us to use an RDBMS. In fact another hypothesis I had thought
> about was setting up a ZEO server to abstract the data, perhaps this
> is a better route ?

"abstract"?


Please define both.

-aj

--
ZOPYX Ltd. & Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany
Web: www.zopyx.com - Email: info@zopyx.com - Phone +49 - 7071 - 793376
E-Publishing, Python, Zope & Plone development, Consulting

Christophe Appell wrote at 2006-4-14 17:43 +0200:
>On 4/13/06, jpenny@ykksnap-america.com <jpenny@ykksnap-america.com> wrote:
>> Well, this is one reason that I strongly prefer keeping my code
>> on the file system. The availability of change-control software
>> is the other. I happen to like git.
>
>Well, we are probably going to do this for files (some forms have
>associated document files), but I don't see how this would be overly
>possible for form data, am I off base ? Oh, I would love to implement
>change control software but I figure let's ask for emergency changes
>first and then we can look at luxury options.

We have a situation similar to yours and handle it in the following
way:

We classify things into

* logic/presentation -- maintained on the file sytem

They are synchronized via CVS/SVN

* configuration -- maintained in the ZODB, in separated objects,
and transfered with "ZSyncer" from development to
production sites.

* content -- maintained either in the ZODB or external databases
(depending on whether they are poorly or highly structured)

They are never transfered from development to production site
but occasionally in the other direction.

There are a few object types that unfortunately mix configuration
and content. Catalogs are a prominent example.
We handle them by manually performing the reconfiguration in the production
site that we previously tested in the development site.

--
Dieter
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db

I hope this can help you.

|Thanks for responding. Well, I choose this list because I thought
|(without extensive Zope knowledge) a possible solution would be to use
|an external DB, like MySQL for example, on a separate server for all
|client data and only use the ZoDB for site/programming related data.
|My thought was then that I could point both the production server and
|the test server at that DB server. I take it this isn't easy to
|achieve ?

Yes. You can separate client data from ZODB. I uses 2 PC - one PC as production
server and the other PC as Development Server and Test Server. Note: 2 server
applications on same PC. I uses the Postgresql database on same PC to store my client data.
What you need is database adapter to connect Zope to Postgresql.

|> and you have found the fatal flaw with Zope:
|> it is really hard to separate programming/configuration from the site
|> data. Those two things are intertwined in a single binary file, the
|> ZODB and things that are easy to do from the admin interface (2
|second
|> changes) take a long time to script.

I use Zope as Application Server: meaning all my application
coding are kept in Zope. No client data is store in ZODB other
than the required user id. Zope allows you (2.7 or later) multiple
instance of Zope on the same PC. I used once instance to run
Development Server and a 2nd instance for Test server. Separate
zope instance is like having separate zope server. The application
code between the two instances are synchronize manually using
IMPORT/EXPORT function. Hence, separate team can test the
system separate without affecting each other.

|It was my understanding though that you can use an external DB in
|conjunction with the ZoDB, wouldn't it be possible to put my
|client-entered data (form responses mostly) on this external DB ?

Postgresql allows you to create multiple database - developmentDB
testDB. So, I can connect development server to development DB
and testServer to testDB. However, database synchronizing is
performed manually (through backup & restore function) - data
is mainly static.


When the application is fully tested, it then exported
to Zope Production Server using IMPORT/EXPORT function.

CY



---------------------------------
Talk is cheap. Use Yahoo! Messenger to make PC-to-Phone calls. Great rates starting at 1&cent;/min.

On 4/14/06, Andreas Jung <lists@zopyx.com> wrote:
> > In this case _everything_ is currently in the ZoDB and I want to be
> > able to protect some of the data.
> "protect"?

Meaning I wish to ensure that client-entered data cannot be easily
stepped on by the developer.

> > I am in no way attempting to impose
> > a database choice, there is no reason in particular in this case that
> > pushes us to use an RDBMS. In fact another hypothesis I had thought
> > about was setting up a ZEO server to abstract the data, perhaps this
> > is a better route ?
> "abstract"?

Here I am using abstract to mean separate from, distinct. My goal is
simply to ensure the code for the site is kept separately from the
client-entered data so that the code can be updated and replaced
without touching the client data.

Does that make sense ?

Christopher Appell
_______________________________________________
Zope-DB mailing list
Zope-DB@zope.org
http://mail.zope.org/mailman/listinfo/zope-db