Mailing List Archive

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

- From reading: your database model appears pretty much relational.
Where would you take advantages from using Zope as framework?
This sounds like a task for Pyramid + RDBMS or a graph DB.

- -aj

Sareesh Sudhakaran wrote:
> I have a personal project - a web application I wanted to develop -
> but I'm confused on which route to take. I am not under any time
> constraint.
>
>
> *About the App:*
>
> The best I can describe it is as a kind of expert system (but not AI)
> that needs to find the best workflow for a process, given a set of
> initial and final parameters. E.g. a 'capsule' of data must pass
> through many 'tools' or 'environments' to reach a desired output -
> something like a very complicated car wash.
>
> Let's say there are many tools that can be used at various stages in
> the process. I have estimated there are at least 500 tools as of
> now, and it is bound to grow in the future as newer tools are
> introduced. Existing tools will also have version updates.
>
> Each tool, on average, has at least 100 properties that define the
> tool. Some of them have as high as 1000 unique properties. Some of
> these tools are linked to each other - e.g if one tool is selected,
> there are only n tools that can correspond to it for the next step
> in the process. I also have the problem of 'matching' the tools for
> analysis. E.g. Tool A might have only three fixed rpms - 100, 200
> and 500, but Tool B might have rpms from 20 to 2000. I'm not sure how
> I can construct a database without spelling out each number, as in
> the example above.
>
> The total number of tools needed for the process can be defined at
> the beginning, however, it will change as the application becomes
> more complex in the future. I plan to address every contingency in
> the process. The idea is - if the user inputs the initial parameters
> and the desired outcome (another set of parameters), the app must
> find the 'best' path - sort of like a decision tree. The best path
> can be the fastest, cheapest, etc. I would like the user to choose
> what is best for him/her.
>
> Unfortunately, parameters might change, relationships might change
> (but not regularly) - the 'rules' I will be using might be revised
> for better accuracy in prediction.
>
> I also need to track each user's path and solutions' for future
> reference (but no personal details except username and email address
> for logging in). Maybe when the app is up and running, I'd like to
> make it more democratic, with users contributing to refining the
> logic/rules involved.
>
> If possible, I would also like the app to output a graphical
> flowchart at the end showing the workflow with all tools grouped in
> an easy to understand layout.
>
>
> *My questions:*
>
> 1. Will the app be better served with a relational DB like mySQL or
> an Object database? After a lot of research I've guessed that my
> particular case might be better served with Python and Zope/ZODB.
> But I might be wrong? Maybe PHP+mySQL or Django is a better fit? 2.
> Can anyone provide general advice on how to go about beginning such
> a project in ZOPE. Which is the best place to start learning for a
> newbie? 3. Can anyone recommend a good shared hosting provider that
> supports Zope fully but is not expensive? 4. Is there a module or
> app that is open source that I can use to output a graphical
> flowchart based on the results, or will I be better served
> programming it from scratch with Python?
>
> I would appreciate any help in getting started. Thank you in
> advance. I have tried most online forums but have not good any
> productive answers. Most of the answers I got were pro-PHP+mySQL.
>
>
> Adam
>
> _______________________________________________ Zope maillist -
> Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No
> cross posts or HTML encoding! ** (Related lists -
> https://mail.zope.org/mailman/listinfo/zope-announce
> https://mail.zope.org/mailman/listinfo/zope-dev )

- --
ZOPYX Limited | zopyx group
Charlottenstr. 37/1 | The full-service network for Zope & Plone
D-72070 Tübingen | Produce & Publish
www.zopyx.com | www.produce-and-publish.com
- ------------------------------------------------------------------------
E-Publishing, Python, Zope & Plone development, Consulting


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQGUBAEBAgAGBQJO2yPrAAoJEADcfz7u4AZjnJULviZSrj8nLwzeqjgxGT+d9/aU
1x0bj/E3zuPFeKtWkevp26K4OiCe/yw7PmgVXh1xBbfLhFzUcet9Ubdu0eIjfE/8
dC8hayQf4fR8KD3J06CbkMAEJsJur3gTPoFxzDWd7S9ybwuFjb3E14AhiQGPpdYN
DVwE6U3t240Wj7ebNTblASQnvI2sfzr9r2tYVqZc8O+SMyROL1oIs56RNofT4pz5
p9OXiYlnHmn1keCbEKnYk1e2zTU7kFJdGQz/Uy+yV4QTiU7nMwhxWCx1gBKxVgNk
XseqPdWKl9epz/h1Pb0qPTvK6PXl46Cj/0Bls/XU6yHDDpB49SqMXai2i6VIAzdL
lik6JwSPVxBv24eTngrisx2qvhl1ln63EM+AWwky7jUgNl0cwALNdR3Gj+zDVioc
ZWAZfWcuI3TFKYpwJiwzNgT0DSbOLvlM/09xXPsxmE+rfXT8arTxMSEgaQS60LqJ
gil8bt+flbzjAJ3GEwza2KmoLpS/zxc=
=gVTz
-----END PGP SIGNATURE-----

Thanks AJ. I assumed an OODBMS would be the right choice because of the object nature of my 'tools'. Of course, it was an assumption. Instead of Pyramid+RDBMS, can I use PHP+mySQL (my current hosting provider supports this).If ZODB isn't for my project, then would a GraphDB help? I have no idea on where to start with Graph DB - I've read the wikis and it's made me more confused.
-ss


> Date: Sun, 4 Dec 2011 08:40:28 +0100
> From: lists@zopyx.com
> To: aysand@hotmail.com
> CC: zope@zope.org
> Subject: Re: [Zope] Help in deciding approach to Web App
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> - From reading: your database model appears pretty much relational.
> Where would you take advantages from using Zope as framework?
> This sounds like a task for Pyramid + RDBMS or a graph DB.
>
> - -aj
>
> Sareesh Sudhakaran wrote:
> > I have a personal project - a web application I wanted to develop -
> > but I'm confused on which route to take. I am not under any time
> > constraint.
> >
> >
> > *About the App:*
> >
> > The best I can describe it is as a kind of expert system (but not AI)
> > that needs to find the best workflow for a process, given a set of
> > initial and final parameters. E.g. a 'capsule' of data must pass
> > through many 'tools' or 'environments' to reach a desired output -
> > something like a very complicated car wash.
> >
> > Let's say there are many tools that can be used at various stages in
> > the process. I have estimated there are at least 500 tools as of
> > now, and it is bound to grow in the future as newer tools are
> > introduced. Existing tools will also have version updates.
> >
> > Each tool, on average, has at least 100 properties that define the
> > tool. Some of them have as high as 1000 unique properties. Some of
> > these tools are linked to each other - e.g if one tool is selected,
> > there are only n tools that can correspond to it for the next step
> > in the process. I also have the problem of 'matching' the tools for
> > analysis. E.g. Tool A might have only three fixed rpms - 100, 200
> > and 500, but Tool B might have rpms from 20 to 2000. I'm not sure how
> > I can construct a database without spelling out each number, as in
> > the example above.
> >
> > The total number of tools needed for the process can be defined at
> > the beginning, however, it will change as the application becomes
> > more complex in the future. I plan to address every contingency in
> > the process. The idea is - if the user inputs the initial parameters
> > and the desired outcome (another set of parameters), the app must
> > find the 'best' path - sort of like a decision tree. The best path
> > can be the fastest, cheapest, etc. I would like the user to choose
> > what is best for him/her.
> >
> > Unfortunately, parameters might change, relationships might change
> > (but not regularly) - the 'rules' I will be using might be revised
> > for better accuracy in prediction.
> >
> > I also need to track each user's path and solutions' for future
> > reference (but no personal details except username and email address
> > for logging in). Maybe when the app is up and running, I'd like to
> > make it more democratic, with users contributing to refining the
> > logic/rules involved.
> >
> > If possible, I would also like the app to output a graphical
> > flowchart at the end showing the workflow with all tools grouped in
> > an easy to understand layout.
> >
> >
> > *My questions:*
> >
> > 1. Will the app be better served with a relational DB like mySQL or
> > an Object database? After a lot of research I've guessed that my
> > particular case might be better served with Python and Zope/ZODB.
> > But I might be wrong? Maybe PHP+mySQL or Django is a better fit? 2.
> > Can anyone provide general advice on how to go about beginning such
> > a project in ZOPE. Which is the best place to start learning for a
> > newbie? 3. Can anyone recommend a good shared hosting provider that
> > supports Zope fully but is not expensive? 4. Is there a module or
> > app that is open source that I can use to output a graphical
> > flowchart based on the results, or will I be better served
> > programming it from scratch with Python?
> >
> > I would appreciate any help in getting started. Thank you in
> > advance. I have tried most online forums but have not good any
> > productive answers. Most of the answers I got were pro-PHP+mySQL.
> >
> >
> > Adam
> >
> > _______________________________________________ Zope maillist -
> > Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No
> > cross posts or HTML encoding! ** (Related lists -
> > https://mail.zope.org/mailman/listinfo/zope-announce
> > https://mail.zope.org/mailman/listinfo/zope-dev )
>
> - --
> ZOPYX Limited | zopyx group
> Charlottenstr. 37/1 | The full-service network for Zope & Plone
> D-72070 Tübingen | Produce & Publish
> www.zopyx.com | www.produce-and-publish.com
> - ------------------------------------------------------------------------
> E-Publishing, Python, Zope & Plone development, Consulting
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (Darwin)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iQGUBAEBAgAGBQJO2yPrAAoJEADcfz7u4AZjnJULviZSrj8nLwzeqjgxGT+d9/aU
> 1x0bj/E3zuPFeKtWkevp26K4OiCe/yw7PmgVXh1xBbfLhFzUcet9Ubdu0eIjfE/8
> dC8hayQf4fR8KD3J06CbkMAEJsJur3gTPoFxzDWd7S9ybwuFjb3E14AhiQGPpdYN
> DVwE6U3t240Wj7ebNTblASQnvI2sfzr9r2tYVqZc8O+SMyROL1oIs56RNofT4pz5
> p9OXiYlnHmn1keCbEKnYk1e2zTU7kFJdGQz/Uy+yV4QTiU7nMwhxWCx1gBKxVgNk
> XseqPdWKl9epz/h1Pb0qPTvK6PXl46Cj/0Bls/XU6yHDDpB49SqMXai2i6VIAzdL
> lik6JwSPVxBv24eTngrisx2qvhl1ln63EM+AWwky7jUgNl0cwALNdR3Gj+zDVioc
> ZWAZfWcuI3TFKYpwJiwzNgT0DSbOLvlM/09xXPsxmE+rfXT8arTxMSEgaQS60LqJ
> gil8bt+flbzjAJ3GEwza2KmoLpS/zxc=
> =gVTz
> -----END PGP SIGNATURE-----

Thanks Niels. Just to clarify:Does my particular instance fall under an OODBMS model or a RDBMS model (with ORM if necessary)?
I will begin by reading the Zope Book. Thanks for your assistance. Appreciate it.
-ss


> Subject: Re: [Zope] Help in deciding approach to Web App
> From: nd@syndicat.com
> Date: Sun, 4 Dec 2011 09:24:10 +0100
> To: aysand@hotmail.com; zope@zope.org
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
>
>
>
>
> Sareesh Sudhakaran <aysand@hotmail.com> schrieb:
>
> >My questions:Will the app be better served with a relational DB like
> >mySQL or an Object database? After a lot of research I've guessed that
> >my particular case might be better served with Python and Zope/ZODB.
> >But I might be wrong? Maybe PHP+mySQL or Django is a better fit?Can
> >anyone provide general advice on how to go about beginning such a
> >project in ZOPE.
> This hardly depends from your data structure, but with Zope you have the option to use ZODB and SQL DBs like MySQL in parallel. Typical relational data should go into a SQLDB while complex / oo Data structures should go into ZODB.
>
> We developed several complex to very complex web based applications - incl. expert systems - on Zope ZODB plus MySQL. I can't believe that someone would be able to solve such issues with PHP/SQL within the same time / ressources.
>
>
> Which is the best place to start learning for a
> >newbie?
> You should start by trying the short zope practice / excercise as described in the Zope book.
>
> The Zope Book should be the best source for getting into Zope step by step from "nothing". ß)
>
>
> Can anyone recommend a good shared hosting provider that
> >supports Zope fully but is not expensive?
> Looking for "zope hosting" or similiar in google should advice you. Our company - as one of many - offers Zope hosting to.
>
>
> Is there a module or app that
> >is open source that I can use to output a graphical flowchart based on
> >the results, or will I be better served programming it from scratch
> >with Python?
> There are different modules as i.e. Python Imaging (PIL) and higher level modules.
>
>
> I would appreciate any help in getting started. Thank you
> >in advance. I have tried most online forums but have not good any
> >productive answers. Most of the answers I got were pro-PHP+mySQL.
>
> Yes, because the peoples did not know anything other solution usually...
>
>
> cheers,
>
>
> Niels.
>
> - --
> Niels Dettenbach
> Syndicat IT&Internet
> http://www.syndicat.com
> -----BEGIN PGP SIGNATURE-----
> Version: APG v1.0.8
>
> iIEEAREIAEEFAk7bLio6HE5pZWxzIERldHRlbmJhY2ggKFN5bmRpY2F0IElUJklu
> dGVybmV0KSA8bmRAc3luZGljYXQuY29tPgAKCRBU3ERlZRyiDUhlAJ4/XPl3Oet6
> XN4UlkQ611FNoWqZCwCfZ1IPVjaLMD32wOlFE9cDnrm6bJQ=
> =ukyi
> -----END PGP SIGNATURE-----
>

On 12/04/2011 09:52 AM, Sareesh Sudhakaran wrote:
> Thanks Niels. Just to clarify:
> Does my particular instance fall under an OODBMS model or a RDBMS
> model (with ORM if necessary)?
>
>
Data modelling is a bit of an art and probably you could tackle your
problem with any approach. I think the important is for you to figure
out which model suits more your personality. No kidding.

I would personally start with the RDBMS approach considering only the
information you provide. Also, you can easily use zope in combination
with a RDBMS. When you read the book, beware that zope has been changing
from a "through the web" approach, to a typical file system based
approach, which is a loss, but it seems to be what suits the needs of
the zope developers.

The approach I use is:

zpt page -> (one) Python Script -> (some) ZSQL Methods -> MySQL

The zpt defines the layout of the pages, the ZSQL Methods retrieve the
data needed and the Python script massages the data to make it suitable
for the ZPT page. Or the other way around, from user input in a form to
storage in the database.

The advantage of the relational approach is that it is a very well
understood model and although different people will still come to
different models there are sufficient objective guidelines out there
(e.g., the normalization rules, and then criteria for when to
denormalise) to help you along. Furthermore, there are lots of people in
db-related forums that can help you.

Also, RDBMS provides you with a "standard" query language, SQL, which
plenty of systems and tools can use. In general, RDBMS gives you the
safest approach to keep your data and not loose it when you need to
migrate either the front-end or the back-end. This language is very
powerful and can avoid you a lot of low level programming.

However, plenty of people can not deal well with SQL because it follows
a paradigm so different from the classic imperative programming. With
SQL, you specify the "what", with the other languages you specify the
"how". The advantage of the "what" is that you can do a lot of data
processing in a few lines of code. The problem with the "what" is that
because you don't know the "how" of it, you feel you don't have control
and you are led to say the language is "obscure" or "unreadable".

However, even if you are not comfortable with the "what" (you have to
try to know), you can still rely on an library like SQLAlchemy to keep
you a bit in the comfort zone of the "how". So instead of learning SQL,
you need to learn the API of a specific library. Your choice. I
recommend the first by far.

The real main issue with Relational is that it is a highly structured
data model. It allows you to keep high quality data but if you don't get
it right soon enough in the development cycle, some later changes can
have a huge impact in the application, requiring rewrites. Furthermore,
it works the best when you have many objects with the same properties.
If you have many entities all different from each other (the tools in
your case, maybe), then maybe a OODBMS might be better. But here, there
is no standard language, or standard whatever. Perosnally, I would avoid
as much as possible to put data in something like ZODB (I use it merely
to store and manage the application).

The problem with your specific case is that it does not seem to be a
typical case of books and authors, which might be a risk for someone
without experience. The issue "Tool A might have only three fixed rpms -
100, 200 and 500, but Tool B might have rpms from 20 to 2000", is indeed
tricky. I suspect in general the needs of your system will be too
specific to be able to rely only on SQL queries. You would need to put a
lot of intelligence in the data (really highly structured) and it might
become unmanageable or not scalable.

I guess you will need to put a lot of intelligence in the Python Script.
So, the ZSQL retrieves the relevant tool data and then makes the tool
choice. The knowledge of the meaning of the attributes is maintained in
programming.

I should say I am not totally sure the Python Script is the best
approach here, in terms of management facilities. But Python is surely a
very good language due to its readability. However, you might need to
use External methods or a more typical file-system based Python approach.

Or maybe you actually need to create a Domain Specific Language to
encode the intelligence needed for your tool selection process. If your
python code becomes repetitive, with patterns showing up, then a DSL
might be a good approach, but this might be too much engineering for you
at this stage. It looks like you are in a typical CIM scenario and I
remember handling a similar problem 20 years ago. I vaguely remember at
that time to use custom graph structures in C and the the intelligence
was coded in Lisp/Scheme. So, there is a big range of solutions to try
out :)

If you have time, then the simple approach

zpt page -> (one) Python Script -> (some) ZSQL Methods -> MySQL database

might be a good starting point. You should not need to spend much time
to implement a prototype using this approach. In the worse case scenario
it helps you understand better your problem and what could be a better
approach with little investment. Essentially you try the classical
approach and if it does not work well, either you are doing something
wrong, or you have a better understanding of your needs and where to go.

Good luck,
Fernando

Thanks Fernando! I really appreciate the time and effort you have put in answering my query. My personality sides with Python but my hosting provider does not support Django or Zope.
As you mentioned, if I have to use mySQL, isn't it better for me to go with PHP+mySQL - easier to learn and deploy? Can I just start out with a framework like Symphony instead?
In the future I'll have to use either Python or C/C++ for my business logic and math. But the focus now is to get a prototype out, and if I'm doomed to change everything later I might as well start with the easiest and most obvious. Is this a viable starting point compared to what you have suggested? Or am I missing something?
-Sareesh
Date: Sun, 4 Dec 2011 13:28:36 +0100
From: fernando@cmartins.nl
To: aysand@hotmail.com
CC: nd@syndicat.com; zope@zope.org
Subject: Re: [Zope] Help in deciding approach to Web App









On 12/04/2011 09:52 AM, Sareesh Sudhakaran wrote:



Thanks
Niels. Just to clarify:
Does
my particular instance fall under an OODBMS model or a RDBMS
model (with ORM if necessary)?







Data modelling is a bit of an art and probably you could tackle your
problem with any approach. I think the important is for you to
figure out which model suits more your personality. No kidding.



I would personally start with the RDBMS approach considering only
the information you provide. Also, you can easily use zope in
combination with a RDBMS. When you read the book, beware that zope
has been changing from a "through the web" approach, to a typical
file system based approach, which is a loss, but it seems to be what
suits the needs of the zope developers.



The approach I use is:



zpt page -> (one) Python Script -> (some) ZSQL Methods ->
MySQL



The zpt defines the layout of the pages, the ZSQL Methods retrieve
the data needed and the Python script massages the data to make it
suitable for the ZPT page. Or the other way around, from user input
in a form to storage in the database.



The advantage of the relational approach is that it is a very well
understood model and although different people will still come to
different models there are sufficient objective guidelines out there
(e.g., the normalization rules, and then criteria for when to
denormalise) to help you along. Furthermore, there are lots of
people in db-related forums that can help you.



Also, RDBMS provides you with a "standard" query language, SQL,
which plenty of systems and tools can use. In general, RDBMS gives
you the safest approach to keep your data and not loose it when you
need to migrate either the front-end or the back-end. This language
is very powerful and can avoid you a lot of low level programming.



However, plenty of people can not deal well with SQL because it
follows a paradigm so different from the classic imperative
programming. With SQL, you specify the "what", with the other
languages you specify the "how". The advantage of the "what" is that
you can do a lot of data processing in a few lines of code. The
problem with the "what" is that because you don't know the "how" of
it, you feel you don't have control and you are led to say the
language is "obscure" or "unreadable".



However, even if you are not comfortable with the "what" (you have
to try to know), you can still rely on an library like SQLAlchemy to
keep you a bit in the comfort zone of the "how". So instead of
learning SQL, you need to learn the API of a specific library. Your
choice. I recommend the first by far.



The real main issue with Relational is that it is a highly
structured data model. It allows you to keep high quality data but
if you don't get it right soon enough in the development cycle, some
later changes can have a huge impact in the application, requiring
rewrites. Furthermore, it works the best when you have many objects
with the same properties. If you have many entities all different
from each other (the tools in your case, maybe), then maybe a OODBMS
might be better. But here, there is no standard language, or
standard whatever. Perosnally, I would avoid as much as possible to
put data in something like ZODB (I use it merely to store and manage
the application).



The problem with your specific case is that it does not seem to be a
typical case of books and authors, which might be a risk for someone
without experience. The issue "Tool A might have only three fixed
rpms - 100, 200 and 500, but Tool B might have rpms from 20 to
2000", is indeed tricky. I suspect in general the needs of your
system will be too specific to be able to rely only on SQL queries.
You would need to put a lot of intelligence in the data (really
highly structured) and it might become unmanageable or not scalable.




I guess you will need to put a lot of intelligence in the Python
Script. So, the ZSQL retrieves the relevant tool data and then makes
the tool choice. The knowledge of the meaning of the attributes is
maintained in programming.



I should say I am not totally sure the Python Script is the best
approach here, in terms of management facilities. But Python is
surely a very good language due to its readability. However, you
might need to use External methods or a more typical file-system
based Python approach.



Or maybe you actually need to create a Domain Specific Language to
encode the intelligence needed for your tool selection process. If
your python code becomes repetitive, with patterns showing up, then
a DSL might be a good approach, but this might be too much
engineering for you at this stage. It looks like you are in a
typical CIM scenario and I remember handling a similar problem 20
years ago. I vaguely remember at that time to use custom graph
structures in C and the the intelligence was coded in Lisp/Scheme.
So, there is a big range of solutions to try out :)



If you have time, then the simple approach



zpt page -> (one) Python Script -> (some) ZSQL Methods ->
MySQL database



might be a good starting point. You should not need to spend much
time to implement a prototype using this approach. In the worse case
scenario it helps you understand better your problem and what could
be a better approach with little investment. Essentially you try the
classical approach and if it does not work well, either you are
doing something wrong, or you have a better understanding of your
needs and where to go.



Good luck,

Fernando

On 12/04/2011 05:15 PM, Sareesh Sudhakaran wrote:
> As you mentioned, if I have to use mySQL, isn't it better for me to go
> with PHP+mySQL - easier to learn and deploy? Can I just start out with
> a framework like Symphony instead?

Well, if all you have is PHP + MySQL in your provider, there is no
"which is better" question, is it?

You might want to look at http://phptal.org/ a library that provides a
templating system similar to ZPT. The advantage is the better separation
between presentation and business layers.

Regards,
Fernando

Aloha,

Very briefly, from what you describe, it looks like you are dealing with
large numbers of complex objects (your 'tools') that interact with each
other - and with some other elements that are going through this
workflow process? And a context for the process? ...not sure of that part...

In any case, this looks (to me anyhow) like a very object-oriented
system you are modeling so an object oriented approach and language
would seem most suitable. Python is one reasonable language option; zope
for the web publishing aspect of the project would fit well with that. I
haven't worked with other OO languages enough to usefully compare.

It also sounds to me like the web publishing is the lesser part of
this...? That is, the workflow of interacting tools is the real 'app'
here, a process which does not all get shown somehow on a web page...or
does it? Or just the outcomes?

LAMP platform is so common and cheap that it's all a lot of people know
about. It can be used well, and, it is all too easy to make horrible
hacks and Frankestein monster apps in PHP/MySQL.

Meaning, a highly structured (yet powerful) OO programming language will
support you in avoiding that.

Also, for development (or just to explore if python+zope might meet your
needs) you don't need any hosting, you can install python+zope on your
development system and try it out.

best wishes...
John S.


On 12/03/2011 07:12 PM, Sareesh Sudhakaran wrote:
> I have a personal project - a web application I wanted to develop - but
> I'm confused on which route to take. I am not under any time constraint.
>
>
> *About the App:*
>
> The best I can describe it is as a kind of expert system (but not AI)
> that needs to find the best workflow for a process, given a set of
> initial and final parameters. E.g. a 'capsule' of data must pass through
> many 'tools' or 'environments' to reach a desired output - something
> like a very complicated car wash.
>
> Let's say there are many tools that can be used at various stages in the
> process. I have estimated there are at least 500 tools as of now, and it
> is bound to grow in the future as newer tools are introduced. Existing
> tools will also have version updates.
>
> Each tool, on average, has at least 100 properties that define the tool.
> Some of them have as high as 1000 unique properties. Some of these tools
> are linked to each other - e.g if one tool is selected, there are only n
> tools that can correspond to it for the next step in the process. I also
> have the problem of 'matching' the tools for analysis. E.g. Tool A might
> have only three fixed rpms - 100, 200 and 500, but Tool B might have
> rpms from 20 to 2000. I'm not sure how I can construct a database
> without spelling out each number, as in the example above.
>
> The total number of tools needed for the process can be defined at the
> beginning, however, it will change as the application becomes more
> complex in the future. I plan to address every contingency in the
> process. The idea is - if the user inputs the initial parameters and the
> desired outcome (another set of parameters), the app must find the
> 'best' path - sort of like a decision tree. The best path can be the
> fastest, cheapest, etc. I would like the user to choose what is best for
> him/her.
>
> Unfortunately, parameters might change, relationships might change (but
> not regularly) - the 'rules' I will be using might be revised for better
> accuracy in prediction.
>
> I also need to track each user's path and solutions' for future
> reference (but no personal details except username and email address for
> logging in). Maybe when the app is up and running, I'd like to make it
> more democratic, with users contributing to refining the logic/rules
> involved.
>
> If possible, I would also like the app to output a graphical flowchart
> at the end showing the workflow with all tools grouped in an easy to
> understand layout.
>
>
> *My questions:*
>
> 1. Will the app be better served with a relational DB like mySQL or an
> Object database? After a lot of research I've guessed that my
> particular case might be better served with Python and Zope/ZODB.
> But I might be wrong? Maybe PHP+mySQL or Django is a better fit?
> 2. Can anyone provide general advice on how to go about beginning such
> a project in ZOPE. Which is the best place to start learning for a
> newbie?
> 3. Can anyone recommend a good shared hosting provider that supports
> Zope fully but is not expensive?
> 4. Is there a module or app that is open source that I can use to
> output a graphical flowchart based on the results, or will I be
> better served programming it from scratch with Python?
>
> I would appreciate any help in getting started. Thank you in advance. I
> have tried most online forums but have not good any productive answers.
> Most of the answers I got were pro-PHP+mySQL.
>
>
> Adam
>
>
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> https://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> https://mail.zope.org/mailman/listinfo/zope-announce
> https://mail.zope.org/mailman/listinfo/zope-dev )

--
John Schinnerer - M.A., Whole Systems Design
--------------------------------------------
- Eco-Living -
Whole Systems Design Services
People - Place - Learning - Integration
john@eco-living.net
http://eco-living.net
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )

Thanks Fernando. I would choose ZOPE or Django and a new provider at the drop of a hat - if someone can confirm if that's the way to go. However, since, there are too many grey areas, it might be better if I stuck to what I have and see how things turn out. Once again, thanks for your support. Appreciate it!
- Sareesh


Date: Sun, 4 Dec 2011 18:19:25 +0100
From: fernando@cmartins.nl
To: aysand@hotmail.com
CC: nd@syndicat.com; zope@zope.org
Subject: Re: [Zope] Help in deciding approach to Web App









On 12/04/2011 05:15 PM, Sareesh Sudhakaran wrote:



As you mentioned, if I have to use mySQL, isn't it better for me
to go with PHP+mySQL - easier to learn and deploy? Can I just
start out with a framework like Symphony instead?





Well, if all you have is PHP + MySQL in your provider, there is no
"which is better" question, is it?



You might want to look at http://phptal.org/ a library that provides
a templating system similar to ZPT. The advantage is the better
separation between presentation and business layers.



Regards,

Fernando

On 12/04/2011 09:31 PM, John Schinnerer wrote:
> In any case, this looks (to me anyhow) like a very object-oriented
> system you are modeling so an object oriented approach and language
> would seem most suitable.

And how would you create (and update) objects in Python for:

"at least 500 tools as of now, and it is bound to grow in the future as
newer tools are introduced. Existing tools will also have version updates.

Each tool, on average, has at least 100 properties that define the tool.
Some of them have as high as 1000 unique properties."

Regards,
Fernando
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )

On 12/04/2011 09:56 PM, Fernando Martins wrote:
> On 12/04/2011 09:31 PM, John Schinnerer wrote:
>> In any case, this looks (to me anyhow) like a very object-oriented
>> system you are modeling so an object oriented approach and language
>> would seem most suitable.
>
> And how would you create (and update) objects in Python for:
>
> "at least 500 tools as of now, and it is bound to grow in the future as
> newer tools are introduced. Existing tools will also have version updates.
>
> Each tool, on average, has at least 100 properties that define the tool.
> Some of them have as high as 1000 unique properties."

How familiar are you with OOP?

What I mean is, when I read your high-level description of what you want
to do, I imagine objects interacting with other. In your "car wash"
example I see that, for instance. Or any kind of work-flow, which is
what this sounds like to me. A state machine.

Each tool is an object; it knows what its properties and abilities and
possible states are and can communicate them to other objects and can
accept and act on communication from other objects. In that way the
objects interact with one another to do whatever it is you build them to
do.

I'm thinking of your "car wash" metaphor. In a car wash there are a
variety of elements (objects) that communicate with one another to move
the car through and to wash it as it moves through. Simple example:

* mechanism that pulls or pushes the car through
* mechanism that sprays it with water and soap
* mechanism that scrubs and wipes and rubs it
* mechanism that sprays it with rinse water
* drying mechanism(s)

These all need to signal one another so they do the right thing at the
right time. Spray (or scrub or blow hot air) only when the car is in the
right place for each.

If some tools can be built by adding to/extending other tools, that is
an object-friendly situation, since enhancing an existing tool to make
another that you need saves you having to create all tools from scratch.

In the car wash, maybe the different sized scrubbers used are all made
by bolting together two or more of the smallest size scrubber. Maybe in
parallel, maybe in series, depends on what is needed.
Maybe the same scrubber controller can have different size brushes
attached to it.

In short, an object - tool - is code and data that interacts with other
tools (other entities of code and data).

An upgrade to a tool would involve changing the code and/or data that
constitutes the tool. If that introduces some new way of interacting
that other tools need to also know, then you add that to those tools also.

Adding more tools means coding them. If they can be based on existing
tools, so much the easier (and more object-appropriate).

So that's some high-level information about an object-oriented approach
to what I think your project is about.

Tthe flexibility you appear to need as the system grows may be
problematic for an RDB. And, my bias is OO, so perhaps someone who
thinks in RDB-space can describe at high level how this would look in an
RDB implmentation.

cheers,
John S.





--
John Schinnerer - M.A., Whole Systems Design
--------------------------------------------
- Eco-Living -
Whole Systems Design Services
People - Place - Learning - Integration
john@eco-living.net
http://eco-living.net
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )

Am Sonntag, 4. Dezember 2011, 16:15:13 schrieben Sie:
> As you mentioned, if I have to use mySQL, isn't it better for me to go with
> PHP+mySQL - easier to learn and deploy?

...just from my experience:

PHP is - for different, but mainly technical/historical reasons - very widely
spread within web applications, one major reason was/is i.e. the large
(because "easy") availability on low cost hosting environments in the past -
but the most advantages was/are on the side of the hosting providers....

PHP might be easier to learn then other languages or frameworks, but
maintaining large / complex applications / software projects within PHP could
be a real mess.

We develop nearly any web application with Zope / ZODB since >= 10 years but
are a hosting company byself - so we was not bound to PHP as many other
internet hosting users in the past. A colleagues company produces very high
level expert systems on Perl and Catalyst - requiring high skilled Perl
programmers.

From my experience developing within Zope / ZODB (with Python, DTML and/or
ZPT) allows very high quality products within very short timeframes and even
further maintaining the project is relative ressource efficient - especially
compared to PHP.

Most web application data structures (i.e. a "simple" web page) fit's much
better by a oo object strategy then a relational (RDBMS) one.

The major typical ressource hole within typical PHP+SQL web applications or
i.e. a CMS solution is the translation of typical data objects into tables and
vice versa. Producing i.e. one "simple" CMS page within a PHP-SQL CMS easily
could trigger hundreds of SQL requests into many different tables - a
significant overhead which has to implemented by developers and handled by the
machines.

But this is my view onto the issue - just my two cents...



cheers,


Niels.

--
---
Niels Dettenbach
Syndicat IT&Internet
http://www.syndicat.com/

On Dec 5, 2011 09:48 "John Schinnerer" <john@eco-living.net> wrote:

> On 12/04/2011 09:56 PM, Fernando Martins wrote:
> > On 12/04/2011 09:31 PM, John Schinnerer wrote:
> > > In any case, this looks (to me anyhow) like a very object-oriented
> > > system you are modeling so an object oriented approach and
> > > language
> > > would seem most suitable.
> >
> > And how would you create (and update) objects in Python for:
> >
> > "at least 500 tools as of now, and it is bound to grow in the future
> > as
> > newer tools are introduced. Existing tools will also have version
> > updates.
> >
> > Each tool, on average, has at least 100 properties that define the
> > tool.
> > Some of them have as high as 1000 unique properties."
>
> How familiar are you with OOP?
>
>
I think you are confusing me with the OP. And you did not answer my
question. Are you recommending that a programmer codes all these objects
by hand in Python?

The know-how of what constitutes a tool, their properties and even the
tool selection criteria is not developer know-how. Therefore, this
information should be defined outside the program in way that the tool
expert(s) can manage it. Which leads us to some storage solution, an RDB
being the most common.

Since, as I pointed out before, SQL most likely would not be able to do
the tool selection alone, Python would merely load the data from the
external source and create objects and expertise on the fly.

Anyway, I agree with you that the main issue does not seem to be the web
publishing solution, but rather how to represent the tool information
and how to do tool selection, which is off-topic.

Regards,
Fernando

Hi NielsI agree with you, even though I have no experience.
But I'm restricted by hosting options for Zope at the moment, and will revert to Python once the project is deployed - and when I figure out whether mySQL is good enough or not. I hate having to type all those extra characters in php though.sareesh


> From: nd@syndicat.com
> To: aysand@hotmail.com; zope@zope.org
> Subject: Re: [Zope] Help in deciding approach to Web App
> Date: Mon, 5 Dec 2011 11:25:18 +0100
>
> Am Sonntag, 4. Dezember 2011, 16:15:13 schrieben Sie:
> > As you mentioned, if I have to use mySQL, isn't it better for me to go with
> > PHP+mySQL - easier to learn and deploy?
>
> ...just from my experience:
>
> PHP is - for different, but mainly technical/historical reasons - very widely
> spread within web applications, one major reason was/is i.e. the large
> (because "easy") availability on low cost hosting environments in the past -
> but the most advantages was/are on the side of the hosting providers....
>
> PHP might be easier to learn then other languages or frameworks, but
> maintaining large / complex applications / software projects within PHP could
> be a real mess.
>
> We develop nearly any web application with Zope / ZODB since >= 10 years but
> are a hosting company byself - so we was not bound to PHP as many other
> internet hosting users in the past. A colleagues company produces very high
> level expert systems on Perl and Catalyst - requiring high skilled Perl
> programmers.
>
> From my experience developing within Zope / ZODB (with Python, DTML and/or
> ZPT) allows very high quality products within very short timeframes and even
> further maintaining the project is relative ressource efficient - especially
> compared to PHP.
>
> Most web application data structures (i.e. a "simple" web page) fit's much
> better by a oo object strategy then a relational (RDBMS) one.
>
> The major typical ressource hole within typical PHP+SQL web applications or
> i.e. a CMS solution is the translation of typical data objects into tables and
> vice versa. Producing i.e. one "simple" CMS page within a PHP-SQL CMS easily
> could trigger hundreds of SQL requests into many different tables - a
> significant overhead which has to implemented by developers and handled by the
> machines.
>
> But this is my view onto the issue - just my two cents...
>
>
>
> cheers,
>
>
> Niels.
>
> --
> ---
> Niels Dettenbach
> Syndicat IT&Internet
> http://www.syndicat.com/

Am Montag, 5. Dezember 2011, 11:37:46 schrieb Sareesh Sudhakaran:
> But I'm restricted by hosting options for Zope at the moment, and will
> revert to Python once the project is deployed - and when I figure out
> whether mySQL is good enough or not. I hate having to type all those extra
> characters in php though.sareesh

If i talk about Zope / Python i mean Zope (with Zope Python Script Objects
and/or external (Python) Methods). At a earlier stage Zope devels discussed
for integrating ingres or another RDBMS natively into Zope - but this is not
longer the case as there are many Zope adapters / integrations / products
available for different major RDBMS like MySQL or Postgres.

For the data structures where you have to handle large tables MySQL would be
the first choice while oo data structures would preferrably go into your ZODB.
I.e. we handle large amounts of user data records within MySQL while all of
the web content objects or even complex shopping products are handled within
ZODB - both within the same Shopping Cart application. This all depends highly
from you data model. By theory you are able to handle both in just on of the
DB solutions.

With Zope you have many options to use external database solutions within your
Zope based application.


best regards,


Niels.
--
---
Niels Dettenbach
Syndicat IT&Internet
http://www.syndicat.com/

On Dec 5, 2011 10:25 "Niels Dettenbach" <nd@syndicat.com> wrote:

> From my experience developing within Zope / ZODB (with Python, DTML
> and/or ZPT) allows very high quality products within very short
> timeframes and even further maintaining the project is relative
> ressource efficient - especially compared to PHP.
>
>
How would you put 500+ objects (for the tools) each with hundreds or
thousands of attributes in in ZODB?

Regards,
Fernando

On Sun, Dec 4, 2011 at 06:12, Sareesh Sudhakaran <aysand@hotmail.com> wrote:
> I have a personal project - a web application I wanted to develop - but I'm
> confused on which route to take. I am not under any time constraint.

Your case is complex and the answer is non-obvious. I think you will
have to try to see.

> The best I can describe it is as a kind of expert system (but not AI) that
> needs to find the best workflow for a process, given a set of initial and
> final parameters. E.g. a 'capsule' of data must pass through many 'tools' or
> 'environments' to reach a desired output - something like a very complicated
> car wash.
>
> Let's say there are many tools that can be used at various stages in the
> process. I have estimated there are at least 500 tools as of now, and it is
> bound to grow in the future as newer tools are introduced. Existing tools
> will also have version updates.
>
> Each tool, on average, has at least 100 properties that define the tool.

This kind of complexity and flexibility that your case has does lend
itself well to the ZODB. But it is by no means impossible to do in a
relational database. What you would need to do here if you did this in
SQL is to have one table for the tools, and one table for the
properties, and one table that for each tool and property has a value.
This might be a bit complex to use of you use an ORM, but it should be doable.

> problem of 'matching' the tools for analysis. E.g. Tool A might have only
> three fixed rpms - 100, 200 and 500, but Tool B might have rpms from 20 to
> 2000. I'm not sure how I can construct a database without spelling out each
> number, as in the example above.

Well, if you want to search for a specific RPM range it does get
complicated, because then you probably want to store the RPM values as
integers. And that means that you in the property table needs to have
several columns depending on the value type. If the property needs
text, you need to fetch it from the text column. If the property is an
integer you need to fetch it from the integer column. If the property
is a reference to another table, because it is a multi-select property
with a limited set, you need to fetch it from a column for that, which
in turn refers to another table with the actual values.

This *does* get very complex very quickly. Of course, the ZODB has few
such problems.

> Will the app be better served with a relational DB like mySQL or an Object
> database? After a lot of research I've guessed that my particular case might
> be better served with Python and Zope/ZODB. But I might be wrong? Maybe
> PHP+mySQL or Django is a better fit?

First of all, find yourself a web-framework you like. Then use that.
Most likely, that framework is going to limit you to using SQL. There
are frameworks that don't specifically Pyramid integrates nicely with
ZODB and hence gives you a choice. Then I would simply try to see if
you are able to model the data in SQL at all, or if you dig yourself
into unholy complexity. That should not take more than a couple of
days of work to figure that out, if you concentrate on building a
database and filling it with real or realistic data. If you don't dig
yourself into a hole, the go with SQL, since that's what you know. If
you do, try to build the model with ZODB and see of that works better
for you.

The ZODB might very well be the right choice here. But note that Zope
is not. Zope was a trailblazer in the web framework world that's been
around for 15 years now, and has as a result made some choices which
turned out to not be the best ones in the long run, and accumulated a
lot of cruft. There is work on fixing this, but that will take time,
and the framework will be in a great flux during that time. As a
result, Zope is not currently a good choice if you start a project. As
languages go, Python rules. As web frameworks go, there are more
Python frameworks than you can shake a stick at. I'd recommend either
Django (because there are so many people using it, you will be able to
find help ) or Pyramid (because it's really cool and supports ZODB
well).

//Lennart
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )

Thank you all for your replies. I have an idea on how to begin: I'll go with mySQL and see how it works out. I will use both PHP and Python for the prototype.
My issue no longer fits within the scope of this mailing list. Without your support it would have been impossible for me to get started. Thanks!sareesh