Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Zope>
  3. Users
ZCatalog queries performance
st at taenzer
May 17, 2015, 8:09 AM
Post #1 of 5 (2027 views)
Permalink
Hello,

I’m querying a ZCatalog with a lot of indexes and some meta data for event objects to get related events by tags.
These need to be displayed with a title image (which itself is a sub object of the event object) and a pre-defined short text.

My python query script gets the results, I then loop over them and get the object with unrestrictedTraverse(x.getPath(), None).
I understood getObject is not really performant, but getting the objects with unrestrictedTraverse is not a lot faster.

What would be the best way to query for these objects and then display the results with the image?
Should I add the url of the image as meta data (title_image etc.)?

What is the best way to display search results?
Is getting the whole object via unrestrictedTraverse() a good idea?
What are the alternatives?

Best,
Sebastian
_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: ZCatalog queries performance [ In reply to ]
tseaver at palladion
May 17, 2015, 10:37 AM
Post #2 of 5 (1985 views)
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/17/2015 11:09 AM, Sebastian Tänzer wrote:
> What are the alternatives?

The most performant implementation would be to generate the URL for the
image using a fixed suffix (relative to the result object). E.g., ::

<ul>
<li tal:repeat="event context/listEvents">
<a href="event/getUrl">
<img src="string:{event/getUrl}/event_image" />
<span tal:content="event/title">TITLE</span>
</a>
</li>
</ul>

You could achieve that fixed suffix either by mandating a fixed object
ID for the image, or by adding a method / view to the event which
redirected to the actual image URL.



Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBAgAGBQJVWNHiAAoJEPKpaDSJE9HY/nYQALbHx3WvyjEk/6rsgomcWJhN
va4ZgWEjQtFuSlzHOf30CGKIEwNX+EAYWw6juMznQo/4Yq5b2g4Ur8cA9q1gYgFB
2ZwW4/W4wKsaAJxQ6sckinAKhlWGJugXdpcxnbbU3V0ePoUtgPsO0GVIlPoDV+WF
/qEnWYWTu+7xHf/DV5iiHKTqYM0Oqv1aNTuZd9s8/emHoDuU8RpzVLscZuJEYk+B
5mdFFSfNJ9wer79fxQH8yM+8Vz6IkVx4XCCTa9/9QM9qHTVLqvv+AllbtolQ4gms
ASkisuW7p/XvJgK5bMbUFvEkDZQQDMFAuRKe2lnVT6rAFXiR04fHk6GKKtW3WyEb
Cdq4GYcKz4ZYdF4l2QV8h+FOZRAf6qWW9CCiRsuihaiYB57Uk8hI/GPk00TXL78z
rmOWtZ86d7hlCarP8k+T+1b5b+RQUB8danop21ys3SeH+FMiEPxjIuhY0i43n6xQ
dg6HJ8ry7KihYipaRJrC0tqVFluToWFdLUaM1qRSJHpcKi3TpHKiKXGzNSHzeI1y
GD7zb3HF+lYqsy74wPWIp/flSmJxJMg5givfHt96/sFvaaNstvjIY6KiKUUW0JMy
8b/fF4ufHu2ZGoT8TSMCZRa4KpNMcat9Edo/MsgOeir65a3PnFj9Br8DmpZJwTbV
9KZEKY4Aq+EJq+RPmH2D
=U40Y
-----END PGP SIGNATURE-----

_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: ZCatalog queries performance [ In reply to ]
st at taenzer
May 19, 2015, 1:54 AM
Post #3 of 5 (1980 views)
Permalink
Thanks for the feedback and help, Tres.

Let’s say I want to display an accordion with all events of a month displaying quite a lot of data about the event (30-50 events on one page, collapsible jQuery accordion). At the moment I get the results and put the objects into the result set via unrestrictedTraverse using getPath(). Would it be better to only get the path and get the object in the template? Does this make any difference performance wise?

Which data of the event objects would I keep as ZCatalog meta columns and which not? Documentation is not really helpful on this.

The tricky part is that events are linked with different locations (event location, selling points) that are not children of the event object but objects in a different folder. The event object itself only keeps the links to these location objects and I get them when iterating over the result set in the template.

Any advice what would be the best way for this performance-wise?
Is it okay to get the object itself via traversal? Is there a faster way to get the object data?

> Am 17.05.2015 um 19:37 schrieb Tres Seaver <tseaver@palladion.com>:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 05/17/2015 11:09 AM, Sebastian Tänzer wrote:
>> What are the alternatives?
>
> The most performant implementation would be to generate the URL for the
> image using a fixed suffix (relative to the result object). E.g., ::
>
> <ul>
> <li tal:repeat="event context/listEvents">
> <a href="event/getUrl">
> <img src="string:{event/getUrl}/event_image" />
> <span tal:content="event/title">TITLE</span>
> </a>
> </li>
> </ul>
>
> You could achieve that fixed suffix either by mandating a fixed object
> ID for the image, or by adding a method / view to the event which
> redirected to the actual image URL.
>
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver +1 540-429-0999 tseaver@palladion.com
> Palladion Software "Excellence by Design" http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
>
> iQIcBAEBAgAGBQJVWNHiAAoJEPKpaDSJE9HY/nYQALbHx3WvyjEk/6rsgomcWJhN
> va4ZgWEjQtFuSlzHOf30CGKIEwNX+EAYWw6juMznQo/4Yq5b2g4Ur8cA9q1gYgFB
> 2ZwW4/W4wKsaAJxQ6sckinAKhlWGJugXdpcxnbbU3V0ePoUtgPsO0GVIlPoDV+WF
> /qEnWYWTu+7xHf/DV5iiHKTqYM0Oqv1aNTuZd9s8/emHoDuU8RpzVLscZuJEYk+B
> 5mdFFSfNJ9wer79fxQH8yM+8Vz6IkVx4XCCTa9/9QM9qHTVLqvv+AllbtolQ4gms
> ASkisuW7p/XvJgK5bMbUFvEkDZQQDMFAuRKe2lnVT6rAFXiR04fHk6GKKtW3WyEb
> Cdq4GYcKz4ZYdF4l2QV8h+FOZRAf6qWW9CCiRsuihaiYB57Uk8hI/GPk00TXL78z
> rmOWtZ86d7hlCarP8k+T+1b5b+RQUB8danop21ys3SeH+FMiEPxjIuhY0i43n6xQ
> dg6HJ8ry7KihYipaRJrC0tqVFluToWFdLUaM1qRSJHpcKi3TpHKiKXGzNSHzeI1y
> GD7zb3HF+lYqsy74wPWIp/flSmJxJMg5givfHt96/sFvaaNstvjIY6KiKUUW0JMy
> 8b/fF4ufHu2ZGoT8TSMCZRa4KpNMcat9Edo/MsgOeir65a3PnFj9Br8DmpZJwTbV
> 9KZEKY4Aq+EJq+RPmH2D
> =U40Y
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> https://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> https://mail.zope.org/mailman/listinfo/zope-announce
> https://mail.zope.org/mailman/listinfo/zope-dev )


_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: ZCatalog queries performance [ In reply to ]
tseaver at palladion
May 19, 2015, 3:50 AM
Post #4 of 5 (1972 views)
Permalink
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/19/2015 04:54 AM, Sebastian Tänzer wrote:

> Let’s say I want to display an accordion with all events of a month
> displaying quite a lot of data about the event (30-50 events on one
> page, collapsible jQuery accordion). At the moment I get the results
> and put the objects into the result set via unrestrictedTraverse
> using getPath(). Would it be better to only get the path and get the
> object in the template? Does this make any difference performance
> wise?

Nope. Doing it in trusted code (an actual method, and ExternalMethod, or
a view) would be faster than either a script or a template. Measure it
to decide whether to switch, however.

> Which data of the event objects would I keep as ZCatalog meta columns
> and which not? Documentation is not really helpful on this.

It is super application-dependent. The costs for maintaining "extra"
columns of metaata are relatively small (bloating the Record object, and
therefore its pickle, mostly).

> The tricky part is that events are linked with different locations
> (event location, selling points) that are not children of the event
> object but objects in a different folder. The event object itself
> only keeps the links to these location objects and I get them when
> iterating over the result set in the template.

If you can define a URL-addressible method of the event which returns or
redirects to those objects, you can get away with hard-wiring that in
your template, using 'brain.getPath()' as a prefix.

> Any advice what would be the best way for this performance-wise? Is
> it okay to get the object itself via traversal? Is there a faster way
> to get the object data?

Fetching the object is fast if it is already in the ZODB cache, but slow
if not, nd may cause other, more "popular" / important objects to be
evicted. Where possible, design for catalog queries which don't need to
fetch the object.



Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tseaver@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQIcBAEBAgAGBQJVWxWLAAoJEPKpaDSJE9HY0aQP/1mC0Ll5//c24XQPcvMwS18Y
/gZfOe/IeL61CRrI2Sv+RMQ6N6gXpL8oM/KsWU2wphfPVYC96VxiD28ubhmWVk+f
GFbmN4YchYSNHND2kzcjxyHd1+U6+c/NhkcfgL9GtLZBsSNtMZut2pV7sN7VvjhJ
0F1Kwu6HxZ5yvEjZOk0tzDfrhh0I4Y3elFTwJkHXq/Z/3xN+NTVNpYXRAw5AWGJC
A0Y/ll/cx1+JIKlMbjhMNqG+k61JGD8MdZXIvBwJ7jij1VIGYfYG5/7wZieMjCTL
/SBMkOnODchSzSk2wV1B0ZgpbgTf+DYuZRzKCrYc7QOktfQhqEBOS5ljIjpWkDcq
wqsxPBD0uyu/K3+dd4egL3Fcvg5DDfm9//Qo2B8JAkLWO5XeJ4UiyZHLYWZXSpXI
SdjL2zFnhTZhp9pxIv+V4W/XrzMEDAr3UOFlrikiGd4oxTBffFuxuSr2mvpqshTQ
TaS8Vgty9SmqMaPuvMZPc7FoPExsC4k4sNO9XgSXKT/5xo3FErLTWVCL1/qVHN3I
6wHvSrh8n1Z/m60P6OaUzIHF/97sjbxOO75d0RZCnaTFWt2TizrrsrwaBZ2p6Yyo
83l5QxPLDge6EZNb1MMewZ9vW8EamDhkR2pKTsQTEVDj2p4gVhmh+J5pjN6BKj50
D98HPv4JrWhf965/Pay4
=Sv38
-----END PGP SIGNATURE-----

_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: ZCatalog queries performance [ In reply to ]
st at taenzer
May 19, 2015, 3:57 AM
Post #5 of 5 (1980 views)
Permalink
Thanks, this answered all of my questions :-)

Best, Sebastian

> Am 19.05.2015 um 12:50 schrieb Tres Seaver <tseaver@palladion.com>:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>> On 05/19/2015 04:54 AM, Sebastian Tänzer wrote:
>>
>> Let’s say I want to display an accordion with all events of a month
>> displaying quite a lot of data about the event (30-50 events on one
>> page, collapsible jQuery accordion). At the moment I get the results
>> and put the objects into the result set via unrestrictedTraverse
>> using getPath(). Would it be better to only get the path and get the
>> object in the template? Does this make any difference performance
>> wise?
>
> Nope. Doing it in trusted code (an actual method, and ExternalMethod, or
> a view) would be faster than either a script or a template. Measure it
> to decide whether to switch, however.
>
>> Which data of the event objects would I keep as ZCatalog meta columns
>> and which not? Documentation is not really helpful on this.
>
> It is super application-dependent. The costs for maintaining "extra"
> columns of metaata are relatively small (bloating the Record object, and
> therefore its pickle, mostly).
>
>> The tricky part is that events are linked with different locations
>> (event location, selling points) that are not children of the event
>> object but objects in a different folder. The event object itself
>> only keeps the links to these location objects and I get them when
>> iterating over the result set in the template.
>
> If you can define a URL-addressible method of the event which returns or
> redirects to those objects, you can get away with hard-wiring that in
> your template, using 'brain.getPath()' as a prefix.
>
>> Any advice what would be the best way for this performance-wise? Is
>> it okay to get the object itself via traversal? Is there a faster way
>> to get the object data?
>
> Fetching the object is fast if it is already in the ZODB cache, but slow
> if not, nd may cause other, more "popular" / important objects to be
> evicted. Where possible, design for catalog queries which don't need to
> fetch the object.
>
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver +1 540-429-0999 tseaver@palladion.com
> Palladion Software "Excellence by Design" http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
>
> iQIcBAEBAgAGBQJVWxWLAAoJEPKpaDSJE9HY0aQP/1mC0Ll5//c24XQPcvMwS18Y
> /gZfOe/IeL61CRrI2Sv+RMQ6N6gXpL8oM/KsWU2wphfPVYC96VxiD28ubhmWVk+f
> GFbmN4YchYSNHND2kzcjxyHd1+U6+c/NhkcfgL9GtLZBsSNtMZut2pV7sN7VvjhJ
> 0F1Kwu6HxZ5yvEjZOk0tzDfrhh0I4Y3elFTwJkHXq/Z/3xN+NTVNpYXRAw5AWGJC
> A0Y/ll/cx1+JIKlMbjhMNqG+k61JGD8MdZXIvBwJ7jij1VIGYfYG5/7wZieMjCTL
> /SBMkOnODchSzSk2wV1B0ZgpbgTf+DYuZRzKCrYc7QOktfQhqEBOS5ljIjpWkDcq
> wqsxPBD0uyu/K3+dd4egL3Fcvg5DDfm9//Qo2B8JAkLWO5XeJ4UiyZHLYWZXSpXI
> SdjL2zFnhTZhp9pxIv+V4W/XrzMEDAr3UOFlrikiGd4oxTBffFuxuSr2mvpqshTQ
> TaS8Vgty9SmqMaPuvMZPc7FoPExsC4k4sNO9XgSXKT/5xo3FErLTWVCL1/qVHN3I
> 6wHvSrh8n1Z/m60P6OaUzIHF/97sjbxOO75d0RZCnaTFWt2TizrrsrwaBZ2p6Yyo
> 83l5QxPLDge6EZNb1MMewZ9vW8EamDhkR2pKTsQTEVDj2p4gVhmh+J5pjN6BKj50
> D98HPv4JrWhf965/Pay4
> =Sv38
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Zope maillist - Zope@zope.org
> https://mail.zope.org/mailman/listinfo/zope
> ** No cross posts or HTML encoding! **
> (Related lists -
> https://mail.zope.org/mailman/listinfo/zope-announce
> https://mail.zope.org/mailman/listinfo/zope-dev )

_______________________________________________
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal