Mailing List Archive

Lots of categories/OCs make publishing slow
Hi everybody,

On the IFEX site, there are lots and lots of cover stories: one for
every country, plus a bunch for multi-country regions.

We use categories for countries. So here are some categories:

/united_states/
/haiti/

...you get the picture.


The multi-country region cover stories are in lots of categories. For
example, the "North America" cover is in these categories:

/canada/
/mexico/
/united_states/

The "Africa" cover is in many dozens of categories.

In addition, each cover is associated with lots of output channels,
mostly for languages. For example:

Web
Web (French)
Web (Spanish)
Web (Arabic)
RSS
RSS (French)
RSS (Spanish)
RSS (Arabic)

Anyway, if a cover has 50 categories and eight output channels,
Bricolage wants to create 400 files and distribute them all. We, on the
other hand, only want to create files for the primary categories, which
would mean eight files, rather than 400.

We originally dealt with this by putting a line at the start of the
<%init> block in the autohandler:

$m->clear_and_abort unless ($burner->get_cat->get_id ==
$story->get_primary_category->get_id);


This made things very slightly faster, but we were still dealing with a
long, long wait to publish any of the big covers.

Last week, I changed Bric::Util::Burner to only bother with the primary
categories of stories, and suddenly we're super quick.

It got me thinking: Has anybody else run into this? Is there any
appetite for a config directive that would allow story publishing to be
limited to primary categories?

(I know that there's consensus that there are already too many config
directives. Sorry.)


Cheers,

Bret

--
Bret Dawson
Producer
Pectopah Productions Inc.
(416) 895-7635
bret@pectopah.com
www.pectopah.com
Re: Lots of categories/OCs make publishing slow [ In reply to ]
On Jan 26, 2010, at 11:19 AM, Bret Dawson wrote:

> Anyway, if a cover has 50 categories and eight output channels,
> Bricolage wants to create 400 files and distribute them all. We, on the
> other hand, only want to create files for the primary categories, which
> would mean eight files, rather than 400.
>
> We originally dealt with this by putting a line at the start of the
> <%init> block in the autohandler:
>
> $m->clear_and_abort unless ($burner->get_cat->get_id ==
> $story->get_primary_category->get_id);

That should indeed give you only the 8 files you want.

> This made things very slightly faster, but we were still dealing with a
> long, long wait to publish any of the big covers.
>
> Last week, I changed Bric::Util::Burner to only bother with the primary
> categories of stories, and suddenly we're super quick.

Huh. Were you getting only 8 files before you made that change?

> It got me thinking: Has anybody else run into this? Is there any
> appetite for a config directive that would allow story publishing to be
> limited to primary categories?

Yes.

> (I know that there's consensus that there are already too many config
> directives. Sorry.)

Ah, fuck it.

David
Re: Lots of categories/OCs make publishing slow [ In reply to ]
On 26-Jan-10, at 8:14 PM, David E. Wheeler wrote:

>> It got me thinking: Has anybody else run into this? Is there any
>> appetite for a config directive that would allow story publishing
>> to be
>> limited to primary categories?
>
> Yes.
>
>> (I know that there's consensus that there are already too many config
>> directives. Sorry.)
>
> Ah, fuck it.

+1

Running into similar publish time issues and it would be great to be
able to turn-off secondary category publishing globally.

--
Phillip Smith // Simplifier of Technology // COMMUNITY BANDWIDTH
www.communitybandwidth.ca // www.phillipadsmith.com