Mailing List Archive: Media naming weirdness

Media naming weirdness

fulekia at denison

Oct 15, 2009, 7:56 AM

Post #1 of 8 (2572 views)

We were excited to see that bug #1419 was fixed in 1.10.7, as
inconsistent media filenames have been real pain. I've noticed some
weird things though, and I'm not sure what to do, and what is/isn't a
bug.

First, I uploaded an image named "freddie on pony.JPG" to bricolage.
On preview or publish, everything is lowercased as per the output
channel settings, but the spaces are URL encoded in the actual
filename written to disk ("freddie%20on%20pony.jpg"), which makes
Apache 404 because somewhere in the request process the %20's are
decoded back to whitespace. The only way Apache will actually serve
the file is to double-escape ("freddie%2520on%2520pony.jpg"). Is this
a bug, or do we need to change all templates to double URI escape
everything? I'm hoping the former...

Another thing: expire jobs for media that were created and previewed/
published pre-1.10.7 are failing if the filenames are now different
with the new behavior. Sometimes the files were actually removed from
the destination, sometimes not. The job failure notes just say this:
-------------------------------------------------------
No parameters passed to ANY()
[/usr/local/bricolage/lib/Bric/Util/DBI.pm:323]
[/usr/local/bricolage/lib/Bric/Util/Burner.pm:1311]
[/usr/local/bricolage/lib/Bric/Util/Job/Pub.pm:201]
[/usr/local/bricolage/lib/Bric/Util/Job.pm:1889]
[/usr/local/bricolage/bin/bric_queued:244]
[/usr/local/bricolage/bin/bric_queued:213]
-------------------------------------------------------

I've poked around the media_instance table a bit, and they all seem to
be files that had spaces or uppercase extensions, and some versions
are getting URI escaped and lowercased, and some aren't. Weird.

Anybody seen this with 1.10.7?

-Aaron

---------------------------------
Aaron Fuleki
Senior Web Architect
Denison University
740.587.5752
---------------------------------

Re: Media naming weirdness [ In reply to ]

david at kineticode

Oct 15, 2009, 12:06 PM

Post #2 of 8 (2500 views)

On Oct 15, 2009, at 7:56 AM, Aaron Fuleki wrote:

> First, I uploaded an image named "freddie on pony.JPG" to
> bricolage. On preview or publish, everything is lowercased as per
> the output channel settings, but the spaces are URL encoded in the
> actual filename written to disk ("freddie%20on%20pony.jpg"), which
> makes Apache 404 because somewhere in the request process the %20's
> are decoded back to whitespace. The only way Apache will actually
> serve the file is to double-escape ("freddie%2520on%2520pony.jpg").
> Is this a bug, or do we need to change all templates to double URI
> escape everything? I'm hoping the former...

It's a bug.

> Another thing: expire jobs for media that were created and previewed/
> published pre-1.10.7 are failing if the filenames are now different
> with the new behavior. Sometimes the files were actually removed
> from the destination, sometimes not. The job failure notes just say
> this:
> -------------------------------------------------------
> No parameters passed to ANY()
> [/usr/local/bricolage/lib/Bric/Util/DBI.pm:323]
> [/usr/local/bricolage/lib/Bric/Util/Burner.pm:1311]
> [/usr/local/bricolage/lib/Bric/Util/Job/Pub.pm:201]
> [/usr/local/bricolage/lib/Bric/Util/Job.pm:1889]
> [/usr/local/bricolage/bin/bric_queued:244]
> [/usr/local/bricolage/bin/bric_queued:213]
> -------------------------------------------------------
>
> I've poked around the media_instance table a bit, and they all seem
> to be files that had spaces or uppercase extensions, and some
> versions are getting URI escaped and lowercased, and some aren't.
> Weird.

This is because the bug was fixed, but existing assets weren't
updated. You'll need to fix them manually, I'm afraid.

Best,

David

Re: Media naming weirdness [ In reply to ]

fulekia at denison

Oct 15, 2009, 12:20 PM

Post #3 of 8 (2491 views)

> It's a bug.

I'll poke around lighthouse and add that if necessary - I have a small
pile of bugs to enter already. I am concerned that this one's in the
wild now, and our users may start breaking media.

> This is because the bug was fixed, but existing assets weren't
> updated. You'll need to fix them manually, I'm afraid.

I thought you were going to say that :-) Would a bulk republish fix
it? Republish media to get new filenames, republish stories to update
links and img/object tags?

-Aaron

---------------------------------
Aaron Fuleki
Senior Web Architect
Denison University
740.587.5752
---------------------------------

Re: Media naming weirdness [ In reply to ]

david at kineticode

Oct 15, 2009, 12:35 PM

Post #4 of 8 (2505 views)

On Oct 15, 2009, at 12:20 PM, Aaron Fuleki wrote:

> I thought you were going to say that :-) Would a bulk republish fix
> it? Republish media to get new filenames, republish stories to
> update links and img/object tags?

No, I think the files need to be re-uploaded to each story. I suggest
removing the spaces.

Best,

David

Re: Media naming weirdness [ In reply to ]

fulekia at denison

Oct 15, 2009, 12:43 PM

Post #5 of 8 (2498 views)

> No, I think the files need to be re-uploaded to each story. I
> suggest removing the spaces.

Could that be done by SOAP exporting/importing the media, or a script
to update the files and media_instance records? We have 1038 media
assets with spaces in their names :-(

-Aaron

---------------------------------
Aaron Fuleki
Senior Web Architect
Denison University
740.587.5752
---------------------------------

Re: Media naming weirdness [ In reply to ]

david at kineticode

Oct 15, 2009, 12:47 PM

Post #6 of 8 (2505 views)

On Oct 15, 2009, at 12:43 PM, Aaron Fuleki wrote:

> Could that be done by SOAP exporting/importing the media, or a
> script to update the files and media_instance records? We have 1038
> media assets with spaces in their names :-(

Yes. YOu'd need to write a script to export them all, change the file
name, and update. Shouldn't be too hard, but it'll take a while for
that many documents.

Best

David

Re: Media naming weirdness [ In reply to ]

fulekia at denison

Oct 15, 2009, 1:11 PM

Post #7 of 8 (2504 views)

>> First, I uploaded an image named "freddie on pony.JPG" to
>> bricolage. On preview or publish, everything is lowercased as per
>> the output channel settings, but the spaces are URL encoded in the
>> actual filename written to disk ("freddie%20on%20pony.jpg"), which
>> makes Apache 404 because somewhere in the request process the %20's
>> are decoded back to whitespace. The only way Apache will actually
>> serve the file is to double-escape ("freddie%2520on
>> %2520pony.jpg"). Is this a bug, or do we need to change all
>> templates to double URI escape everything? I'm hoping the former...
>
> It's a bug.

Is it this?
http://bricolage.lighthouseapp.com/projects/29601/tickets/82-uploading-media-with-spaces-in-the-name-fails-to-preview

It's not just previewing under 1.10.7. I have a student testing more
combinations to get better STR.

-Aaron

---------------------------------
Aaron Fuleki
Senior Web Architect
Denison University
740.587.5752
---------------------------------

Re: Media naming weirdness [ In reply to ]

david at kineticode

Oct 15, 2009, 11:38 PM

Post #8 of 8 (2501 views)

On Oct 15, 2009, at 1:11 PM, Aaron Fuleki wrote:

> Is it this?
> http://bricolage.lighthouseapp.com/projects/29601/tickets/82-uploading-media-with-spaces-in-the-name-fails-to-preview

Looks like it.

> It's not just previewing under 1.10.7. I have a student testing
> more combinations to get better STR.

Thanks.

David