Mailing List Archive

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Jul 16, 2013, 2:49 PM

Post #3 of 15 (16228 views)

Search for those pesky characters that MS Word inserts: m-dashes, smart quotes, and so on. I found that if those are in a template, not properly encoded, they will cause the VirtualFTP to fail like that.

Phillip.

On 2013-07-16, at 4:13 PM, Bryan Carney <bryancarney@gmail.com> wrote:

> Might anyone know what the cause of transfer failures would be via the
> virtual ftp for bricolage?
>
> It is certain specific templates files that fail and then close the FTP
> connection attempt after failing.
>
> I have tried ensuring the templates are checked in and in current /
> deployed version
>
> It will consistently happen on certain files only, from multiple clients
> and IP addresses / setups.
>
> Here is the output from an ftp client (filezilla):
>
> Command: CWD /IFEX/Utilities/util
>
> Response: 250 Changed directory OK.
>
> Command: TYPE I
>
> Response: 200 TYPE changed to I.
>
> Command: PASV
>
> Response: 227 Entering Passive Mode (208,70,244,162,254,92)
>
> Command: RETR render_html_ifex_daily.mc
>
> Response: 150 Opening BINARY mode data connection for file
> render_html_ifex_daily.mc.
>
> Error: Connection closed by server
>
> Error: File transfer failed
>
>
> Here is a successful transfer:
>
>
> Command: CWD /IFEX/Utilities/util
>
> Response: 250 Changed directory OK.
>
> Command: TYPE I
>
> Response: 200 TYPE changed to I.
>
> Command: PASV
>
> Response: 227 Entering Passive Mode (208,70,244,162,228,173)
>
> Command: RETR render_headline.mc
>
> Response: 150 Opening BINARY mode data connection for file
> render_headline.mc.
>
> Response: 226 File retrieval complete. Data connection has been closed.
>
> Status: File transfer successful, transferred 2,439 bytes in 1 second

--
Phillip Smith
http://phillipadsmith.com

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Jul 17, 2013, 3:17 AM

Post #4 of 15 (16227 views)

AWilson at RFXTechnologies

On Jul 16, 2013, at 11:49 PM, Phillip Smith <ps@phillipadsmith.com> wrote:

> Search for those pesky characters that MS Word inserts: m-dashes, smart quotes, and so on. I found that if those are in a template, not properly encoded, they will cause the VirtualFTP to fail like that.

If so, it’s either a bug, or there needs to be a better way to emit the relevant error message.

Best,

David

RE: Virtual FTP transfer fails on certain templates [ In reply to ]

Jul 17, 2013, 7:20 AM

Post #5 of 15 (16227 views)

I can confirm that having those stupid MSWORD inserts will break the VirtualFTP.
I had a template that was always failing until this morning, when I found the offending apostrophe that was a closing single quote and changed it.

Adam

-----Original Message-----
From: users@lists.bricolagecms.org [mailto:users@lists.bricolagecms.org] On Behalf Of David E. Wheeler
Sent: Wednesday, July 17, 2013 6:17 AM
To: users@lists.bricolagecms.org
Subject: Re: Virtual FTP transfer fails on certain templates

On Jul 16, 2013, at 11:49 PM, Phillip Smith <ps@phillipadsmith.com> wrote:

> Search for those pesky characters that MS Word inserts: m-dashes, smart quotes, and so on. I found that if those are in a template, not properly encoded, they will cause the VirtualFTP to fail like that.

If so, it's either a bug, or there needs to be a better way to emit the relevant error message.

Best,

David

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

bryan at bryancarney

Jul 17, 2013, 8:38 AM

Post #6 of 15 (16232 views)

Yayy thanks all --

I hunted and used a few scripts before this one seemed to work:

http://www.nousphere.net/cleanspecial.php

I'll do a diff before & after and see if I can find the offending
characters -- I had already tried smart quote replacement.

On Wed, Jul 17, 2013 at 10:20 AM, Adam Wilson
<AWilson@rfxtechnologies.com>wrote:

> I can confirm that having those stupid MSWORD inserts will break the
> VirtualFTP.
> I had a template that was always failing until this morning, when I found
> the offending apostrophe that was a closing single quote and changed it.
>
> Adam
>
> -----Original Message-----
> From: users@lists.bricolagecms.org [mailto:users@lists.bricolagecms.org]
> On Behalf Of David E. Wheeler
> Sent: Wednesday, July 17, 2013 6:17 AM
> To: users@lists.bricolagecms.org
> Subject: Re: Virtual FTP transfer fails on certain templates
>
> On Jul 16, 2013, at 11:49 PM, Phillip Smith <ps@phillipadsmith.com> wrote:
>
> > Search for those pesky characters that MS Word inserts: m-dashes, smart
> quotes, and so on. I found that if those are in a template, not properly
> encoded, they will cause the VirtualFTP to fail like that.
>
> If so, it's either a bug, or there needs to be a better way to emit the
> relevant error message.
>
> Best,
>
> David
>
>

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

bryancarney at gmail

Jul 17, 2013, 8:38 AM

Post #7 of 15 (16229 views)

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Jul 17, 2013, 8:47 AM

Post #8 of 15 (16245 views)

On Jul 17, 2013, at 5:38 PM, Bryan Carney <bryancarney@gmail.com> wrote:

> Yayy thanks all --
>
> I hunted and used a few scripts before this one seemed to work:
>
> http://www.nousphere.net/cleanspecial.php
>
> I'll do a diff before & after and see if I can find the offending
> characters -- I had already tried smart quote replacement.

You can also use Encode::ZapCP1252::fix_cp1252() to change CP1252 characters to UTF-8.

https://metacpan.org/module/Encode::ZapCP1252

Best,

David

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Oct 6, 2013, 2:06 PM

Post #9 of 15 (15854 views)

On 2013-07-17, at 5:17 AM, David E. Wheeler <david@kineticode.com> wrote:

> On Jul 16, 2013, at 11:49 PM, Phillip Smith <ps@phillipadsmith.com> wrote:
>
>> Search for those pesky characters that MS Word inserts: m-dashes, smart quotes, and so on. I found that if those are in a template, not properly encoded, they will cause the VirtualFTP to fail like that.
>
> If so, it’s either a bug, or there needs to be a better way to emit the relevant error message.

Any leads on where to start hunting this down / fixing the problem? Thinking Bric::Util::FTP and/or Bric::Util::FTP::FileHandle ?

Ideally, the file save operation would be refused in a more elegant manner if the template contained the offending characters, or the offending characters would be replaced before the file save operation?

Phillip.

--
Phillip Smith
http://phillipadsmith.com
http://twitter.com/phillipadsmith
http://linkedin.com/in/phillipadsmith

Oaxaca: (011 52 1) 951 154 4620
Toronto: (647) 361-8248
Toll-free: (888) 418-1868

If your email inbox is out of control, check out http://sanebox.com/t/s0q7m

Save our in-boxes! http://emailcharter.org

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Oct 7, 2013, 10:29 AM

Post #10 of 15 (15841 views)

On Oct 6, 2013, at 2:06 PM, Phillip Smith <ps@phillipadsmith.com> wrote:

>> If so, it’s either a bug, or there needs to be a better way to emit the relevant error message.
>
> Any leads on where to start hunting this down / fixing the problem? Thinking Bric::Util::FTP and/or Bric::Util::FTP::FileHandle ?

Sounds right, yes.

> Ideally, the file save operation would be refused in a more elegant manner if the template contained the offending characters, or the offending characters would be replaced before the file save operation?

What’s the original error here? The context is missing from this thread in my mail reader.

D

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Oct 8, 2013, 5:36 PM

Post #11 of 15 (15837 views)

On 2013-10-07, at 12:29 PM, David E. Wheeler <david@kineticode.com> wrote:

> On Oct 6, 2013, at 2:06 PM, Phillip Smith <ps@phillipadsmith.com> wrote:
>
>>> If so, it’s either a bug, or there needs to be a better way to emit the relevant error message.
>>
>> Any leads on where to start hunting this down / fixing the problem? Thinking Bric::Util::FTP and/or Bric::Util::FTP::FileHandle ?
>
> Sounds right, yes.
>
>> Ideally, the file save operation would be refused in a more elegant manner if the template contained the offending characters, or the offending characters would be replaced before the file save operation?
>
> What’s the original error here? The context is missing from this thread in my mail reader.

http://www.gossamer-threads.com/lists/bricolage/users/40933

Basically, if I'm editing a template and some text/copy in the template has, say, a smart quote or what-have-you via MS Word or its ilk, the virtual FTP server simply closes the connection. If you remove the offending character, all is good again.

--
Phillip Smith
http://phillipadsmith.com

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Oct 9, 2013, 1:30 PM

Post #12 of 15 (15831 views)

On Oct 8, 2013, at 5:36 PM, Phillip Smith <ps@phillipadsmith.com> wrote:

>>> Ideally, the file save operation would be refused in a more elegant manner if the template contained the offending characters, or the offending characters would be replaced before the file save operation?
>>
>> What’s the original error here? The context is missing from this thread in my mail reader.
>
> http://www.gossamer-threads.com/lists/bricolage/users/40933
>
> Basically, if I'm editing a template and some text/copy in the template has, say, a smart quote or what-have-you via MS Word or its ilk, the virtual FTP server simply closes the connection. If you remove the offending character, all is good again.

Oh. In that case, I would say that it should try to clean up the file. It look as though Bricolage currently assumes that the file will be submitted in the user’s preferred encoding:

# Encode and save the new code
Bric::Util::CharTrans->new($user->get_pref('Character Set'))->to_utf8($sref)
if ENCODE_OK;

So there will be an error for any encoding disagreements. If your preferred encoding is ASCII and you submit Big5, you could get an error.

So this code should probably be wrapped in an exception handler that returns the error to the client. In fact, it should throw a regular Bric exception, which the virtual FTP server should return to the client before closing the connection. I mean, what happens if you try to update a template checked-out by someone else, try to deploy one you don't have permission to deploy? In all these cases, there should be an error message returned to the client. Is there? Is an encoding error special, or do we return insufficient error information altogether?

Best,

David

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Oct 9, 2013, 1:32 PM

Post #13 of 15 (15838 views)

On 2013-10-09, at 3:30 PM, "David E. Wheeler" <david@kineticode.com> wrote:

> On Oct 8, 2013, at 5:36 PM, Phillip Smith <ps@phillipadsmith.com> wrote:
>
>>>> Ideally, the file save operation would be refused in a more elegant manner if the template contained the offending characters, or the offending characters would be replaced before the file save operation?
>>>
>>> What’s the original error here? The context is missing from this thread in my mail reader.
>>
>> http://www.gossamer-threads.com/lists/bricolage/users/40933
>>
>> Basically, if I'm editing a template and some text/copy in the template has, say, a smart quote or what-have-you via MS Word or its ilk, the virtual FTP server simply closes the connection. If you remove the offending character, all is good again.
>
> Oh. In that case, I would say that it should try to clean up the file. It look as though Bricolage currently assumes that the file will be submitted in the user’s preferred encoding:
>
> # Encode and save the new code
> Bric::Util::CharTrans->new($user->get_pref('Character Set'))->to_utf8($sref)
> if ENCODE_OK;
>
> So there will be an error for any encoding disagreements. If your preferred encoding is ASCII and you submit Big5, you could get an error.
>
> So this code should probably be wrapped in an exception handler that returns the error to the client. In fact, it should throw a regular Bric exception, which the virtual FTP server should return to the client before closing the connection. I mean, what happens if you try to update a template checked-out by someone else, try to deploy one you don't have permission to deploy? In all these cases, there should be an error message returned to the client. Is there? Is an encoding error special, or do we return insufficient error information altogether?

Let me test those scenarios. I'm just trying to get the bric_ftp running on my local copy of Bricolage at the moment…

--
Phillip Smith
http://phillipadsmith.com

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

bryancarney at gmail

Oct 9, 2013, 1:50 PM

Post #14 of 15 (15838 views)

I usually have at least one or two templates on the go which I have to copy contents from the web editor, then replace (from ftp) before bricolage will let me download subsequent revisions and I haven't been able to narrow the cases down to smart quotes.

This script substitutes the ms word special characters that I believe has still failed sometimes … in case that helps.

http://www.nousphere.net/cleanspecial.php

Manually setting encoding in whatever editor doesn't seem to make a difference for me either.

On Oct 9, 2013, at 4:32 PM, Phillip Smith <ps@phillipadsmith.com> wrote:

>
> On 2013-10-09, at 3:30 PM, "David E. Wheeler" <david@kineticode.com> wrote:
>
>> On Oct 8, 2013, at 5:36 PM, Phillip Smith <ps@phillipadsmith.com> wrote:
>>
>>>>> Ideally, the file save operation would be refused in a more elegant manner if the template contained the offending characters, or the offending characters would be replaced before the file save operation?
>>>>
>>>> What’s the original error here? The context is missing from this thread in my mail reader.
>>>
>>> http://www.gossamer-threads.com/lists/bricolage/users/40933
>>>
>>> Basically, if I'm editing a template and some text/copy in the template has, say, a smart quote or what-have-you via MS Word or its ilk, the virtual FTP server simply closes the connection. If you remove the offending character, all is good again.
>>
>> Oh. In that case, I would say that it should try to clean up the file. It look as though Bricolage currently assumes that the file will be submitted in the user’s preferred encoding:
>>
>> # Encode and save the new code
>> Bric::Util::CharTrans->new($user->get_pref('Character Set'))->to_utf8($sref)
>> if ENCODE_OK;
>>
>> So there will be an error for any encoding disagreements. If your preferred encoding is ASCII and you submit Big5, you could get an error.
>>
>> So this code should probably be wrapped in an exception handler that returns the error to the client. In fact, it should throw a regular Bric exception, which the virtual FTP server should return to the client before closing the connection. I mean, what happens if you try to update a template checked-out by someone else, try to deploy one you don't have permission to deploy? In all these cases, there should be an error message returned to the client. Is there? Is an encoding error special, or do we return insufficient error information altogether?
>
> Let me test those scenarios. I'm just trying to get the bric_ftp running on my local copy of Bricolage at the moment…
>
> --
> Phillip Smith
> http://phillipadsmith.com
>

Re: Virtual FTP transfer fails on certain templates [ In reply to ]

Oct 9, 2013, 4:18 PM

Post #15 of 15 (15834 views)