Mailing List Archive

[issue32912] Raise non-silent warning for invalid escape sequences
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

> In other words, it seems to me that getting in the way of
> this broken end-user strategy is a *good* thing, since it
> warns of possible mistakes.

Here's an example from the current version of Bottle (0.12.17):

/Users/raymond/Dropbox/Public/sj205/notes2/bottle.py:3433: SyntaxWarning: invalid escape sequence \[
_re_tok += '|([\[\{\(])'
/Users/raymond/Dropbox/Public/sj205/notes2/bottle.py:3434: SyntaxWarning: invalid escape sequence \]
_re_tok += '|([\]\}\)])

These warnings would spew out during a recent Python course where we used 3.8b2. It mae it difficult to teach building templates in an iterative style. We had to abandon 3.8 and go back to 3.7 to proceed with the templating lesson.

IMO, spewing out these warnings for things users can't do anything about is a usability travesty.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

Here's another example from the current build of the docs using Sphinx:

(venv) ~/npython/Doc $ make html
mkdir -p build
Building NEWS from Misc/NEWS.d with blurb
PATH=./venv/bin:$PATH sphinx-build -b html -d build/doctrees -D latex_elements.papersize= -W . build/html
Running Sphinx v2.1.0
/Users/raymond/npython/venv/lib/python3.8/site-packages/docutils/writers/latex2e/__init__.py:2978: SyntaxWarning: invalid escape sequence \l
self.out.append('}] \leavevmode ')

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

I think it is poor form to bombard end-users with warnings about things they can't fix.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment:

The warning in docutils was fixed in https://sourceforge.net/p/docutils/code/8255/ 1.5 months ago. It was an outliner, all other occurrences of \leavevmode were either with the double backslash or in raw string literals.

The Bottle code was written 5 years age (https://github.com/bottlepy/bottle/commit/8a5bfe2f70a4a5d22b93e909b207584324817760), it was a part of the raw string. But a bug was introduced when it it was backported to the 0.12 version (https://github.com/bottlepy/bottle/commit/1f6fda636a4ea88adfa4292f3c843e71b1a685d8).

----------
nosy: +Eric Wieser

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

I'm starting to see this error even for plain text strings in existing code:

>>> dunder_methods = '''

d[k] d.__getitem__(k) -> value
\--> d.__missing__(k) -> value
\-----------> raise KeyError(k)
d[k] = v d.__setitem__(k, v) -> None
del d[k] d.__delitem__(k) -> None

'''

SyntaxError: invalid escape sequence \-


These not easily suppressed messages are really annoying. We don't have to inflict this on our users. IMO, this is the least user friendly change to Python 3.8 and as far as I can tell, it is entirely unnecessary (we've lived without it for 28+ years).

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

AFAICT, I'm currently one of the only people in the world using the 3.8 beta for my work on a daily basis. I've encountered these warnings multiple times in multiple contexts. If we think other people won't experience this recurring annoyance, we're in denial.

The entire point of beta testing is to discover problems like this. However, the participants in this thread seem inclined to ignore the evidence and persist with the idea that inflicting this on users makes the language better. Respectfully, I disagree.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Aaron Meurer <asmeurer@gmail.com> added the comment:

Raymond, are you in agreement that these warnings should at some point eventually become syntax errors?

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

> Raymond, are you in agreement that these warnings should
> at some point eventually become syntax errors?

I used to think so, but after experiencing the incessant warnings, I question the value. In inactive sessions (either with the regular REPL or the ipython REPL), they are a recurring annoyance that interferes with data exploration live demos. Perhaps, this should be left for a lint or code analysis tool. Why should we intentionally break code that is currently working fine.

Another issue that I've encountered is that ASCII art becomes gets flagged. Switching to a raw string then kills the unicode escape sequences.

This isn't really a "Raymond doesn't like this" concern. I think anyone who starts using 3.8 on a daily basis for non-toy examples will constantly run into this. Possibly, it catches a real error, but most often it will just be a recurring distractor, especially when teaching Python. All instructors and runners of live demos will need to memorize exactly which characters require an escape and which don't -- it creates a new burden that didn't exist before. And if the source of the problem is in an external library, the result is unactionable by the user, merely making their experience unpleasant.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Aaron Meurer <asmeurer@gmail.com> added the comment:

Well paradoxically, the bugs that this prevents are the ones it doesn't warn about. If someone writes '\tan(x)' thinking it is a string representing a LaTeX formula for the tangent of x, they won't realize that they actually created a string with a tab plus "an(x)". So actually I would argue that the end goal *is* to make people aware of which escape characters exist, or at the very least, always make strings raw if there's even the remotest chance they will contain a backslash character.

Is it the best way to go about this? I don't know. The whole thing sort of makes me think raw strings should have been the default, but it's obviously too late to change that.

I personally don't feel strongly about the warnings being enabled by default or not. My big gripe is that if you actually want the warnings they are difficult to get in a reproducible way. I'm actually surprised they are so annoying for you. Once a py file is compiled into a pyc file the warnings completely disappear, even if you want them!

The fact that you can't use a real escape sequence in a raw string is annoying but not the end of the world given that it's trivial to concatenate strings.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment:

> Another issue that I've encountered is that ASCII art becomes gets flagged. Switching to a raw string then kills the unicode escape sequences.

If you already use escape sequences in your ASCII art, what is the problem of using them for backslashes?

> I think anyone who starts using 3.8 on a daily basis for non-toy examples will constantly run into this.

What is a better solution? The deprecation warning was emitted starting from 3.6. Deprecation warnings were silent by default in 3.6, but they become more visible in 3.7. This helped some projects (which give attention to warnings in they tests) to fix real bugs. But it was not enough, so other bugs are fixed only now, when the warnings become even more visible in 3.8. We see a value of warnings, they help to fix bugs. If we rollback this change, it will cause yet undiscovered bugs be hidden for more time.

In Python 3.0 we make many abrupt breaking changes at once. People complained. Here is an opposite example of very gradual change: two releases with a DeprecationWarning, few more (at least two) releases with a SyntaxWarning, and finally perhaps a SyntaxError.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Nathaniel Smith <njs@pobox.com> added the comment:

I think we haven't *actually* done a proper DeprecationWarning period for this. We tried, but because of the issue with byte-compiling, the warnings were unconditionally suppressed for most users -- even the users who are diligent enough to enable warnings and look at warnings in their test suites.

I can see a good argument for making the change, but if we're going to do it then it's obviously the kind of change that requires a proper deprecation period, and that hasn't happened.

Maybe .pyc files need to be extended to store a list of syntax-related DeprecationWarnings and SyntaxWarnings, that are re-issued every time the .pyc is loaded? Then we'd at least have the technical capability to deprecate this properly.

----------
nosy: +njs

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Change by Ned Deily <nad@python.org>:


----------
nosy: +lukasz.langa

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Gregory P. Smith <greg@krypto.org> added the comment:

Can the pyc compilation step done by our normal package installers be made to treat this warning as an error so that it is forced into the package owners faces instead of overlooked because it was just something on stderr?

This syntax warning is absolutely the right thing to surface for _owners/maintainers_ of a given piece of code. Their strings are wrong and their code quality will improve as a result.

It isn't the right thing to surface to _users_ of someone else's problematic code. (though it does serve as a red flag proving that the owners of that code or person who pinned some dep to an old version haven't proactively tested with modern python)

----------
nosy: +gregory.p.smith

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Change by Serhiy Storchaka <storchaka+cpython@gmail.com>:


----------
pull_requests: +14879
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/15142

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Serhiy Storchaka <storchaka+cpython@gmail.com> added the comment:

PR 15142 reverts that change for 3.8.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Change by Sanyam Khurana <sanyam.khurana01@gmail.com>:


----------
nosy: +CuriousLearner

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Gregory P. Smith <greg@krypto.org> added the comment:


New changeset 4c5b6bac2408f879231c7cd38d67657dd4804e7c by Gregory P. Smith (Serhiy Storchaka) in branch '3.8':
[3.8] bpo-32912: Revert SyntaxWarning on invalid escape sequences (GH-15142)
https://github.com/python/cpython/commit/4c5b6bac2408f879231c7cd38d67657dd4804e7c


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Change by Gregory P. Smith <greg@krypto.org>:


----------
pull_requests: +14927
pull_request: https://github.com/python/cpython/pull/15195

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

Thank you!

----------
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Gregory P. Smith <greg@krypto.org> added the comment:

I'm leaving this open, as we may still want to do it in 3.9+, just in a less disruptive manner. (That, and how, hasn't been decided yet)

Follow the thread(s) on python-dev for the latest on that.

----------
resolution: fixed ->
stage: resolved ->
status: closed -> open
versions: +Python 3.9 -Python 3.8

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Aaron Meurer <asmeurer@gmail.com> added the comment:

Are there issues tracking the things I mentioned, which should IMO happen before this becomes a hard error (making the warnings reproduce even if the file has already been compiled, and making warning message point to the correct line in multiline strings)? And is it too late to potentially get some of those things in 3.8?

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Gregory P. Smith <greg@krypto.org> added the comment:

I haven't looked, so not that i'm aware of. I suggest filing one for each of those.

The warning not pointing to the right line in a multiline literal sounds like a bug to me so that one, if fixed, seems reasonable for 3.8. The release manager gets to decide.

Making the syntax warnings happen upon import of an already compiled pyc is more feature-ish, likely to be 3.9+.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Raymond Hettinger <raymond.hettinger@gmail.com> added the comment:

One possibility is to restrict the warning to a backslash followed by an alphabetic character or backslash, and that we define backslash followed by any other printable character as specifically allowed. This would catch the likely sources of errors without breaking the likes of \, or \-. Also, multiline strings have less of a need for a warning than a single line string.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment:


New changeset b4be87a04a2a8ccfd2480e19dc527589fce53555 by Miss Islington (bot) (Gregory P. Smith) in branch 'master':
bpo-32912: Revert SyntaxWarning on invalid escape sequences. (GH-15195)
https://github.com/python/cpython/commit/b4be87a04a2a8ccfd2480e19dc527589fce53555


----------
nosy: +miss-islington

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue32912] Raise non-silent warning for invalid escape sequences [ In reply to ]
Change by Steve Dower <steve.dower@python.org>:


----------
nosy: +steve.dower

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue32912>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com