Mailing List Archive

[Mark Hammond]
> ...
> MSVC exposes a global variable that contains the default [fopen] mode -
> ie, you can change the default to binary. (_fmode for those with the
> docs)
>
> This has some implications and questions:
> * Will Guido ever bow to pressure (when it arrives :) to expose this via
> the "msvcrt" module?

No. It changes the advertised semantics of Python builtins, and no option
ever does that. If it went in at all, it would have to be exposed as a
Python-level feature that changed the semantics similarly on all
platforms -- and even then Guido wouldn't put it in <wink>.

> ...
> Thus, it appears that to be 100% sure what mode you are using, you should
> not rely on the default, but should _always_ use "b" or "t" on the file
> mode.

And on platforms that have libc options to treat "t" as if it were "b"?
There's no limit to how perverse platform options can get! There's no fully
safe ground to stand on, so Python stands on the minimal guarantees libc
provides. If a user violates those, tough, they can't use Python. Unless,
of course, they contribute a lot of money to the PSA <wink>.

> ...
> Any thoughts or comments? The case for abandoning the CRTL's text mode
> gets stronger and stronger!

C's text mode is, alas, a bad joke. The only thing worse is Microsoft's
half-assed implementation of it <0.5 wink>.

ctrl-z-=-eof-even-gets-in-the-way-under-windows!-ly y'rs - tim

> I asked Guido to provide comments on one of the chapters in our book:
>
> I was discussing appending the mode ("t" or "b") to the open() call
>
> > p.10, bottom: text mode is the default -- I've never seen the 't'
> > option described! (So even if it exists, better be silent about it.)
> > You need to append 'b' to get binary mode instead.

In addition, 't' probably isn't even supported on many Unix systems!

> This brings up an interesting issue.
>
> MSVC exposes a global variable that contains the default mode - ie, you can
> change the default to binary. (_fmode for those with the docs)

The best thing to do with this variable is to ignore it. In large
programs like Python that link together pieces of code that never ever
heard about each other, making global changes to the semantics of
standard library functions is a bad thing. Code that sets it or
requires you to set it is broken.

> This has some implications and questions:
> * Will Guido ever bow to pressure (when it arrives :) to expose this via
> the "msvcrt" module? I can imagine where it may be useful in a limited
> context. A reasonable argument would be that, like _setmode and other MS
> specific stuff, if it exists it should be exposed.

No. (And I've never bought that argument before -- I always use "is
there sufficient need and no other way.")

> * But even if not, due to the shared CRTL, in COM and other worlds we
> really cant predict what the default is. Although Python does not touch
> it, that does not stop someone else touching it. A web-server built using
> MSVC on Windows may use it?

But would be stupid for it to do so, and I would argue that the web
server was broken. Since they should know better than this, I doubt
they do this (this option is more likely to be used in small,
self-contained programs). Until you find a concrete example, let's
ignore the possibility.

> Thus, it appears that to be 100% sure what mode you are using, you should
> not rely on the default, but should _always_ use "b" or "t" on the file
> mode.

Stop losing sleep over it.

> Any thoughts or comments? The case for abandoning the CRTL's text mode
> gets stronger and stronger!

OK, you write the code :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

[Mark]
> I asked Guido to provide comments on one of the chapters in our
> book:
>
> I was discussing appending the mode ("t" or "b") to the open() call

[Guido]
> > p.10, bottom: text mode is the default -- I've never seen the 't'
> > option described! (So even if it exists, better be silent about it.)
> > You need to append 'b' to get binary mode instead.

I hadn't either, until I made the mistake of helping Mr
took-6-exchanges-before-he-used-the-right-DLL Embedder, who used it
in his code. Certainly not mentioned in man fopen on my Linux box.

> This brings up an interesting issue.
>
> MSVC exposes a global variable that contains the default mode - ie,
> you can change the default to binary. (_fmode for those with the
> docs)

Mentally prepend another underscore. This is something for that other
p-language.

>... The case for abandoning the CRTL's text
> mode gets stronger and stronger!

If you're tying this in with Tim's Icon worship, note that in these
days of LANS, the issue is yet more complex. It would be dandy if I
could read text any old text file and have it look sane, but I may be
writing it to a different machine without any way of knowing that.

When I bother to manipulate these things, I usually choose to use
*nix style text files. But I don't deal with Macs, and the only
common Windows tool that can't deal with plain \n is Notepad.

and-stripcr.py-is-everywhere-available-on-my-Linux-box-ly y'rs

- Gordon

>> I was discussing appending the mode ("t" or "b") to the open() call

> In addition, 't' probably isn't even supported on many Unix systems!

't' is not ANSI C, so there's no guarantee that it's portable. Hate to say
it, but Python should really strip t out before passing a mode string to
fopen!

[Mark]
>> ... The case for abandoning the CRTL's text mode gets stronger
>> and stronger!

[Gordon]
> If you're tying this in with Tim's Icon worship,

Icon inherits stdio behavior-- for the most part --too. It does define its
own mode string characters, though (like "t" for translated and "u" for
untranslated); Icon has been ported to platforms that can't even spell libc,
let alone support it.

> note that in these days of LANS, the issue is yet more complex. It would
> be dandy if I could read text any old text file and have it look sane, but
> I may be writing it to a different machine without any way of knowing
that.

So where's the problem? No matter *what* machine you end up on, Python
could read the thing fine. Or are you assuming some fantasy world in which
people sometimes run software other than Python <wink>?

Caveat: give the C std a close reading. It guarantees much less about text
mode than anyone who hasn't studied it would believe; e.g., text mode
doesn't guarantee to preserve chars with the high bit set, or most control
chars either (MS's treatment of CTRL-Z as EOF under text mode conforms to
the std!). Also doesn't guarantee to preserve a line-- even if composed of
nothing but printable chars --if it's longer than 509(!) characters.

That's what I mean when I say stdio's text mode is a bad joke.

> When I bother to manipulate these things, I usually choose to use
> *nix style text files. But I don't deal with Macs, and the only
> common Windows tool that can't deal with plain \n is Notepad.

I generally create text files in binary mode, faking the \n convention by
hand. Of course, I didn't do this before I became a Windows Guy <0.5 wink>.

> and-stripcr.py-is-everywhere-available-on-my-Linux-box-ly y'rs

A plug for my linefix.py (Python FTP contrib, under System), which converts
among Unix/Windows/Mac in any direction (by default, from any to Unix).

who-needs-linux-when-there's-a-python-in-the-window-ly y'rs - tim

Tim Peters wrote:
>
> >> I was discussing appending the mode ("t" or "b") to the open() call
>
> > In addition, 't' probably isn't even supported on many Unix systems!
>
> 't' is not ANSI C, so there's no guarantee that it's portable. Hate to say
> it, but Python should really strip t out before passing a mode string to
> fopen!

Should we also filter the socket type when creating sockets? Or the
address family? What if I pass "bamboozle" as the fopen mode? Should
that become "bab" after filtering? Oh, but what about those two "b"
characters? Maybe just reduce it to one? We also can't forget to filter
chmod() arguments... can't have unknown bits set.

etc etc

In other words, I think the idea of "stripping out the t" is bunk.
Python is not fatherly. It gives you the rope and lets you figure it out
for yourself. You should know that :-)

Cheers,
-g

--
Greg Stein, http://www.lyra.org/

> 't' is not ANSI C, so there's no guarantee that it's
> portable. Hate to say
> it, but Python should really strip t out before passing a
> mode string to
> fopen!

OK - thanks all - it is clear that this MS aberration is not, and never
will be supported by Python. Not being a standards sort of guy <wink> I
must admit I assumed both the "t" and "b" were standards.

Thanks for the clarifications!

Mark.

[Tim]
> 't' is not ANSI C, so there's no guarantee that it's portable.
> Hate to say it, but Python should really strip t out before passing
> a mode string to fopen!

[Greg Stein]
> Should we also filter the socket type when creating sockets? Or the
> address family?

Filtering 't' is a matter of increasing portability by throwing out an
option that doesn't do anything on the platforms that accept it, yet can
cause a program to die on platforms that don't -- despite that it says
nothing. So it's helpful to toss it, not restrictive.

> What if I pass "bamboozle" as the fopen mode? Should that become "bab"
> after filtering? Oh, but what about those two "b" characters?

Those go far beyond what I suggested, Greg. Even so <wink>, it would indeed
help a great many non-C programmers if Python defined the mode strings it
accepts & barfed on others by default.

The builtin open is impossible for a non-C weenie to understand from the
docs (as a frustrated sister delights in reminding me). It should be made
friendlier. Experts can use a new os.fopen if they need to pass
"bamboozle"; fine by me; I do think the builtins should hide as much
ill-defined libc crap as possible (btw, "open" is unique in this respect).

> Maybe just reduce it to one? We also can't forget to filter chmod()
> arguments... can't have unknown bits set.

I at least agree that chmod has a miserable UI <wink>.

> etc etc
>
> In other words, I think the idea of "stripping out the t" is bunk.
> Python is not fatherly. It gives you the rope and lets you figure it out
> for yourself. You should know that :-)

So should Mark -- but we have his testimony that, like most other people, he
has no idea what's "std C" and what isn't. In this case he should have
noticed that Python's "open" docs don't admit to "t"'s existence either, but
even so I see no reason to take comfort in the expectation that he'll
eventually be hanged for this sin.

ypu-i'd-rather-"open"-died-when-passed-"t"-ly y'rs - tim