Mailing List Archive

Looks like the data to be written is buffered, so actual write takes
place after readlines(), when close() flushes buffers.

See io package documentation, BufferedIOBase.

The solution is file.flush() after file.write()

Can't deny, such a behaviour looks utterly weird. Try to avoid designing
your software this way.

Axy.

On 19/02/2023 14:03, Azizbek Khamdamov wrote:
> Example 1 (works as expected)
>
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities
> # the following code adds new record to the beginning of the file,
> expected behaviour
> file.write("new city\n")
>
> file.close()
>
>
> Example 2 (weird behaviour)
>
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities
> # the following code DOES NOT add new record TO THE BEGINNING of the
> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
> new content should be added to the beginning of the file (as in
> Example 1)
> file.write("new city\n")
>
> file.readlines()
> file.close()
>
> I could not find anything in documentation to explain this strange
> behaviour. Why is this happening?
--
https://mail.python.org/mailman/listinfo/python-list

On 2023-02-19 16:57:02 +0000, Axy via Python-list wrote:
> Looks like the data to be written is buffered, so actual write takes place
> after readlines(), when close() flushes buffers.
>
> See io package documentation, BufferedIOBase.
>
> The solution is file.flush() after file.write()

Or alternatively, file.seek() to the intended position when switching
between reading and writing. (The C standard says you have to do this. I
can't find it in the Python docs, but apparently Python behaves the
same.)

> On 19/02/2023 14:03, Azizbek Khamdamov wrote:
> > Example 2 (weird behaviour)
> >
> > file = open("D:\Programming\Python\working_with_files\cities.txt",
> > 'r+') ## contains list cities
> > # the following code DOES NOT add new record TO THE BEGINNING of the
> > file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
> > new content should be added to the beginning of the file (as in
> > Example 1)
> > file.write("new city\n")

Also note that you can't ADD anything at the beginning (or in the
middle) of a file. You will overwrite existing content if you try this.
You can only add at the end of the file. If you want to insert
something, you have to rewrite everything from that position.

(So typically, for small files you wouldn't update a file in place, you
would just replace it completely. For large data sets which need to be
updated you would generally use some kind of database.)

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

On 2023-02-19 14:03, Azizbek Khamdamov wrote:
> Example 1 (works as expected)
>
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities
> # the following code adds new record to the beginning of the file,
> expected behaviour
> file.write("new city\n")
>
> file.close()
>
>
> Example 2 (weird behaviour)
>
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities
> # the following code DOES NOT add new record TO THE BEGINNING of the
> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
> new content should be added to the beginning of the file (as in
> Example 1)
> file.write("new city\n")
>
> file.readlines()
> file.close()
>
> I could not find anything in documentation to explain this strange
> behaviour. Why is this happening?

It works correctly if you use file.flush() or file.tell() before
switching from writing to reading.
--
https://mail.python.org/mailman/listinfo/python-list

On 2/19/2023 11:57 AM, Axy via Python-list wrote:
> Looks like the data to be written is buffered, so actual write takes
> place after readlines(), when close() flushes buffers.
>
> See io package documentation, BufferedIOBase.
>
> The solution is file.flush() after file.write()

Another possibility, from the Python docs:

"...TextIOWrapper (i.e., files opened with mode='r+') ... To disable
buffering in TextIOWrapper, consider using the write_through flag for
io.TextIOWrapper.reconfigure()"


Also from the docs:

"Warning: Calling f.write() without using the with keyword or calling
f.close() might result in the arguments of f.write() not being
completely written to the disk, even if the program exits successfully."

I realize that in the example, close() was called ... but not
immediately after the write().

> Can't deny, such a behaviour looks utterly weird. Try to avoid designing
> your software this way.
>
> Axy.
>
> On 19/02/2023 14:03, Azizbek Khamdamov wrote:
>> Example 1 (works as expected)
>>
>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> 'r+') ## contains list cities
>> # the following code adds new record to the beginning of the file,
>> expected behaviour
>> file.write("new city\n")
>>
>> file.close()
>>
>>
>> Example 2 (weird behaviour)
>>
>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> 'r+') ## contains list cities
>> # the following code DOES NOT add new record TO THE BEGINNING of the
>> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
>> new content should be added to the beginning of the file (as in
>> Example 1)
>> file.write("new city\n")
>>
>> file.readlines()
>> file.close()
>>
>> I could not find anything in documentation to explain this strange
>> behaviour. Why is this happening?

--
https://mail.python.org/mailman/listinfo/python-list

On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
<azizbek.khamdamov@gmail.com> wrote:
>
> Example 1 (works as expected)
>
> file = open("D:\Programming\Python\working_with_files\cities.txt",
> 'r+') ## contains list cities

Side note: You happened to get lucky with P, w, and c, but for the
future, I recommend using forward slashes for your paths:

open("D:/Programming/Python/working_with_files/cities.txt", "r+")

Otherwise, you may run into annoying and hard-to-solve problems. Or
alternatively, you'll upgrade to a newer Python and start getting
warnings, which would at least tell you that there's a problem.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 2/19/2023 1:53 PM, Chris Angelico wrote:
> On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
> <azizbek.khamdamov@gmail.com> wrote:
>>
>> Example 1 (works as expected)
>>
>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> 'r+') ## contains list cities
>
> Side note: You happened to get lucky with P, w, and c, but for the
> future, I recommend using forward slashes for your paths:
>
> open("D:/Programming/Python/working_with_files/cities.txt", "r+")
>
> Otherwise, you may run into annoying and hard-to-solve problems. Or
> alternatively, you'll upgrade to a newer Python and start getting
> warnings, which would at least tell you that there's a problem.

Or use r'...' strings. If you are copying a path to clipboard from
Windows Explorer - a fairly common operation - it's much easier to
prepend the "r" than to change all the backslashes to forward slashes.

--
https://mail.python.org/mailman/listinfo/python-list

On 2023-02-19 12:59:43 -0500, Thomas Passin wrote:
> On 2/19/2023 11:57 AM, Axy via Python-list wrote:
> > Looks like the data to be written is buffered, so actual write takes
> > place after readlines(), when close() flushes buffers.
> >
> > See io package documentation, BufferedIOBase.
> >
> > The solution is file.flush() after file.write()
>
> Another possibility, from the Python docs:
>
> "...TextIOWrapper (i.e., files opened with mode='r+') ... To disable
> buffering in TextIOWrapper, consider using the write_through flag for
> io.TextIOWrapper.reconfigure()"

That actually doesn't help (I tried it before writing my answer). The
binary layer below the text layer also buffers ...

> Also from the docs:
>
> "Warning: Calling f.write() without using the with keyword or calling
> f.close() might result in the arguments of f.write() not being completely
> written to the disk, even if the program exits successfully."

He does call file.close():

> > > file.close()

so that doesn't seem relevant.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"

On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1@tompassin.net> wrote:
>
> On 2/19/2023 1:53 PM, Chris Angelico wrote:
> > On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
> > <azizbek.khamdamov@gmail.com> wrote:
> >>
> >> Example 1 (works as expected)
> >>
> >> file = open("D:\Programming\Python\working_with_files\cities.txt",
> >> 'r+') ## contains list cities
> >
> > Side note: You happened to get lucky with P, w, and c, but for the
> > future, I recommend using forward slashes for your paths:
> >
> > open("D:/Programming/Python/working_with_files/cities.txt", "r+")
> >
> > Otherwise, you may run into annoying and hard-to-solve problems. Or
> > alternatively, you'll upgrade to a newer Python and start getting
> > warnings, which would at least tell you that there's a problem.
>
> Or use r'...' strings. If you are copying a path to clipboard from
> Windows Explorer - a fairly common operation - it's much easier to
> prepend the "r" than to change all the backslashes to forward slashes.
>

Yep, either way. I personally prefer using forward slashes since
there's no way to end an r-string with a single backslash, which is
often useful when building a path:

# won't work
path = r"c:\path\to\some\"
open(path + "file")

# will work
path = "c:/path/to/some/"
open(path + "file")

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

On 2023-02-19 19:31, Chris Angelico wrote:
> On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1@tompassin.net> wrote:
>>
>> On 2/19/2023 1:53 PM, Chris Angelico wrote:
>> > On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
>> > <azizbek.khamdamov@gmail.com> wrote:
>> >>
>> >> Example 1 (works as expected)
>> >>
>> >> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> >> 'r+') ## contains list cities
>> >
>> > Side note: You happened to get lucky with P, w, and c, but for the
>> > future, I recommend using forward slashes for your paths:
>> >
>> > open("D:/Programming/Python/working_with_files/cities.txt", "r+")
>> >
>> > Otherwise, you may run into annoying and hard-to-solve problems. Or
>> > alternatively, you'll upgrade to a newer Python and start getting
>> > warnings, which would at least tell you that there's a problem.
>>
>> Or use r'...' strings. If you are copying a path to clipboard from
>> Windows Explorer - a fairly common operation - it's much easier to
>> prepend the "r" than to change all the backslashes to forward slashes.
>>
>
> Yep, either way. I personally prefer using forward slashes since
> there's no way to end an r-string with a single backslash, which is
> often useful when building a path:
>
> # won't work
> path = r"c:\path\to\some\"
> open(path + "file")
>
> # will work
> path = "c:/path/to/some/"
> open(path + "file")
>
When I build a path, I use os.path.join, so it's only a problem if one
of the strings refers to the root of a drive, e.g. "C:\\".
--
https://mail.python.org/mailman/listinfo/python-list

On 2/19/2023 2:31 PM, Chris Angelico wrote:
> On Mon, 20 Feb 2023 at 06:24, Thomas Passin <list1@tompassin.net> wrote:
>>
>> On 2/19/2023 1:53 PM, Chris Angelico wrote:
>>> On Mon, 20 Feb 2023 at 03:41, Azizbek Khamdamov
>>> <azizbek.khamdamov@gmail.com> wrote:
>>>>
>>>> Example 1 (works as expected)
>>>>
>>>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>>>> 'r+') ## contains list cities
>>>
>>> Side note: You happened to get lucky with P, w, and c, but for the
>>> future, I recommend using forward slashes for your paths:
>>>
>>> open("D:/Programming/Python/working_with_files/cities.txt", "r+")
>>>
>>> Otherwise, you may run into annoying and hard-to-solve problems. Or
>>> alternatively, you'll upgrade to a newer Python and start getting
>>> warnings, which would at least tell you that there's a problem.
>>
>> Or use r'...' strings. If you are copying a path to clipboard from
>> Windows Explorer - a fairly common operation - it's much easier to
>> prepend the "r" than to change all the backslashes to forward slashes.
>>
>
> Yep, either way. I personally prefer using forward slashes since
> there's no way to end an r-string with a single backslash, which is
> often useful when building a path:
>
> # won't work
> path = r"c:\path\to\some\"
> open(path + "file")
>
> # will work
> path = "c:/path/to/some/"
> open(path + "file")

Never hit that one before. To be classified as "Learn something new
every day"!

I've been using pathlib more lately. It does clever things with "/" -

>>> from pathlib import PurePath
>>> p1 = PurePath('c:/this') / 'is' / 'a' /'test'
>>> p1
PureWindowsPath('c:/this/is/a/test')
# on Linux, would be a PurePosixPath

and also

>>> p2 = PurePath('c:/this/') /'is' /'a' /'test'
>>> p2
PureWindowsPath('c:/this/is/a/test')

Despite the apparent forward slashes, the right separators get used for
the OS. And the Paths can be used with open(), etc.


--
https://mail.python.org/mailman/listinfo/python-list

Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:
> ...
>Example 2 (weird behaviour)
>
>file = open("D:\Programming\Python\working_with_files\cities.txt",
>'r+') ## contains list cities
># the following code DOES NOT add new record TO THE BEGINNING of the
>file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
>new content should be added to the beginning of the file (as in
>Example 1)
>file.write("new city\n")
>
>file.readlines()
>file.close()
>
>I could not find anything in documentation to explain this strange
>behaviour. Why is this happening?

The effect of "r+" (and friends) is specified by the C standard.

The Linux doc (of `fopen`) tells us that ANSI C requires that
a file positioning command (e.g. `seek`) must intervene
between input and output operations. Your example above violates
this condition. Therefore, weird behavior is to be expected.

--
https://mail.python.org/mailman/listinfo/python-list

On 2/19/23 14:06, Dieter Maurer wrote:
> Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:
>> ...
>> Example 2 (weird behaviour)
>>
>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>> 'r+') ## contains list cities
>> # the following code DOES NOT add new record TO THE BEGINNING of the
>> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
>> new content should be added to the beginning of the file (as in
>> Example 1)
>> file.write("new city\n")
>>
>> file.readlines()
>> file.close()
>>
>> I could not find anything in documentation to explain this strange
>> behaviour. Why is this happening?
>
> The effect of "r+" (and friends) is specified by the C standard.
>
> The Linux doc (of `fopen`) tells us that ANSI C requires that
> a file positioning command (e.g. `seek`) must intervene
> between input and output operations. Your example above violates
> this condition. Therefore, weird behavior is to be expected.

If this isn't sufficiently described, someone should raise an issue
against the Python docs. I know that many concepts are "inherited from"
environments generally in the POSIX space and the C language, because
that's where Python was hatched (all of which makes perfect sense to me,
who's been working in those spaces for...ever), but a Python programmer
shouldn't have to read the ISO C standard (which is not free, although
you can find copies on-line), or the POSIX standard (which also is not
free, though manpages for systems like Linux cover the same material),
in order to figure out how Python is going to work.
--
https://mail.python.org/mailman/listinfo/python-list

On 2/19/2023 6:10 PM, Mats Wichmann wrote:
> On 2/19/23 14:06, Dieter Maurer wrote:
>> Azizbek Khamdamov wrote at 2023-2-19 19:03 +0500:
>>> ...
>>> Example 2 (weird behaviour)
>>>
>>> file = open("D:\Programming\Python\working_with_files\cities.txt",
>>> 'r+') ## contains list cities
>>> # the following code DOES NOT add new record TO THE BEGINNING of the
>>> file IF FOLLOWED BY readline() and readlines()# Expected behaviour:
>>> new content should be added to the beginning of the file (as in
>>> Example 1)
>>> file.write("new city\n")
>>>
>>> file.readlines()
>>> file.close()
>>>
>>> I could not find anything in documentation to explain this strange
>>> behaviour. Why is this happening?
>>
>> The effect of "r+" (and friends) is specified by the C standard.
>>
>> The Linux doc (of `fopen`) tells us that ANSI C requires that
>> a file positioning command (e.g. `seek`) must intervene
>> between input and output operations. Your example above violates
>> this condition. Therefore, weird behavior is to be expected.
>
> If this isn't sufficiently described, someone should raise an issue
> against the Python docs.  I know that many concepts are "inherited from"
> environments generally in the POSIX space and the C language, because
> that's where Python was hatched (all of which makes perfect sense to me,
> who's been working in those spaces for...ever), but a Python programmer
> shouldn't have to read the ISO C standard (which is not free, although
> you can find copies on-line), or the POSIX standard (which also is not
> free, though manpages for systems like Linux cover the same material),
> in order to figure out how Python is going to work.

The Python docs say that text file behavior is done by Python, not C,
and so its behavior will be consistent across all platforms. I didn't
find any words about this case, though.
--
https://mail.python.org/mailman/listinfo/python-list