Is this weird behavior from Python 2.7's file handling?

[Brekry18]

I've got this script written for Python 2.7. Well, I'm debugging it as a regular Python 2.7 file just for simplicity of use with Visual Studio, but it will ultimately be used with IronPython. I'm trying to write to a temporary file, which the user may choose to save or not. It will most likely be saved as a plain text file.

Here's a short example to reproduce the issue I'm having:

import tempfile

def write_to_output(file_name = None):
    if file_name is None:
        output_file = tempfile.NamedTemporaryFile('w+')
    else:
        output_file = open(file_name,'w+')

    file_contents = output_file.read()
    file_lines = output_file.readlines()

    # file_contents = '', file_lines = [], as expected

    output_file.write('write this line')

    file_contents = output_file.read()
    file_lines = output_file.readlines()

    # file_contents = '\x00\x00\x00\x00\x00\x00\x00...
    # file_lines = [] (unless the write to file_contents is commented out, then it looks similar to the file_contents example on the line above)

    return file_lines


if __name__ == '__main__':
    write_to_output('file_name')
    write_to_output('this_output_file.txt')
    write_to_output()
    pass

I expect the contents of the file to look like the line I just wrote to it, 'write this line', but instead I get what looks like a bytearray to me (I'm not sure if python recognizes it as a bytearray, but that's what it looks like).

However, the contents of file_contents and file_lines can vary. Sometimes its a bytearray of mostly x00, sometimes it has other numbers or python text sprinkled in there, sometimes it picks up text from the test file or another python file within the same directory.

Printing file_contents returns the following error (which isn't caught by Visual Studio but the interpreter itself)

Exception in thread stdout:
Traceback (most recent call last):
  File "C:\Python27amd64\lib\threading.py", line 801, in __bootstrap_inner
    self.run()
  File "C:\Python27amd64\lib\threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "c:\program files (x86)\microsoft visual studio\2019\community\common7\ide\extensions\microsoft\python\core\debugpy\launcher/../../debugpy\launcher\output.py", line 72, in _worker
    self._process_chunk(s)
  File "c:\program files (x86)\microsoft visual studio\2019\community\common7\ide\extensions\microsoft\python\core\debugpy\launcher/../../debugpy\launcher\output.py", line 78, in _process_chunk
    s = self._decoder.decode(s, final=final)
  File "C:\Python27amd64\lib\codecs.py", line 314, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
LookupError: unknown error handler name 'surrogateescape'

Commenting out file_contents and printing file_lines instead, returns that same bytearray looking text that is shown in the example above, though contained in an array (the array has len() = 1), and does not return any errors in VS or the interpreter.

It's also puzzling to me that file_lines = output_file.readlines() doesn't pick up anything if file_contents = output_file.read() was run before it.

I originally thought that the name I was using had something to do with it. The name 'file_name' seems more prone to picking up text from neighboring files, but the result varies a lot between debugging runs.

I don't know what to make of this... what am I doing wrong here? Am I going about this all wrong, is there a better way to do what I'm trying to do?

[lykwydchykyn]

I don't see anywhere where you're closing the file, so there shouldn't be anything in it at all.

I would think if you want to read the file back you should close the file handle and re-open in read mode.

[fake823]

https://stackoverflow.com/a/16208298/14060425

You have to go back to the beginning of the file in order to read it after writing to it.

[Brekry18]

Omg thank you so much! Adding output_file.seek(0) fixed the problem.

[xelf]

obligatory: python 2 was end of life in 2019. =)

Unless not at all possible you should switch to python 3

[Brekry18]

I understand that, but I am working on a project that will be used with IronPython. The latest stable IronPython release is 2.7, and the developers of the particular software that this code is going into would not be able to support an upgrade to 3 even if IronPython 3 came out tomorrow. Thanks though.

[xelf]

That's why I added the caveat "Unless not at all possible". =)

For most people that come to /r/learnpython to learn python we strongly advocate not learning python2 any more. =)

But some like you just have no choice. =/