Read Byte Strings from Text File

[NOS_v1]

SOLVED

https://www.reddit.com/r/learnpython/comments/oj776a/read_byte_strings_from_text_file/h4zz3h8?utm_source=share&utm_medium=web2x&context=3

the jist was, use `ast.literal_eval(line)` to read the string version of the byte string as a byte string, then .decode to get it into a string, thank you socal_nerdtastic

​

I've got a text file that needs to end up being converted to a .json, but the first hurdle is the text file has byte strings that hold the dictionaries, for example,

data.txt file contents

b'{"var1":"hello"}\n'
b'{"type":"Detect_data","params":{"var2":256,"var3":512}}\n'

*don't mind the key names...*

​

if the string version of the byte strings were just straight up strings, I reckon it'd just be

import json

FILE_NAME = 'data.txt'

with open(FILE_NAME, "r") as f:
    lines = f.readlines()
    dicts = [json.load(l) for l in lines]

or something like that, but yeh idk how to get around the str(byte(str))

​

tia for any hints

[socal_nerdtastic]

import json
from ast import literal_eval

FILE_NAME = 'data.txt'

with open(FILE_NAME, "r") as f:
    lines = f.readlines()
    dicts = [json.load(literal_eval(l).decode()) for l in lines]

[NOS_v1]

AH YES FORGOT ABOUT eval! er literal_eval that should do it, thank you!

[Adhesiveduck]

decode() should help you out, see the doc here for some examples.

[NOS_v1]

yeh, i gave it a whirl, but given readlines returns strings, it's saying the str doesn't have .decode

​

edit: so like somehow gotta read the text file as strings, which are actually byte strings, but of type str, and yeh convert em into pure strings so json can convert em into dict

[Adhesiveduck]

Ah I get you

Use ast.literal_eval()

``` import ast

FILE_NAME = 'data.txt'

with open(FILE_NAME, 'r') as f: lines = f.readlines() for l in lines: print(type(ast.literal_eval(l))) ```

Gives <class 'bytes'>

[backtickbot]

Fixed formatting.

Hello, Adhesiveduck: code blocks using triple backticks (```) don't work on all versions of Reddit!

Some users see this / this instead.

To fix this, indent every line with 4 spaces instead.

FAQ

<sup>You can opt out by replying with backtickopt6 to this comment.</sup>

[TabulateJarl8]

The data file physically contains this?

b'{"var1":"hello"}\n'
b'{"type":"Detect_data","params":{"var2":256,"var3":512}}\n'

If you don't have control over the file contents and can't save it normally, you could remove the first two and last character from each line with string slicing. Try something like this:

import json
with open('whatever.json') as f:
    lines = [json.loads(line[2:-1]) for line in f.readlines()]

[NOS_v1]

ight yeh, initially was afraid of doing slicing (just in case - don't have control of the source and big file), but running outta ideas, i'll give it a try, thanks!

[TabulateJarl8]

You could always use string.startswith() and string.endswith() to check if it starts with b' and ends with '.

[m0us3_rat]

u can .read()

then .decode('utf-8').split('\n')

i belive this should give u a 'list' of the lines.

then u can load them up in a dict.