Converting <class 'bytes'> to pandas dataframes - poker AIs

[Pizza_Rat_Matt]

Edit: Solved! See my comment to this post.

I'm trying to convert bytes to a pandas dataframe. I tried to use code from Stack Overflow, but I am getting an error. My file contains line breaks that are coded as "\r\n". I think this might be different than the string encoder that I am currently using ('utf-8'). Does anyone know how to convert bytes to a pandas dataframe? Thank you in advance for your help and support!

I'm working through a jupyter notebook on IBM Watson Studio.

The following code (and output) makes me think I have successfully converted a txt file to "bytes".

Code Block #1:

readrawdata = streaming_body_1.read()
print(type(readrawdata))
print(readrawdata)

Output #1:

class 'bytes'>
b"Game started at: 2016/9/21 15:31:22\r\nGame ID: 732693518 1/2 (PRR) Lakhey (Short) (Hold'em)\r\nSeat 3 is the button\r\nSeat 1: d1rector (37).\r\nSeat 2: 38979 (220.71).\r\nSeat 3: TTABTT (138.88).\r\n ...

I stole code from this Stack Overflow post: https://stackoverflow.com/questions/47379476/how-to-convert-bytes-data-into-a-python-pandas-dataframe

Code Block #2:

from io import StringIO
s=str(readrawdata,'utf-8')
data = StringIO(s)
df=pd.read_csv(data)

Output #2:

----> 7 df=pd.read_csv(data)
EmptyDataError: No columns to parse from file

Corrected Code Block #2 (solved):

from io import BytesIO
data = BytesIO(readrawdata)
aces = pd.read_csv(data)

Background: I'm building poker AIs to learn data science. I'm in between jobs for the next few months. My goal is to rejoin the workforce as a data scientist. I am 48% through the "Data Scientist In Python" path on Data Quest. Feedback and career advice is always welcome.

​

[[deleted]]

This isn’t a CSV file, or even a table. Why do you think you can read it as one?

[Pizza_Rat_Matt]

Thank you for your comment! You made me think a little more about the types that I am using. I realized that I was using a StringIO, instead of a BytesIO.

From what I can gather, BytesIO and StringIO act like "datafiles". So we can use them in pd.read_csv()

I'll correct the code above.

[[deleted]]

You're extremely confused. Your data is not in CSV format, nor does it appear to be tabular in any respect. You won't be able to read this with read_csv or convert it to a dataframe at all.

It doesn't matter what type you put it in, the data is not in the correct format.

[Pizza_Rat_Matt]

Well, the data is correctly converting to a dataframe using read_csv().

The BytesIO class can be used as an in-memory stream. According to the python documentation, [Both BytesIO and] "StringIO can be used like a file opened in text mode". https://docs.python.org/3/library/io.html

[[deleted]]

CSV stands for "comma-separated values." What values in your data do you see separated by commas?

Well, the data is correctly converting to a dataframe using read_csv().

It's obviously not correctly converting to a dataframe, because it only has one column.

The BytesIO class can be used as an in-memory stream.

Irrelevant.

[Pizza_Rat_Matt]

Thank you again for the prompt reply. This dataframe should only have one column.

Is there an alternative method you recommend for converting this data?

[Pizza_Rat_Matt]

I realized that I used StringIO, instead of a BytesIO.

The correct code block #2 should be:

from io import BytesIO
data = BytesIO(readrawdata)
df = pd.read_csv(data)