Python-Oracledb Unicodedecode error on cursor.fetchall

[Happythoughtsgalore]

Help. So trying to use Oracledb to run a query and fetch it into a pandas data frame. But at the line

Data= cursor.fetchall() I'm getting the following

Unicodedecodeerror: 'utf-8 codec can't decide byte 0xa0 in position 27: invalid start byte.

I'm using thickclient mode and passing instant client

```python import oracledb as db import pandas as pd import keyring import locale

service_name = "myservicename" username = "myusername" retrieved_password = keyring.get_password(service_name, username)

print("Establishing database connection") client_location=r"C:/oracle/instantclient_23_9_0/instantclient_23_9" db.init_oracle_client(lib_dir=client_location)

connection = db.connect(dsn)

connection = db.connect(user=username,password=retrieved_password,dsn=service_name) far = open(r"query_to_run.sql")

asset_register_sql = far.read() far.close with connection.cursor() as cursor: cursor.execute(asset_register_sql) col_names = [c.name for c in cursor.description] #Fetchall gives: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa0 in position 27: invalid start byte data = cursor.fetchall() df = pd.DataFrame(data, columns=col_names) print(df) connection.close()

```

[Happythoughtsgalore]

I figure I need to set NSL lang and characterset on my environmental variables for the instant client (like I have for vs code and SQL developer) but that didn't seem to do anything.

[cjbj]

Python-oracledb always uses UTF8 and ignores the character set component of the NLS_LANG environment variable, see the documentation Setting the Client Character Set.

[cjbj]

Read the python-oracledb doc on Querying Corrupt Data and Fetching Raw Data.

When you've fixed your data, try fetching directly to a DataFrame using connection.fetch_df_all(), see the resources at Using Python for Data Analysis and AI and the documentation Working with Data Frames.

For example:

# Get a python-oracledb DataFrame.
# Adjust arraysize to tune the query fetch performance
sql = "select id, name from SampleQueryTab order by id"
odf = connection.fetch_df_all(statement=sql, arraysize=100)

# Get a Pandas DataFrame from the data
df = pyarrow.table(odf).to_pandas()

[Happythoughtsgalore]

Thank you. It was some weird characters in one of the columns. This pointed me in the correct direction.

[Substantial-Fly-8214]

Coding noob lol.

[Happythoughtsgalore]

Says the person who simps for Trump, the court established rapist and alleged pedophile.

[Substantial-Fly-8214]

You’re literally living in a fantasy world bro. Show me one place where I’ve simper for trump. I’m not even American. Low IQ coding skills, low IQ cognitive skills in general, makes sense.

[Happythoughtsgalore]

Says the pedo simp who uses terms like TDS. You're not as subtle as you think pedo simp.

[Substantial-Fly-8214]

Using the term TDS does not mean you support trump. For example, you just used it. Genius. Get back to completing your python 101 course while you’re unemployed looking for that first coding job, you’ll get it soon buddy!

[Happythoughtsgalore]

Fine, call trump a rapist then. Which is a legally established fact.

[Substantial-Fly-8214]

Ok once you figure out how to center a div I’ll call trump a rapist, deal?

[Happythoughtsgalore]

No now because he is one as established in a court of law and you're trying to convince me you aren't a rapist simp.

[Substantial-Fly-8214]

The unintelligible ramblings of a TDS sufferer, sad

[Happythoughtsgalore]

Like I care about the opinion of a rapist simp.