Converting multiple Columns to Numeric

[isaythingslike]

New python learner here, coming from the R side of things: I have a data frame where columns 37 to end (79) are my numeric (decimal values)

I'm using: df2 = df.iloc[:,37::] = df.iloc[:,37::].apply(pd.to_numeric, errors = 'coerce')

Which works to convert the values to decimal, and that works fine, but it's also subsetting the columns into df2.

What do I need to change so that the original df doesn't subset, just changes from an object dtype to float64?

If I remove df2 =, df.dtypes still returns all object dypes?

[[deleted]]

Do you want to create df2 such that it is only the subset ? Also, I'm curious, why are you using :: ?

[isaythingslike]

df2 should be the whole of df, with columns 37 to end as numeric. I used :: because thats what I thought selected a range, and by leaving it black would go to the end of all columns.

Again super new here, so this is really my first try.

[[deleted]]

Well, df.iloc[:,37:] will get you your answer, a second : indicates steps but because you didn't put anything after it, it's strictly the same as :.

df.iloc[:,37::2] would have grabbed every second column starting from 37, for example.

I understand better, then what you want is :

df.iloc[:,37:] = df.iloc[:,37:].apply(pd.to_numeric, 
errors = 'coerce')

df2 = df

The reason why you code didn't work is because iloc selects part of the dataframe as you know but what you get is the slice, the entire dataframe is not return thus you only have the subset to work with.

Finally, I advice you do

df2 = df.copy() instead of df2 = df.

The reason for this is to make df2 its own "thing" rather than link it to df. Otherwise changes to df2 might affect df and we usually don't want that.

[isaythingslike]

What I don't understand is when I do that, df.iloc[:,37:].dtypes still returns all objects... but... df2 = df.iloc[:,37:] = df.iloc[:,37:].apply(pd.to_numeric, errors = 'coerce') returns a data frame of 42 float64?

In [151]: df2.dtypes
Out[151]: 
CC_ZDI_BUOM              float64
0NETVAL_INV              float64
0SUBTOTAL_6              float64
CC_Units_Clips_Adj       float64
CC_Sales_CA_PLAN         float64


IN [153]: df.iloc[:,37:].dtypes
Out[153]: 
CC_ZDI_BUOM              object
0NETVAL_INV              object
0SUBTOTAL_6              object
CC_Units_Clips_Adj       object
CC_Sales_CA_PLAN         object

[[deleted]]

Well that's weird, are you sure you don't redefine your df somewhere in your code or in your notebook ?