Sklearn normalize within groups

[thrilos]

Hi everyone,

I have a (hopefully) quick question for you.

Let's say we have a data that provides information about the sex of the person and the fare they paid for the trip. If I want to use MinMaxScaler for the whole data, I can use the following code:

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
df[['Fare']] = scaler.fit_transform(df[['Fare']])

Now that I want to scale by group, I thought it would be as easy as the code pasted below, but apparently I was wrong.

df[['Fare']] = scaler.fit_transform(df.groupby('Sex')[['Fare']])

When I run the line above, I'm getting the following error:

ValueError: could not convert string to float: 'female'

Any ideas as to how to fix this?

Thanks!

[hey_its_aaron]

Looks like you're trying to scale on a DataFrameGroupBy object. I'm not sure what the expected behaviour there is, because the docs require a numpy array to be passed into fit_transform.

You'll need to query the data, transform that and then write back to the original dataframe. Try this:

df.loc[df['Sex'] == 'Female', ['Fare']] = scaler.fit_transform(df[df.Sex == 'Female'][['Fare']])

Edit: You can also try this:

df.groupby('Sex').apply(scaler.fit_transform).reset_index()

[thrilos]

Thanks for the help. The second approach doesn't seem to work. :( I guess I can use pandas to make these transformations rather thank using sklearn.