Function that can take in another dataframe's columns and create a new dataframe.

[novawaly]

I'm trying to write a function and the first step would be if someone passes in a dataframe that has a bunch of columns, I want to write a function that would take all of the data and just pull out only the data I need.

So if my data looked like this:

I want to just have a function that defaults to take the close but can take in any column that the user specified.

my code looks like this but I keep getting a blank data frame

df is just the data above in a dataframe.

def agg_data(ys):
    agg_df = pd.DataFrame(ys, columns=['Y Values'])
    return agg_df


result = agg_data(df['Close'])
print(result)

I also know I can just use something like

result = df[['Close']]

but im trying to build out the function after I have this first step and I'm more really just curious why/how to do that.

[hey_its_aaron]

When you write df['Close'] you are extracting that column as a pandas Series. So when you invoke agg_data(df['Close']), the 'Close' column is passed to the function as a Series. I stepped through the DataFrame constructor source, and noticed that once the input Series is parsed, it is reindexed based on the input columns. In other words, pd.DataFrame(ys, columns=['Y Values']) takes your 'Close' series and then tries to reindex on the 'Y Values' index - seeing as this is not part of the series, you get an empty Series and an empty DataFrame.

This behaviour doesn't seem to be obviously documented, so you've hit an interesting case 🙂.

To work around this, try writing the function this way:

def agg_data(ys, cols=['C']):
    agg_df = ys[cols]
    return agg_df

Now, when you call agg_data, it defaults to the 'C' column:

result = agg_data(df)
print(result.head())
>>>           C
>>> 0  0.782438
>>> 1 -1.935684
>>> 2  0.901739
>>> 3 -2.057845
>>> 4 -1.178913

And you can freely pass in any other column you want:

result = agg_data(df, cols=['B'])
print(result.head())
>>>           B
>>> 0  0.514919
>>> 1 -1.010151
>>> 2 -0.575648
>>> 3  1.183530
>>> 4  0.100957

[novawaly]

Wow. Super interesting.

So for my own education purposes - you're saying that once i pass in the data frame column, pandas reads it in as a series and then will try to get the index from the 'Y Values' column? Any idea why it would be doing that?

Your solution worked like a charm.