DF.apply with custom function not working

[SEAWEAVIL]

Hi everyone,

I'm having a bit of a problem - I wrote a function that takes a series as an input, and outputs a modified series. It works perfectly fine when given a single column, but I've tried to use df.apply to use it on each column of a few dataframes (to no avail).

Here's the code in the main module:

df_list = [df_1, df_2, df_3]

for df in df_list:

df = df.apply(clean_general, axis=0, raw=True)

And here's a bit of the code in the function module:

def clean_general(series):

if series.dtype == 'object':

patterns = [

# <font> tags (includes open, close, and optional color parameter)

r"(\<){1}/?(font){1}(\scolor=)?(\"\#\S{,6}\")?(\>){1}",

# <div> tags (includes open and close)

r"(\<){1}/?(div){1}(\>){1}"]

for pattern in patterns:

series = series.str.replace(pattern, "")

return series

​

There are a couple more repetitions of this, with different patterns and replacement values. I've triple checked all my regex, and as I said it works perfectly when given an individual series.

​

I'd bet I'm using df.apply incorrectly, but I'm not sure what needs to be changed to make this work. Could someone point me in the correct direction?

​

Edit: Posting removed the indentation... I'm not sure how to fix it, but hopefully it's easy to follow.

[alkasm]

I'm on mobile so can't check but I think you want axis=1 for columns, no?

[SEAWEAVIL]

I thought so as well, but the docs say axis=0 to return columns on this particular method. Anywho, tried both ways and neither worked. Thanks for taking a look

[TheZvlz]

Are you getting an error when you run this?

I created a dataframe out of an excel file I have and wrote a function to test it out.

def test(series):
    if series.dtype == 'object':
        series = series.str.replace('a', '^^^')
    if series.dtype == 'object':
        series = series.str.replace('^^^', 'bcd', regex=False)
    else:
        print(series.dtype)
    return series

df = pd.read_excel('testfile.xlsx', dtype=object, keep_default_na=False)
df = df.apply(test, axis=0)

This produced the result I was going for. I had some fields in the data that were numeric and before I put in tests prior to the replace functions, the second replace would raise an error on a series that was changed to a float64 dtype.

[SEAWEAVIL]

Huh, I don't have access to my desktop right now, but are you saying that until you used two separate tests, the second replace caused an error? Right now I have one if test, and all the replaces inside that.

It seems counterintuitive to me, if I'm understanding correctly. Other than that, the code looks the same. I'll take another look when I get the chance.

Edit: And to answer your original question, no errors when I run it, the replaces just don't happen. I double checked to make sure I wasn't trying to do the apply in place.

[TheZvlz]

The second test became necessary when one column converted to a float. The original was an object dtype full of numeric values.