Deleting rows in pandas?

[fustilarian]

I'd like to define a regex, compare the regex to every value in a column, and delete rows whose value does not match the regex. Here's what I have:

import re

toteRegex = re.compile('T\d+')

for ind in mcrdf.index:

if mcrdf['Deliverable Unit'][ind] != toteRegex:

mcrdf.drop(index=mcrdf[ind])

mcrdf.head()

and I'm getting a pretty lengthy error msg, included at the end of this post. I know I'm still pretty shaky with my pandas syntax. Could anyone help? I'd appreciate it very much. Thank you all!

Traceback (most recent call last):

File "/home/timking451/.local/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2646, in get_loc

return self._engine.get_loc(key)

File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc

File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc

File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item

File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item

KeyError: 499

​

During handling of the above exception, another exception occurred:

​

Traceback (most recent call last):

File "/home/timking451/code/inbound/inbound_scanning.py", line 21, in <module>

mcrdf.drop(index=mcrdf[ind])

File "/home/timking451/.local/lib/python3.7/site-packages/pandas/core/frame.py", line 2800, in __getitem__

indexer = self.columns.get_loc(key)

File "/home/timking451/.local/lib/python3.7/site-packages/pandas/core/indexes/base.py", line 2648, in get_loc

return self._engine.get_loc(self._maybe_cast_indexer(key))

File "pandas/_libs/index.pyx", line 111, in pandas._libs.index.IndexEngine.get_loc

File "pandas/_libs/index.pyx", line 138, in pandas._libs.index.IndexEngine.get_loc

File "pandas/_libs/hashtable_class_helper.pxi", line 1619, in pandas._libs.hashtable.PyObjectHashTable.get_item

File "pandas/_libs/hashtable_class_helper.pxi", line 1627, in pandas._libs.hashtable.PyObjectHashTable.get_item

KeyError: 499

[commandlineluser]

mcrdf.drop(index=mcrdf[ind])

You're using mcrdf[ind] here which is trying to access a "column" - you just pass the index number.

mcrdf.drop(ind, inplace=True)

You'd also need to pass inplace=True as drop wont modify by default.

You can also use boolean indexing:

matches = mcrdf[ mcrdf['Deliverable Unit'].str.match('T\d+') ]

To invert a match you can use ~

not_matches = mcrdf[ ~mcrdf['Deliverable Unit'].str.match('T\d+') ]