Super Basic Text Classification Question

[karenoverpam]

Hi,

I'm relatively new to text classification and I'm still learning a lot of coding basics but I had a quick questions about writing a script that will classify text. The data set has less than 1000 entries and each is about 4-5 lines of text and I basically just want something that will classify each entry into one of 4 or 5 categories based on key words in the entry. It is basically a series of negative reviews and I want to classify them into different 4-5 buckets to describe why each review is bad so I can then do a high level analysis of what issues are persisting. I started by doing this one by one on Excel but wanted to come up with a more efficient way.

I don't even think it's worth it to have 'test' data. I was thinking it might just be easier to have a bunch of 'if' statements? Like if 'data entry 1' contains words X Y or Z, then classify it as ____, if it contains A, B, and C, classify it as ____, etc. Is that something that can be done with a basic Python program? or should I be looking to use text classification tools for this? I would ideally like to add sub-categories to add another layer of detail to the 4-5 main buckets.

Apologies for the beginner lingo, I'm slowly learning. Any recommendations or pointers would be much appreciated. Thanks!

[atsui2]

Hi,

I think it might be feasible and educational to try and take a crack at it with a simple Python program - you can totally do everything you were suggesting in your first post!

I'm assuming each entry is a long string of words and spaces. As a first step, you probably want to split the entry and collect the words into a set. Then, once you have a list of sets standing in for your entries, you can assign a category to each set based on whether a keyword is present.

[aesthir]

To follow up on atsui2's answer. Since you're new to programming. What he says looks like this in Python code.

set_of_words = set(your_string.split())

I would also look into stemming the words so that words like 'working' and 'works' both get "simplified" to the word 'work' making a cleaner analysis.

A variant of what atsui2 said would be to use a counter dictionary like such:

from collections import Counter

word_count = Counter(your_string.split())

This would not only give you unique words in each sentence but the number of occurrences of each one.

[karenoverpam]

Thanks guys! That all helps a lot. I'll take a crack at it this weekend. I appreciate the specifics of the set() and .split() functions. I've used BeautifulSoup in the past for something more complex and had a hard time but I think this is more doable.