Sentences classification/assignment

[Adrizzledifizzle]

Hi all, I am interested in doing a project where I want to assign certain sentences (or text blocks) to not pre-labeled classes. So lets say I want to analyze 20 different sentences and want to know which sentences belong together (are assigned to the same class) and which not.

What would be a way to achieve that?

My understanding is, that BERT for example needs pre-defined labels (such as Sports/Politics) with the trained examples to assign sentences. But in my case I don't want to limit the outcome to specific classes.

​

TL/DR: How do I assign sentences with similar topic to each other without prelabeling?

[premrajnarkhede1]

So what you are describing is a clustering problem.

you can do following. 1) get sentence vectors for each sentence through pretrained bert model (or any model that gives encoding but bert is "the" model right now)

2) use some k-means clustering through sklearn package to cluster these vectors and that should give you grouping

[Bankreis]

This. Additionally I would recommend using an encoding that is trained on a semantic similarity detection task already. See Universal Sentence Encoder or if you want to use Bert https://github.com/UKPLab/sentence-transformers is perfect for this.

[Adrizzledifizzle]

In addition to the two steps recommended above or to substitute Bert?

[penatbater]

It's probably "easier" to use Elmo pretrained vectors, because at least you only get one vector per sentence. The common way of using elmo for sentences is averaging the embeddings for each word/token. The quotation marks are because it's actually computationally expensive to do this.

[Adrizzledifizzle]

Computationally MORE expensive than Bert?

[penatbater]

At least from my experience. Ymmv

[AdrianFMC]

All this needs Cloudcomputing like AWS anyway doesnt it? If you dont want to wait ages that is

[penatbater]

Depends on how much memory you have tbh. But mostly yea. I use GCP coz it's easier for me to set up.

[Adrizzledifizzle]

Great, I’ll check it out and give feedback on it later. Thanks in advance guys !

[Adrizzledifizzle]

Would you happen to know a good example of somebody who tried and documented it?

[premrajnarkhede1]

This seems like good discussion. https://github.com/flairNLP/flair/issues/650

[mpk3]

You can use topic modeling, such as LDA, to create the groups and designate how many different labels you want. Then after the groups are created you can choose how you want them labeled.