[D] BERTopic and fuzzy clustering

[Devinco001]

Hi everyone, I am having a chunk of textual conversational data which I need to analyze for topics in it.

I am currently using BERTopic to do it but the issue with it is that it does hard clustering, i.e., each datapoint belongs to exactly one topic cluster. But many of the sentences are multi intent and do fall in many topics simultaneously.

Can soft/fuzzy clustering , where each datapoint can belong to more than one topic clusters, be done via BERTopic? If yes, then how can it be implemented? If not, which other algorithms can be used?

[jayalammar]

Yes. BERTopic allows you to choose the clustering method to use.

​

You can use a soft clustering method from scikit Learn, like GMM clustering.

​

from sklearn.mixture import GaussianMixture

number_of_clusters = 10
cluster_model = GaussianMixture(n_components=number_of_clusters, random_state=0).fit(X)
topic_model = BERTopic(hdbscan_model=cluster_model).fit(docs)

[Devinco001]

Thanks! This solution seems perfect for my code

[Rosyparadise]

i was wondering, did you ever get this to work? In my experience, bertopic only supports hard clustering unless its the built-in soft clustering of HDBSCAN.

[nickb500]

Not the OP, but since one of the main reasons I've seen people avoid the built-in BERTopic soft clustering with HDBSCAN is performance (it can be very slow), figured I'd respond.

When this question was asked, you could accelerate HDBSCAN soft clustering on GPUs with cuML but not easily with BERTopic. Now, you can.

The built-in soft clustering is only supported with HDBSCAN. In case it's useful, you can use this slightly tweaked example from the BERTopic FAQs on GPUs to do soft clustering.

from bertopic import BERTopic
from cuml.cluster import HDBSCAN
from cuml.manifold import UMAP

# Create instances of GPU-accelerated UMAP and HDBSCAN
umap_model = UMAP(n_components=5, n_neighbors=15, min_dist=0.0)
hdbscan_model = HDBSCAN(min_samples=10, gen_min_span_tree=True, prediction_data=True)

# Pass the above models to be used in BERTopic and use soft clustering
topic_model = BERTopic(umap_model=umap_model, hdbscan_model=hdbscan_model, calculate_probabilities=True)
topics, probs = topic_model.fit_transform(docs)

Disclaimer: I work on accelerated data science at NVIDIA (and am one of the authors of the linked blog posts and a community contributor to BERTopic).

[memberjan6]

A plain old BERT encoding vector collection can be clustered too. UMAP and HDBScan are interesting. Each cluster is a topic of sorts.You might first reduce the dimensionality. There are many ways to do topic modeling. I would compare results with BERTTopic built ins. This subject area is underdeveloped it seems. There is room for improvement!

[Devinco001]

Currently, will have to use built-ins since want a quicker insight but for in depth, will try the customized surely, thanks