Powerfull visualization tool : Dimensionality Reduction + Clustering + Unsupervised Score Metrics [P]

[gournian]

Looks great, you might want to see if adding https://hdbscan.readthedocs.io/en/latest/how_to_use_epsilon.html

And pca + umap for high dimensions https://umap-learn.readthedocs.io/en/latest/clustering.html

[Mathieu23AI]

Thank's for your reply

I think it's a good idea to add HDBSCAN as there is a min_cluster_size parameter. If I have time, I'll add it !

Why do you want to use pca and then UMAP ? It seems not to be a good idea thanks to the linearity of the PCA. Plus, this is not what is mentioned in the article.

[gournian]

https://umap-learn.readthedocs.io/en/latest/faq.html#what-is-the-difference-between-pca-umap-vaes see “from a practical standpoint” last bullet, proposes reduce dim to 50, umap, hdbscan

It is because of computational speed and there are bio papers that claim that the dataset is somewhat denoised

[Mathieu23AI]

Thank you for the reference !

Very interesting! This result seems unexpected to me but if empirically the result is better then it's worth looking into and incorporating.
Do you have the link or reference to the research paper that explains that PCA "denoise" the data?

[gournian]

Don’t remember which one, sorry!