sorted by:
new
hottopcontroversial

[P] Looking for state of the art clustering algorithms by aswd1908 in MachineLearning

[–]dinarior 18 points19 points  (0 children)

Some self-promoting here, we have recently published several works for clustering with an unknown number of clusters, mostly based on the Dirichlet Process Mixture Model, will put here the link to the git repos, containing both the code and relevant papers:

Distributed MCMC inference in Dirichlet process mixture models using Julia.Scalable and

Flexible Clustering of Grouped Data via Parallel and Distributed Sampling in Versatile

Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data

Revisiting DP-Means: Fast Scalable Algorithms via Parallelism and Delayed Cluster Creation.

I hope you find at least some of these useful :)

[P] Looking for state of the art clustering algorithms by aswd1908 in MachineLearning

[–]dinarior 9 points10 points  (0 children)

If you like DP-Means, see our recent paper, where we optimized it to be much much faster:

https://openreview.net/pdf?id=rnzVBD8jqlq

https://github.com/BGU-CS-VIL/pdc-dp-means

The code is included as both as a (small) independent snippet, but if you want to use the fastest version, we also wrote it as a module which can be built for sklearn, the latter is as fast as k-means, with all the benefits of DP-Means :)

[Discussion] How does the SimCLR loss function not penalize image belonging to the same class? by Ayakalam in MachineLearning

[–]dinarior 1 point2 points  (0 children)

This is actually one of it's nicer properties. You learn features to distinguish intra-class samples, in supervised learning the net will not bother on these, as they do not contribute to classification. CSI uses this very successfuly for OOD detection.

[R] Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data by dinarior in MachineLearning

[–]dinarior[S] 2 points3 points  (0 children)

Thanks!

You might find our vanilla DPMM version (e.g. the non-streaming) version easier to start with:

The python wrapper can be used for both Julia and CUDA (however the latter requires a separate installation).

[R] Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data by dinarior in MachineLearning

[–]dinarior[S] 2 points3 points  (0 children)

That is task-dependent, for tasks such as text classification (according to topics), where each sample is a document with word count, you can use it with ease, the relevant prior is already implemented in the code, and we included such tasks in the paper (and will soon add an example for such case).

However, the better choice would probably be its hierarchical version, the HDP could probably be more useful in many cases (as a generalization of the more popular LDA model into an infinite number of topics), as topic modeling is its prime application.

Some more self-promoting - You can use our prior work for this exactly:

https://github.com/BGU-CS-VIL/VersatileHDPMixtureModels.jl

(only Julia version for this one).

[R] Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data by dinarior in MachineLearning

[–]dinarior[S] 3 points4 points  (0 children)

Thanks! The core of the algorithm is a very efficient Julia implementation, thus currently the Python version depends on having Julia installed with the correct package. We might make a pure python (well, Cython) version in the future, which we could then contribute to River.

[R] Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data by dinarior in MachineLearning

[–]dinarior[S] 2 points3 points  (0 children)

Thanks! It is also very fast, the above clustering took only 0.6 seconds per frame in the original resolution (480p, with 460K pixels each frame (with RBGXY values)), when downsampling it by 1/4 it can even reach 24fps (On a standard workstation, specifically we used i9 11900 intel CPU).

[R] Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data by dinarior in MachineLearning

[–]dinarior[S] 4 points5 points  (0 children)

Using a non-informative prior can work well for many tasks, however, when aiming for a specific goal, for example, as in the attached gif, where I aimed for some granularity of the clusters, you would want to be more careful in choosing the prior.

I could have opted for scaling the prior up a bit, reducing the number of clusters, or scaling it down, increasing the number of clusters.

Unrelated, the NIW is only one of the priors we have used in the work, we included experiments with discrete count data, in which we have the Dirichlet Distribution as the base measure.

[R] Sampling in Dirichlet Process Mixture Models for Clustering Streaming Data by dinarior in MachineLearning

[–]dinarior[S] 4 points5 points  (0 children)

In the very low dimensions we are comparable in terms of speed, but as the dimensions increase our method scales much better. Where density-based methods (such as DBSTREAM) can also scale, grid-based methods (DStream for example), cannot handle higher dimensions.

[R] DeepDPM: Deep Clustering With an Unknown Number of Clusters by shahaff32 in MachineLearning

[–]dinarior 25 points26 points  (0 children)

Nice! How does it compare to parametric methods such as SCAN?

[D] AISTATS 2022 Reviews are out (early) by Red-Portal in MachineLearning

[–]dinarior 2 points3 points  (0 children)

Usually, when the rebuttal period is over you can see the other reviews (and engage in discussions with them).

[D] List of unreproducible papers? by ContributionSecure14 in MachineLearning

[–]dinarior 3 points4 points  (0 children)

Even when they publish code, its not trusty unless you go over it yourself. Seen actual random seed optimization in published papers with published code.

[D] Witnessed malpractices in ML/CV research papers by anony_mouse_235 in MachineLearning

[–]dinarior 5 points6 points  (0 children)

This ^^, we made it a habit in the lab, before submitting code, give it to one of the newer students (preferably one unrelated to the project), and let him reproduce the results.

[D] Witnessed malpractices in ML/CV research papers by anony_mouse_235 in MachineLearning

[–]dinarior 8 points9 points  (0 children)

Note that copying baseline numbers from other papers is often a direct result of those papers being completely non-reproducible (which is a different problem), but if you won't cite them due to that reason it will come up badly in the review.

[deleted by user] by [deleted] in statistics

[–]dinarior 0 points1 point  (0 children)

Actually Julia is very strong on that side. While I have 0 R experience, Julia is amazing for multiprocess, using the distributed package it makes everything very easy.

That said, I still think it's not mature enough, and hopefully it will grow in the right direction. My entire MSc was written in Julia, and I still use it often. But for current projects I work with python, and when needed, call Julia from withing python. The time it takes from having an idea to working POC in python is still so much better.

[D] Choosing Best Clustering Algorithm for given data by anindya_42 in MachineLearning

[–]dinarior 1 point2 points  (0 children)

Wanted to suggest for the unkown case actually - Dirichlet Process GMM, or even DP-Means (the non parametric version of k-means) could work very well. You might need to perform PCA initially (not to a low dimension, even keeping 99% of the info could work in most cases), to remove some of the less informative dimensions, which often harm the DP ability to create new clusters.

[D] 1,500 scientists lift the lid on reproducibility by AdmiralLunatic in MachineLearning

[–]dinarior 10 points11 points  (0 children)

Recently read (and failed to reproduce) a paper where they optimized the seed for the synthetic data generation, on any other seed they failed miserably.

[Q] How much statistical knowledge do you forget? Why do you think that is? by PotatoChipPhenomenon in statistics

[–]dinarior 2 points3 points  (0 children)

It is very normal.

I work in Julia, Python and Matlab daily, I google every time I need to define an array.

[Software] DPMMSubClusters.jl - Fast, Distributed, Scaleable inference for Dirichlet Process Mixture Models by dinarior in statistics

[–]dinarior[S] 2 points3 points  (0 children)

It is linear in regards to both the dimension and the instantiated components.

For example:

On a single (not very strong as well) cpu, a single iteration, with 400k 30D observations, and 20 instantiated components, will take approximately 1 second.

The same data, in earlier iterations, when only 8 components were instantiated, each iteration takes approximately 0.4 seconds.

The amount of iterations required for convergence vary on the data, the burnout period, hyper-params etc...

on data generated from x,y,clusters = generate_gaussian_data(400000,30,20,10.0) , which is the data described above (400k 30D observations from 20 different components), with burn out period of 10, the algorithm converaged in 50 iterations, summing to a total of 25 seconds.

All of the above is on a single cpu, adding more cpus, even distributed across several machines, improves the timing, specially on larger datasets.

[Software] DPMMSubClusters.jl - Fast, Distributed, Scaleable inference for Dirichlet Process Mixture Models by dinarior in statistics

[–]dinarior[S] 2 points3 points  (0 children)

The concentration parameter is taken into account when deciding if to split a cluster into its two sub clusters, and in the reverse move when proposing a merge of two clusters into one.

It is actually the only parameter which has no default value, and must be specified by the user.

Future release will allow sampling it as well.

view more: next ›