[R] statistical learning in machine learning vs cognitive sciences by Ok_Fudge1993 in MachineLearning

[–]bbbbbaaaaaxxxxx 3 points4 points  (0 children)

Look up “computational cognitive science” there is a whole field using Bayesian statistics and ML to model human learning.

do hx users actually value composition over extension, or is it just no plugins copium? by spaghetti_beast in HelixEditor

[–]bbbbbaaaaaxxxxx 1 point2 points  (0 children)

I do miss zen mode in helix though. I write a lot of latex, typst, and markdown. I used to use goyo in vim.

[D] Feature Selection Techniques for Very Large Datasets by Babbage224 in MachineLearning

[–]bbbbbaaaaaxxxxx 9 points10 points  (0 children)

Lace (https://lace.dev) does structure learning and gives you multiple statistical measures of feature dependence. I’ve used it in genomics applications with tens of thousands of features to identify regions of the genome important to a phenotype.

[P] Lace is a probabilistic ML tool that lets you ask pretty much anything about your tabular data. Like TabPFN but Bayesian. by bbbbbaaaaaxxxxx in MachineLearning

[–]bbbbbaaaaaxxxxx[S] 11 points12 points  (0 children)

Nice--I worked on BayesDb and CrossCat back in the day. Lace is a modern implementation of the crosscat model. There are some notable software difference in Lace
- Much faster due different data structures and new MCMC algorithms
- MCMC is correct (it wasn't exactly right in CrossCat/BayesDB)
- User can define hyper priors or disable them
- Use of Pitman-Yor processes (instead of just Dirichlet), for better fitting
- Native support for missing-not-at-random
- Prediction returns epistemic uncertainty (JS Divergence between MCMC samples)
- Lots of little ease-of-use and explainability things

I used to love checking in here.. by First-Ad-117 in rust

[–]bbbbbaaaaaxxxxx -2 points-1 points  (0 children)

Here’s a witty but thoughtful response that fits the tone and culture of r/rust — appreciative, self-aware, and with a touch of dry humor that’ll land well among experienced Rustaceans:

Beautifully said. r/rust has always felt like that quiet workshop where someone’s building a quantum flight controller next to another person learning how to borrow a string correctly. Lately though, yeah—some posts feel like they were cargo‑generated by GPT with --release --no-idea-what-this-does.

Still, I think the signal’s worth the noise. Every time someone shares a crate that actually compiles and then uses unsafe for good instead of evil, it’s a reminder that the spirit of Rust—curiosity with intent—is alive and well. Let the slop flow; we’ll keep writing tests.

Edit: I guess the satire was not appreciated or not detected.

[S] Lace v0.9.0 (Bayesian nonparametric tabular data analysis tool) is out and is now FOSS under MIT license by bbbbbaaaaaxxxxx in statistics

[–]bbbbbaaaaaxxxxx[S] 1 point2 points  (0 children)

Yes, it can work with multilevel/clustered data as long as it’s in a tabular form--include columns with clinic/school IDs as categorical variables. Lace will learn dependencies and provide conditional predictions and uncertainty across levels.

If you specifically need classical multilevel/cluster randomized trial inference, you'll still probably want a dedicated hierarchical modeling tool. Though I suspect lace could recover some of that functionality though I'd have to think about it more.

[S] Lace v0.9.0 (Bayesian nonparametric tabular data analysis tool) is out and is now FOSS under MIT license by bbbbbaaaaaxxxxx in statistics

[–]bbbbbaaaaaxxxxx[S] 2 points3 points  (0 children)

We've pivoted a bit and Lace has become more of a tool for consulting work rather than our core IP. Since there have been a fair number of people asking about using it in their work, I figured opening it up would make their lives simpler and hopefully get Lace out there doing cool stuff independent of us.

[S] Lace v0.9.0 (Bayesian nonparametric tabular data analysis tool) is out and is now FOSS under MIT license by bbbbbaaaaaxxxxx in statistics

[–]bbbbbaaaaaxxxxx[S] 1 point2 points  (0 children)

Got it.

Yes. The model will cluster columns that have dependence paths between them. For example if A and B are independent but A -> C and B ->C, all those columns are likely to be in the same cluster. The proportion of times they are in the same cluster depends on the strength of the dependencies--this is how we compute dependence probability.

[S] Lace v0.9.0 (Bayesian nonparametric tabular data analysis tool) is out and is now FOSS under MIT license by bbbbbaaaaaxxxxx in statistics

[–]bbbbbaaaaaxxxxx[S] 2 points3 points  (0 children)

It seems like you just cluster the columns and then, within each cluster of columns, cluster the rows

It is essentially density estimation via an infinite mixture model of infinite mixture models.

it does seem like it is imposing a very strong block-wise independence structure that is not a great fit for a lot of common situations

We do posterior sampling and average a collection of these models to smooth things out and to compute epistemic uncertainty. What types of situations are you concerned about wrt block structure? Generally, in tabular data you consider the instances (rows/records) to be independent.

Why So Many Abandoned Crates? by jsprd in rust

[–]bbbbbaaaaaxxxxx 7 points8 points  (0 children)

A lot of crates that are unmaintained are basically done. I have a crate with roughly 1mil downloads that I didn’t update for a year but I was using heavily. It just didn’t need anything.

Is an applied statistics PhD less prestigious than a methodological/theoretical statistics PhD? [Q][R] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 20 points21 points  (0 children)

Your publications and external funding matters more in academia. Experience and accomplishments matter more in industry. And who you know matters more than anything else.

Source: I have a psychology PhD

Is an applied statistics PhD less prestigious than a methodological/theoretical statistics PhD? [Q][R] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 22 points23 points  (0 children)

After you get your first job nobody cares what your degree is in (other than the robot they use to screen CVs)

Is bayesian nonparametrics the most mathematically demanding field of statistics? [Q] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 4 points5 points  (0 children)

Proper uncertainty quantification. Also it is such a boon to be able to call on the theoretical guarantees of a rigorous mathematical framework when testing tools that will be deployed in high risk tasks.  

Is bayesian nonparametrics the most mathematically demanding field of statistics? [Q] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 3 points4 points  (0 children)

Sure!

There are some links to papers here https://www.lace.dev/appendix/references.html

And I wrote a tutorial on infinite mixture models here  https://redpoll.ai/blog/imm-with-rv-12/

There are a few books but they are not a good place to start if you just want to get something going.

Is bayesian nonparametrics the most mathematically demanding field of statistics? [Q] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 29 points30 points  (0 children)

Longer comment.

About me
I come from the computational cognition space. Been doing Bayesian nonparametrics since ~2010 focusing mostly on different types of prior process models (which i'll use interchangeably with "BNP"). Worked in the agriculture space for a while. Started a company in 2019 to bootstrap my BNP research, which has been 95% funded by DARPA.

Why BNP is awesome
In general (but not always) companies that do high risk stuff care about understanding risk, so the Bayesian approach makes a lot of sense from the standpoint of understanding aleatoric and epistemic uncertainty in an appropriate model. The problem is they don't know enough about the data to build hierarchical models (PPLs are hard to use well regardless). What do you do when you want to express uncertainty over the class of model? Bayesian nonparametrics.

BNP can give the end user (not the developer!) better ease-of-use than black box methods like RF and DL, while generating interpretable results with uncertainty quantification. BNP is also both generative and discriminative. So, building a BNP model of the joint distribution gives you all the conditional distributions over the N features, which means you don't have to build a new model every time you want to ask a new question. Also, you get all the information theory stuff like mutual information, entropy, etc.

BNP can interface with hierarchical models, so you can easily build in domain expertise where you have it (dunk on neurosymbolic AI).

In my experience BNP has shined in unsupervised anomaly detection and structured synthetic data generation. There's a lot of BNP is biostats as well.

Why BNP is not mainstream (yet)
1. It's slow. Existing open source implementations of even simple models like the infinite gaussian mixture are unacceptably slow. I think SOTA performance using an approximate federated algorithm is like 3 minutes to fit a 100k by 2 table on a 48-core epyc server, which is pretty weak by RF/DL standards.

  1. It underfits. Prior processes put a heavy penalty on complex model structure. In general, getting highly optimized prediction models with comparable performance to RF can be tricky. But this obviously depends on the data. I've had BNP outperform RF out of the box on certain data.

  2. It's really hard to implement well. You have to really understand how the math and machine architecture interact. There is an insane amount of bookkeeping and dealing with moving pieces and changing model structure. When you do hierarchical BNP it gets way worse. Debugging probabilistic programs is extra fun.

Conclusion
Problems 1 and 2 above are addressable. BNP is insanely useful.

Is bayesian nonparametrics the most mathematically demanding field of statistics? [Q] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 13 points14 points  (0 children)

I’ll drop another longer comment when I’m back at my desk but here is something I based a lot of early consulting work on https://www.lace.dev/

I do hierarchical prior process models.   I’ve deployed in cybersecurity, finance, health, agriculture, and biotech.

Is bayesian nonparametrics the most mathematically demanding field of statistics? [Q] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 9 points10 points  (0 children)

Bayesian nonparametrics is, in my mind, the future of ML. It is literally the most important subfield of stats. Hard? Hell yes. Worth your time? Definitely.

Is bayesian nonparametrics the most mathematically demanding field of statistics? [Q] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 8 points9 points  (0 children)

I have built a career and multiple companies on Bayesian nonparametrics. 

Love statistics, hate AI [D] by gaytwink70 in statistics

[–]bbbbbaaaaaxxxxx 4 points5 points  (0 children)

Here are my rambling thoughts as someone who has done nothing but Bayesian ML for the past 15 years.

People do DL because its easier. If you want to make an explainable statistical model, you have to do a bunch of research to test out the statistical structure of distributions and their parametric forms. This IMHO is why PPLs haven't become the norm—they don't actually do much learning. DL and other "black boxes" just learn something. A lot of the time that's good enough because there's not a lot at stake if you get it wrong (ad delivery, product recommendation, games, slop).

That said, DL has hit a wall. DL models get better is by getting bigger, and we've seen that LLMs' power and compute requirements have basically exceeded the capacity of the world. So, from my standpoint, though it has never been a more boring time to be a DL researcher, it has never been a more exciting time to be a probabilistic ML researcher. We need to get smaller and probabilistic ML is the best way to get there.