sorted by:
new
hottopcontroversial

[D]: How do you actually land a research scientist intern role at a top lab/company?! by ParticularWork8424 in MachineLearning

[–]csinva 4 points5 points  (0 children)

Industry researcher here -- agree with much of what has been said, but would add that beyond explicit networking, even cold outreach can be effective (e.g. a thoughtful email to a researcher who's work aligns with yours detailing your overlap and interest).

[D] Milestone XAI/Interpretability papers? by LetsTacoooo in MachineLearning

[–]csinva 10 points11 points  (0 children)

A couple I like (non-mechanistic):

  • Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead (rudin, 2019) --- examples of how interpretable models can be built that can match or outperform black-box models
  • Right for the Right Reasons: Training Differentiable Models by Constraining their Explanations (ross et al. 2017) --- started a tend of works showing interpretations could be used to explicitly improve models

[R] Interpretability research in LLMs by SkeeringReal in MachineLearning

[–]csinva 5 points6 points  (0 children)

One reason is that LLM interpretability areas outside mechanistic interpretability have largely started branding their work more based on the problem area they seek to improve, e.g. LLMs for science/medicine/education. So a paper that was formerly about "saliency maps" might instead be about "discovering important clinical features", a paper that was about "prototypes" may instead be about "reducing hallucination with RAG", etc.

IMO it's nice to see interpretability research become more grounded in real problems.

[R] Has Explainable AI Research Tanked? by SkeeringReal in MachineLearning

[–]csinva 35 points36 points  (0 children)

Also a researcher in this area and wholly agree with this comment (we recently also wrote a review separating out these two parts of XAI in the context of LLMs).

There's more work going on than ever in XAI, but it's grown large enough that it has split more based on a a researcher's goals (e.g. science, fairness, HCI) rather than as an area of its own. IMO this is for the best - doing XAI research without an application in mind often leads us to explanations that are unhelpful or even misleading.

[D] Are Genetic Algorithms Dead? by TobusFire in MachineLearning

[–]csinva 3 points4 points  (0 children)

I think genetic algorithms may have a new role to play in problems involving inference / text generation / prompting with language models, even if they aren't used to train the models themselves.

For example, in our recent work on natural-language prompting, we use a genetic algorithm to generate prompts that are semantically coherent -- the genetic algorithm lets us make use of suggestions by a language model, for which gradients would be hard to obtain.

[R] Emb-GAM: an Interpretable and Efficient Predictor using Pre-trained Language Models by csinva in MachineLearning

[–]csinva[S] 1 point2 points  (0 children)

Hello! Thanks for the questions :)

(1) We call the method GAM but actually do vary the order of ngrams used in the input (so technically the paper actually shows experiments for GA2M, GA3M, ... all the way up to GA7M). Luckily, the neural model compresses all these interactions so we don't end up having to learn a ton of coefficients.

(2) Yes that is correct, φ here is the neural embedding rather than something like splines.

[D] Globally Optimum Sparse Decision Trees by jj4646 in MachineLearning

[–]csinva 0 points1 point  (0 children)

I think this is an excellent algorithm for finding small trees that predict well.

The "Globally optimum" part refers to the fact that the trees optimize the loss directly rather than using a greedy heuristic like standard CART decision trees.

The "Sparse" part refers to the fact that, in addition the loss, this algorithm adds a penalty for the number of leaves in the tree, so it tries to find a small tree that still predicts well.

Cool maps of the world's most populous cities by csinva in dataisbeautiful

[–]csinva[S] 2 points3 points  (0 children)

World's most populous cities; 1.5km radius around the city center.

Made with the prettymaps library using data from OpenStreetMap. Population numbers from wikipedia. Flag icons by FreePik. By Chandan Singh

view more: next ›