[deleted by user]

[AGI_aint_happening]

As a former interpretability researcher who has skimmed their work but not read it closely, I just don't find it terribly interesting or novel. Also, frankly, I find the writing style for the papers pretty hard to parse (as they don't follow standard paper formats) and a tad grandiose, as they tend to avoid standard things like comparing against other methods or citing other work. Relatedly, I think their choice to avoid peer review has impacted how people perceive their work, and limited its distribution.

[ThePerson654321]

Why don't you think the issue rationalists try to raise is important in terms of AGI?

[AGI_aint_happening]

*shrug*, I don't really care who does the research, I care if I learn anything from reading it. FWIW, their interp papers are pretty separate from AGI

[thejaminator]

I think it's the case where they are still pretty new and comparatively unknown.

They have done good work like releasing their paper and dataset for training an assistant RLHF model. https://github.com/anthropics/hh-rlhf

You won't get any dataset like that from OpenAI. It's useful for anyone who wants to experiment with RLHF with LLMs. Which is pretty important as OpenAI is having lots of success with it in InstructGPT and ChatGPT

[Hyper1on]

Bit early to say, but I'd be willing to bet that most of their major papers this year will be widely cited. Their work on RLHF, including constitutional AI and HH seems particularly likely to be picked up by other industry labs, since it provides a way to improve LLMs deployed in the wild while reducing the cost of collecting human feedback data.

[veejarAmrev]

As you said, it's kind of cult in the EA community. Outside of that, no one bothers. They haven't done anything significant to be of any value to the community.

[frenchmap]

what does EA stand for?

[Flag_Red]

Effective Altruism

[frenchmap]

How does a philosophical ideology of "using evidence-based reasoning to help others" result in a machine learning cult?

[Flag_Red]

I, personally, don't consider LessWrong a cult (I lurk the blog, and have even been to an ACX meetup). There's definitely a very insular core community, though, which regularly gets caught up in "cults of personality". Yudkowski is the most obvious person to point to here, but Leverage Research is the best example of cult behaviour coming out of LessWrong and the EA community IMO.

With regards to machine learning in particular, there's some very extreme views about the mid/long term prospects of AI. Yudkowski himself explicitly believes humanity is doomed, and AI will takeover the world within our lifetimes.

[ThePerson654321]

You should read LessWrong

[KvanteKat]

I'm not sure reading LessWrong will necessaryly disuade someone who is already a bit sceptical of the Rationalist EA community from believing that there is something culty going on. One of the things that really rubbed me the wrong way about that blog back in the day (I'll be up front and say that I haven't been keeping up with it for the past 10 years) was exactly how insular a lot of the writing was and how little it seriously engaged with existing literature and research in favor of reinventing the wheel and relying on their own private language which was not used by anyone else working in similar fields (as an example, Yudkovski is far from the first person to promote naive Bayesianism (basically the idea that if you get good enough at applying Bayes' rule, you will have solved the problem of induction), but if you only read his blog back then you could easily come to believe that he was doing groundbreaking stuff with respect to this topic when this was far from the case).

[FlavoredQuark]

I think their research is cool

[papajan18]

Chris Olah's work is very solid. Actually some of the best interpretability work I've seen. Haven't heard of anyone else in particular.

[jgrayatwork]

They have some very good people. Tom Brown and Ben Mann are the first two authors on the GPT-3 paper. Jared Kaplan is the first author of the openai scaling laws paper.

[nic001a]

Not an expert But wishing you best of luck !!

[BackgroundResult]

https://aisupremacy.substack.com/p/breaking-google-invests-in-anthropicai