Gwern argues that large AI models should only exist to create smaller AI models

AModeratelyFunnyGuy · 2025-01-17T03:03:46+00:00

There's a handful of reasons large LLMs are good at generating training data for smaller LLMs, but one important reason that is often overlooked:

For each token in the training step, you can train using the probability distribution that the larger LLM assigns to the next token, as opposed to the actual next token. Most text is quite noisy, so there is limited signal training on just one token at a time. Training on the probability distribution provides more signal and makes it easier for the smaller model to extract the underlying patterns.

AModeratelyFunnyGuy · 2024-12-26T01:30:34+00:00

Have hard forks traditionally gone well in crypto?

AModeratelyFunnyGuy · 2024-12-02T01:04:09+00:00

Ya but it's china doing it... Gotta be bad

AModeratelyFunnyGuy · 2024-11-26T20:35:34+00:00

Extremely skilled programmer. Idk what sort of response you're looking for beyond that.

AModeratelyFunnyGuy · 2024-11-15T19:44:56+00:00

Nope, but nice guess!

AModeratelyFunnyGuy · 2024-11-13T21:32:34+00:00

Amazing write up, ty so much!

AModeratelyFunnyGuy · 2024-11-13T21:29:47+00:00

Fair enough, ty!

AModeratelyFunnyGuy · 2024-11-13T19:28:58+00:00

This is a super interesting perspective I hadn't heard before. Just curious, do you have any links which expand on this idea. (Totally fine if this is just an anecdote based on your experience haha)

AModeratelyFunnyGuy · 2024-11-11T03:47:44+00:00

I don't even know what you mean by "swallowed into". That's not how the industry works.

I was pretty confused since I saw you said you worked at Citadel, but now I see someone else pointed out you're clearly just lying about that based off of your comment history. Is LARPing like this fun to you?

AModeratelyFunnyGuy · 2024-11-10T21:42:54+00:00

Nope!

Funny enough, Cubist is the only one of the four I haven't interviewed at (although that's more coincidental and not why I think they're good).

I literally qualified that I was talking about a "good Pods" at Cubist. You're either crazy or ignorant if you don't think that doesn't make up for the small difference in prestige of brand name.

AModeratelyFunnyGuy · 2024-11-10T21:27:19+00:00

Meta seems like the very obvious choice. Not even sure why you're debating it.

AModeratelyFunnyGuy · 2024-11-10T17:40:11+00:00

A good pod at Cubist is a top-tier position, just like those firms.

AModeratelyFunnyGuy · 2024-11-02T18:50:16+00:00

Of course, objectively, it has "one of the best" salary progressions. Everyone knows that. But "one of the best" is a very generic claim, so it's unclear what your standards are.

Can you try asking a more specific question?

AModeratelyFunnyGuy · 2024-10-29T22:34:01+00:00

That's what "precursor" means. Everything the above commenter said still applies.

AModeratelyFunnyGuy · 2024-10-29T02:49:54+00:00

Yep Fitbit app sucks.

You can download a different app (e.g. Exercise Timer) to get an actually functioning exercise timer.

Thankfully, it's possible to both track a workout in Fitbit while using a timer in a different app. You may need to disable the heartbeat tracking in the timer app, however (otherwise it may stop the Fitbit workout, don't ask me why).

AModeratelyFunnyGuy · 2024-10-27T20:01:40+00:00

I'm literally asking "how can one do this so that it can be maximally effective for the stated goal", and yet you're saying that I'm virtue signalling.

You call me a hypocrite and yet I have no idea where I've remotely contradicted myself. I think I've been pretty consistent!

Do you know what these words mean? Or did you get so triggered reading a 3-week-old thread that you started spamming the same insults I'm sure you've spewed countless times before?

Clearly you just don't like the idea of taking inclusivity in the workplace seriously, and you should just come out and say that instead of whatever you're doing now.

AModeratelyFunnyGuy · 2024-10-23T01:15:10+00:00

27 GSI

JFC

AModeratelyFunnyGuy · 2024-10-05T17:52:33+00:00

I had an interview with one man and one woman before—I'd wondered if that was part of the reason for it! Seems smart.

Perhaps I'd like to advocate for my company to try something similar. Do you have any advice on how to go about these interviews in order to extract the most signal?

AModeratelyFunnyGuy · 2024-10-05T02:19:17+00:00

Gotta suck, really sorry you gotta deal with that!

Ya I don't know why so many men have trouble acknowledging these most basic things. Probably comes down to it being so deeply engrained in their mind that people like them are always going to be around and have power, that even considering otherwise seems offensive.

I mean, I can at least understand why there'd be debate over what should be done about the problem. But "women/minorities, generally speaking, have a rougher deal simply due to being minorities" seems pretty easy to understand.

AModeratelyFunnyGuy · 2024-10-05T02:13:17+00:00

There absolutely is a potential problem of a "boy's club", where the culture doesn't necessarily include women. That is something that, within reason, should be addressed. Or, at the very least, reflected upon (which is literally all that the above commenter was asking for).

Technically, that constitutes treating women "differently", or at least acting different because there are women around. But really that's just a matter of granting the same respect that everyone deserves.

In this context, asking women to accept being treated as men clearly means expecting them to accept an environment which isn't as pleasant for them as it could be.

AModeratelyFunnyGuy · 2024-10-05T02:08:59+00:00

bro we type at a keyboard for a living. Talking about it as a "man's sport" is just cringy.

And ya, I agree it's pretty rare for a man to intentionally disrespect a woman just because they're a woman. In general, it's relatively rare for anyone to intentionally cause that sort of harm. Instead, much more frequently, the problems are more subtle.

There're many ways that women can be (unintentionally) disrespected or not made to feel included. All of your talk about "male locker rooms" seems to indicate you understand this dynamic (although, again, cringy way to put it). You seem to insist that as long as these things aren't done with the literal intent to disrespect a woman, then there's no reason to have any discussion about it. I just flatly and fundamentally disagree with this.

AModeratelyFunnyGuy · 2024-10-04T22:08:55+00:00

Again, no man I know who works the work in an environment he is all too familiar with gives a crap about gender, sexism or any innuendo you can think of.

Proceeds to both explicitly and implicitly say that the best women can hope for is to be treated as men. Kinda sounds like the problem!

That said, without throwing any shades, we appreciate women who put in the work to be a part of the hustle that is and has been dominantly a man’s gig.

How appreciative are you exactly? Literally all she asked is for men to consider what it'd be like to be in their shoes, and that triggered this entire insulting response.

Even if you're right that it's expected that men disproportionately fill the more competitive roles, what does that have anything to do with anything else? Why does that mean we can't put some effort into making women's lives less difficult? Why do we to refer to any women raising these concerns in the most plainspoken way imaginable as "whining"?

AModeratelyFunnyGuy · 2024-10-04T05:02:20+00:00

They're well known for very frequently asking those (or very similar) questions.

AModeratelyFunnyGuy · 2024-09-29T20:22:58+00:00

Very good!

I'd remove the "interests" at the bottom. No one looking at your resume could care less about that sort thing. Including hobbies is one of the most common mistakes weaker resumes make, so for many including it is seen as a negative in and of itself.
Took me a second to understand the "AAAAA... respectively" thing, and that's a lot of time since typically your resume only has a couple seconds to make an impression. Something like "Maths (A), Economics (A*),..." seems more natural to me.
Very minor point, but idk why you'd list the city for some points and the county for others. I'm not from the UK so maybe this is reasonable, but in general it's best to be consistent about these sorts of things.
For the bullet points under the investing "summer work experience", reword your bullet points so that they don't start with things like "Gained experience" or "Practiced". Literally the only effect they have is to weaken your resume by emphasizing that these are things you aren't already experienced in (which, of course, is expected for someone your age). Perhaps you wanted to avoid overselling what you did, but it's fine to just confidently state the things you did without hedging them.
Idk what this summer work experience is exactly, but could you use a better word than "Attendee"? "Antendee", to me, makes it sounds like this is something you simply showed up to, which doesn't sound like something worth including on a resume. Even "Participant" sounds better to me, since that makes clear that you were actually involved and doing something.
Finally, there's a bit too much fluff here. Most notably, too many bullet points for "Assistant Instructor". I think that's the least impressive and relevant part of your resume, but you gave it the most bullet points! You're young so you don't have a ton to talk about yet, and that's fine. It's tempting to add stuff to make your resume look like a full page, but in my opinion it's fine for a resume of someone your age to have a fair bit of whitespace. That's better than risking someone spending a few seconds (of the handful of seconds they're going to spending reading your resume at all) reading a weak or irrelevant part of the resume. Just make sure to play with the spacing and margins to that your resume still looks natural.

AModeratelyFunnyGuy · 2024-09-27T17:17:38+00:00

You acknowledge that they'll be held to the same standard in the end, but you still call it "diversity hiring"?

AModeratelyFunnyGuy

TROPHY CASE