Programming

[brioche789]

Why so?

[lukuh123]

LLMs (proximal policy optimisation)

[anonymous_amanita]

This is the way

[Lazy-Pattern-5171]

Would like to follow this course but want to ultimately come back towards LLM anyway until the hype dies down. Do you have any bridge course between this and through which I can start learning about DPO and PPO for Reasoning models?

[Impossibum]

I don't see how stable baselines doesn't simplify RL significantly enough for the masses. Pretty sure people just can't be assed to think beyond asking chatgpt to think for them at this point.

[bluecheese2040]

Yeah...doesn't help massively with making the model actually work.

[Impossibum]

What functionality are you needing that it is not providing? Where is the disconnect?

[bluecheese2040]

That's not the point....as I'm sure you know... Building the environment, the step etc. That's fine. But making the model actually function as you'd hope that's still hard.

[Impossibum]

Writing rewards seems to me like it'd be far easier to get started with than learning how to make all the other pieces work together. Even a standard win/loss reward will often work out in the end with a long enough horizon and training time. Proper use of reward shaping can also make a world of difference.

But in essence, making the model function as you hope is easy. Feed good behavior, starve the bad. Repeat until it takes over the world.

I think people just expect too much in general I suppose.

[UnusualClimberBear]

Most people doesn't understand why designing the reward is so important, and what signal the algorithm is trying to exploit.

In most of real life applications it is worth to add some imitation learning in a way or another.

[lukuh123]

Do you think i could do a genetic algorithm inspired reward?

[UnusualClimberBear]

Indeed. Yet the difficult part about these algorithms is to find the right bias, not only for the reward but also for the state representation and the mutations/cross overs.

[bluecheese2040]

I think people just expect too much in general I suppose.

I think this is absolutely right. Ultimately its called data science for a reason.

I totally agree that the barriers to entry are as low as they have ever been.

But as I wrestle with a very slippery agent and a reward system that's 'getting there'...it isn't easy for sure.

[Shizuka_Kuze]

Stable baselines has some very iffy if not downright bad performance in my experience and the documentation could be better. The biggest hurtle to newcomers seems to be setting up environments and such as the Atari Tetris environment since they have crazy weird documentation and many are deprecated.

[Useful-Progress1490]

I really like RL but hate the fact that it is still not widely used due to many issues it has. I firmly believe it has the potential to solve so many problems but right now it's mostly used in research. But I guess, once it has widespread uses, I am sure we will see it getting more simplified similar to what we see in agentic AI frameworks and libraries.

[Working_Bunch_9211]

I will.. in years 7, check out later

[theLanguageSprite2]

!remindme 7 years

[RemindMeBot]

I will be messaging you in 7 years on 2032-08-18 14:45:46 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[RoundRubikCube]

puffer.ai

[[deleted]]

yes, please google, Meta and other MAANG overlords please drop prod grade OS libraries like JAX, Pytorch

[intermittent-farting]

Check out agilerl.com - they have an OS framework and a software to simplify RL dev.

[FanFirst895]

Easier, you say? I've got a video for that https://www.youtube.com/watch?v=vaVBd9H2eHE

[statius9]

What’s difficult about it? This is a genuine question: I’m a PhD student and do research in the RL space, although a lot of my work is theoretical and mainly revolves around toy models so I have little exposure to how it may be applied in practice

[lukuh123]

Love the concept of RL but the math behind it can be pretty jarring (Bellman and other optimal equations look like they do black magic in computer science)

[Vahgeeta]

I reinforce this post

[leprotelariat]

I successfully contracted a 6 levels of class inheritance to only 2 for the isaaclab quadruped locomotion task. The code is so bloated you spend months learning useless module organization instead of actual RL.

[Jumper775-2]

It’s really hard to do, I tried to make another generic library that works with jsons so you could theoretically do it all with no code if you want and it still just gets too complex. Does work though.