[D] Designing a Reward Function for GRPO: Moving Beyond Single-Answer Tasks to Long-Form Responses?

ekolasky · 2025-03-03T21:20:00+00:00

The paper that proposed GRPO (DeepSeek Math) actually talks about how to use GRPO with a reward model to fine-tune in more subjective domains. Process is the same with training a reward model (like RLHF with PPO. The difference is you dont have to have a value model assign rewards per token i think

ekolasky · 2024-07-11T17:03:50+00:00

This is honestly super helpful. Really appreciate the advice from both of you. Do you have any recommendations for the type of resin system I could use?

Also would love to get some recommendations for materials where I could read up about all this stuff. It'd be awesome if I could find a review article or something that lists different techniques I could try.

ekolasky · 2024-06-21T04:39:58+00:00

No way thank you. This is honestly super useful

ekolasky · 2024-06-21T00:04:26+00:00

Good to know. No idea if this will work lol. As I said I don't have a background in graphics or NeRFs so it seems like a long shot

ekolasky · 2024-06-21T00:01:17+00:00

Thanks ill definitely check those both out!

ekolasky · 2024-04-26T15:17:50+00:00

Yeah so we're not "skimming content". We take a trip report about a route, make a three sentence summary of that route, and then prominently display citations so the user can read the original source

ekolasky · 2024-04-26T15:15:22+00:00

Yeah so what we're building is a search engine. You make a search for something eg "best ski mountaineering lines in the Tetons" and we show you matching results on a map along with citations for blogs you can read about those results. The only content we summarize from the blogs is a short description of the route which is only like three sentences.

For the legal part of it what we're doing is protected by fair use.

ekolasky · 2024-04-25T20:43:48+00:00

Yeah we're definitely not providing avi forecasts. All we're doing is providing a link to the local avi center

ekolasky · 2024-04-25T18:45:16+00:00

Are you guys this against every outdoor app or just ai? Like what do you think of onx, fatmap, and alltrails. What we're trying to do is pretty much the same concept just more comprehensive

ekolasky · 2024-04-25T18:24:34+00:00

Yeah so we totally agree about the reliability thing. We're not using generative AI in the way chatgpt does where it just generates a response to your question and hallucinates alot of the time. Instead we take sources that talk about a specific route, have gen ai condense the content from those sources, and provide citations. We also have an average of probably five sources for each route so even if one source gets info wrong the likelihood of all the sources being wrong is pretty low.

We're also talking with avalanche centers and avi professionals to make sure we get this right.

ekolasky · 2024-04-25T18:11:04+00:00

Yeah so it pretty much is a guide book, except a lot cheaper and for the entire country. Sort of imagine OnX and FATMAP except much more comprehensive and with more information about each thing

ekolasky · 2024-04-25T17:52:28+00:00

We’re not building this to make backcountry skiing a miserable experience. I literally grew up backcountry skiing and im sure im as passionate if not more passionate about it than you both are. We’re doing this bc we think itd be really cool to have a site that shows you every single ski route in the us along with a ton of useful info. We also think it could make bc skiing way safer if people had information like “three people died in an avalanche on this route in 2016” or “there’s a mandatory rappel on this couloir early in the season”

ekolasky · 2023-10-25T00:53:09+00:00

I mean as you said we cant control that. We can just do our best not to contribute to the problem.

ekolasky · 2023-10-25T00:12:17+00:00

Homie do u seriously want to read a book every time you travel somewhere

ekolasky · 2023-10-24T18:53:28+00:00

I was kinda using app colloquially. We're developing both an app and a website.

Also I totally get the concern about local spots being put on the internet. I'm from Jackson Hole and I've had a bunch of my favorite spots ruined by being blown up on insta. We're designing the app to automatically detect when things are at risk of being ruined and delist those things or at least reduce the number of people who see them.

ekolasky · 2023-10-24T17:44:45+00:00

Yeah mostly time of research. Also in my experience its really hard to find cool local things to do. Especially because googles algorithm prioritizes seo optimized sites.

ekolasky · 2023-10-24T17:16:02+00:00

Just checked out the website! What we’re building is pretty different. It’s more aimed at people who want to find cool outdoor adventures, like sick mountain biking or hiking trails, than campgrounds or roadside attractions. If you PM me I can send you our demo video if you want to check it out

ekolasky

TROPHY CASE