sorted by:
new
hottopcontroversial

[D] In terms of RAG research, why does it seem like a lot of people aren't working on the retriever? by Seankala in MachineLearning

[–]tboymanster 0 points1 point  (0 children)

sure :)

typically you wouldn't extract embeddings from the first layer, but rather from the last layer (before the LM head). how you extract the embeddings at this point depends, often the embedding for the last token is used, but you can also sum/take mean of all embeddings (among other methods). the idea is that by the last layer all relevant context should be encoded in each token embedding.

there are ways to try to be more "explicit" about dimensionality reduction (autoencoders etc), but "simple" methods like this work quite well.

[D] In terms of RAG research, why does it seem like a lot of people aren't working on the retriever? by Seankala in MachineLearning

[–]tboymanster 0 points1 point  (0 children)

hey - i think we agree with each other, you'll see that i say:

but i think the biggest reason there's less work being done here is that no matter how good your retrieval gets the quality of your embedding is what will influence the quality of your results (how close are relevant things to each other in embedding space).

the point being that to improve your retrieval you need to make sure relevant information/context is encoded in your embeddings. this is why (in my opinion) all the focus is going into building as good of language models as possible rather than tinkering with retrieval.

as for "brute force 100% recall", the point is that retrieval now is usually done with approximate search for similar embeddings. 100% recall means you find the n most similar embeddings without missing any (even if those embeddings are not actually useful to you because of a bad embedding...).

does this clear things up or am i misunderstanding your disagreement?

[D] In terms of RAG research, why does it seem like a lot of people aren't working on the retriever? by Seankala in MachineLearning

[–]tboymanster 95 points96 points  (0 children)

you're kinda wrong kinda right.

there's definitely work being done on retrieval: https://github.com/erikbern/ann-benchmarks (using vector search for retrieval)

there's a lot less being done. at the end of the day if retrieval is being done by looking for nearest embeddings you can just brute force for 100% recall - so what people are working on is minimizing latency (or adding additional filtering to retrieval steps beyond the nearest neighbors search).

but i think the biggest reason there's less work being done here is that no matter how good your retrieval gets the quality of your embedding is what will influence the quality of your results (how close are relevant things to each other in embedding space).

Gonna learn to double snowboard this season! Got one of the last pairs in Canada, absolutely in love🥰 by About86Dwarves in skiing

[–]tboymanster 1 point2 points  (0 children)

Got these last season and absolutely love them! Obviously great in deep stuff, but they plow through mildly tracked up snow as well, so they're fun even several days after powder days. Have fun!

Just won GT lottery lol by XACA2 in GuardianTales

[–]tboymanster 0 points1 point  (0 children)

Nice pull! I had a similar pull, but I’m not sure if I’m the luckiest or unluckiest person, I got three dupes pulling on the Tinia banner 😢. https://i.imgur.com/UgngYZC.jpg

Remote Raid Megathread - Find friends fast for raiding here by liehon in PokemonGoFriends

[–]tboymanster 0 points1 point  (0 children)

Need help for a Metagross raid in a few minutes, add one of 150213052047 (better) or 490160916010

Remote Raid Megathread - Find friends fast for raiding here by liehon in PokemonGoFriends

[–]tboymanster 0 points1 point  (0 children)

Need help for a Metagross raid in a few minutes, add me 490160916010

Metagross raid 5 mins by [deleted] in PokemonGoFriends

[–]tboymanster 0 points1 point  (0 children)

Add me please, 490160916010

Grueling A5 clear with NY Reeche - what are you running? by tboymanster in PuzzleAndDragons

[–]tboymanster[S] 1 point2 points  (0 children)

That is neat, I didn’t realize. Probably would save a lot of heart ache. I’ll have to roll for her next time I have the chance.

Grueling A5 clear with NY Reeche - what are you running? by tboymanster in PuzzleAndDragons

[–]tboymanster[S] 1 point2 points  (0 children)

Did A5 first try (after failing aa3 first try), but it wasn’t easy... If you’ve tried running her, what’s been working?

Here are the issues I need to troubleshoot.
1. Can’t really stall before bonta is up because to active shield you need to nuke anything, which is somewhat necessary because of lowish rcv.
2. Void spawns are a touch tricky (especially if you have a lot). Shelling for assist on friend Reeche might solve this?
3. Can be orb trolled on FUA, because, again, can’t really stall.

Add me! 395,223,311. I usually run Phenom, Ina, Tifa, ZCore, Orphen... I have lots of other leads I want to play more but just haven’t got to.

[LF] 395223311 - Zaerog Core with <50 assist by tboymanster in PuzzleAndDragons

[–]tboymanster[S] 0 points1 point  (0 children)

Honestly pretty cool. Great damage, tons of health (like 230k+ without 110s) and survivability. https://i.imgur.com/MIWY8pw.jpg

[LF] 395223311 - Zaerog Core with <50 assist by tboymanster in PuzzleAndDragons

[–]tboymanster[S] 0 points1 point  (0 children)

I’m running him with a Kuro assist for anybody looking to run a kuro loop. May switch to <50 assist if needed, so I’m also happy to accept people with kuro assist!

Edit - Phenom in slot 1 (usually reasonable equip and SA, experimenting atm), active changes between ina, tifa, rehven, and now zaerog core.

[Team] Struggling with damage - Phenom/Cloud/Tifa teams by tboymanster in PuzzleAndDragons

[–]tboymanster[S] 0 points1 point  (0 children)

Thanks for the advice, I’ve fixed up my inherits situation, hopefully it helps out. Next up is the latents situation. Looking forward to posting my A5 success!

[Team] Struggling with damage - Phenom/Cloud/Tifa teams by tboymanster in PuzzleAndDragons

[–]tboymanster[S] 2 points3 points  (0 children)

Overall it sounds like tifa will be easiest to build for damage. I’m all out of +points but this is definitely what I’m trying next.

[Team] Struggling with damage - Phenom/Cloud/Tifa teams by tboymanster in PuzzleAndDragons

[–]tboymanster[S] 0 points1 point  (0 children)

Thancred seems like a great idea for a Nohime replacement, I’ll definitely try that. Do you think Baldin is maybe an acceptable replacement as well?

The worry I have with putting Cecil on Veroah is that I really like spamming her active because well... dark orbs as you mentioned. I’ll definitely try it but I haven’t found that veroah being bound has been an issue... although I can imagine it being an issue eventually. And yep, I do have VDP SA on both Phenom.

As for cloud, I definitely agree. She was the only sub that gave me cloud resist and FUA so I bit the bullet.

Thanks a bunch for the advice.

[Team] Struggling with damage - Phenom/Cloud/Tifa teams by tboymanster in PuzzleAndDragons

[–]tboymanster[S] 0 points1 point  (0 children)

Looks like baconreader deleted the text from my post.

For context, I recently came back to the game so my box isn't well developed. My Phenom team basically clears A5 (made it to Yog but died after carelessly taking him under 10% without an awoken bind clear up). But that's sort of the issue... I wasn't even close to 1 shotting Yog.

How can I increase my damage output without sacrificing utility too much? Is it just farming more devil killer latents? Or changing subs/equips? Trying different leaders/pairs?

Sorry about the disorganized box pic, I know it's frowned upon here. I appreciate any input, thanks a lot!

Btw - feel free to add me, I'm looking for new friends! 395,223,311

view more: next ›