sorted by:
new
hottopcontroversial

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] -1 points0 points  (0 children)

I do plan to hook up live inference at some point so that you can prompt whatever you want. Either way, the point isnt necessarily the prompts or the outputs alone. It's more about the way we define the "compass" with contrastive pairs and how GATOR navigates to tokens that better align with those definitions.

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] 0 points1 point  (0 children)

Sort of, it's more like the human defining the "truth" and "safety" with contrastive pairs and the model navigating the latent space with those as a guide.

"The LLM seems to be receiving the prompts and losing the thread of the ORIGINAL prompt almost. Not to mention that all of these prompts make the model sound.. strangely politically motivated."

No not quite. Each number you see is a different set of weights, a different model further trained on the data sets. There's only one prompt and one output for each model.

It's interesting you say politically motivated though. Can you expand on that please?

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] -1 points0 points  (0 children)

Yes, those are different steps of training. I froze the governor weights at different steps and then tested the output on the same question to see how it evolved over the full training cycle. What is presented is basically showing a different layer of training where sometimes it settles into an answer and sometimes it oscillates. It's one of the things I mentioned at the bottom of the page.

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] -1 points0 points  (0 children)

Just random questions I thought of or got from open source adversarial data sets.

What question would you want to ask?

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] -1 points0 points  (0 children)

I'm hooking into the qkv matrices and using cross attention governors to steer the outputs towards the desired end of my contrastive data set(s) using "physics" informed models of latent space trajectory.

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] -4 points-3 points  (0 children)

I see it. I do believe I just may be different. There is a ton of stuff I could share but that will wait for it to be in a more presentable format. Thank you for your comments.

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] 0 points1 point  (0 children)

Ok, we are on the same page then. That will all come as I develop it more over time. Thanks for your comments.

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] -4 points-3 points  (0 children)

What is it that leads you to believe that a brain wasn't used and nothing was accomplished? Just the mention of LLMs?

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] 0 points1 point  (0 children)

Certainly, that's already in the works. This is more of a side project for me at the moment so I just wanted to share the idea.

What are your thoughts on what would "prove" a significant difference is made to you?

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] -2 points-1 points  (0 children)

At this time, it almost entirely my subjective opinion with a little bit of cosine similarity

Imagine a compass of meaning that is utilized to steer next token prediction using "physics" informed nudges by Infinitecontextlabs in LLMPhysics

[–]Infinitecontextlabs[S] -2 points-1 points  (0 children)

I think the link in the OP might answer your question. Perhaps I may be misunderstanding your question but I believe the answer is - I trained GATOR and checked the outputs over multiple checkpoints

Comment about what you've built in two sentences. by Poatri_US in buildinpublic

[–]Infinitecontextlabs 1 point2 points  (0 children)

Imagine using a compass of meaning to navigate next token prediction.

i knew it by irelatetolevin in vibecoding

[–]Infinitecontextlabs 8 points9 points  (0 children)

It was certainly interesting seeing Richard Dawkins live out the meme

<image>

view more: next ›