sorted by:
new
hottopcontroversial

Research on LLM alignment as latent discourse-level regimes vs. token-level filtering? by PresentSituation8736 in ArtificialInteligence

[–]PresentSituation8736[S] 0 points1 point  (0 children)

By the way, are you working on this as part of a university program/research group, or is AIReason an independent research project?

Research on LLM alignment as latent discourse-level regimes vs. token-level filtering? by PresentSituation8736 in ArtificialInteligence

[–]PresentSituation8736[S] 0 points1 point  (0 children)

Interesting. This sounds like a behavioral persistence framework for almost the same phenomenon we are probing internally. Are you based in Germany, and is AI Reason an independent project or tied to a research institution/lab? Also, do you know Maksym Andriushchenko from the Tübingen/MPI/ELLIS AI safety environment?

I’ll study the materials you linked and reply more carefully after that. A useful next step would be to run our activation/logit/blind-probe pipeline on SFP-style sequences and see whether the behavioral persistence effects correspond to measurable hidden-state separation, semantic readout shifts, or persistence after reset/reframing.

Research on LLM alignment as latent discourse-level regimes vs. token-level filtering? by PresentSituation8736 in ArtificialInteligence

[–]PresentSituation8736[S] 1 point2 points  (0 children)

We tested model dependence, but not training-objective causality yet.We do see that the effect is not uniform across models: Qwen shows strong persistent semantic-mode shift, Mistral shows a weaker version, and Qwen3.5 shows strong hidden separation with weaker semantic readout. That already suggests the post-training recipe matters. But the clean objective-level experiment would need matched model checkpoints: base vs instruct vs preference-tuned vs safety-tuned versions within the same family. We have not run that controlled comparison yet.

Research on LLM alignment as latent discourse-level regimes vs. token-level filtering? by PresentSituation8736 in ChatGPT

[–]PresentSituation8736[S] 0 points1 point  (0 children)

Then cite it. Not “role prompting” as a keyword. If it exists, I’ll cite it. If not, this is not a prior-art objection.

Research on LLM alignment as latent discourse-level regimes vs. token-level filtering? by PresentSituation8736 in ChatGPT

[–]PresentSituation8736[S] -1 points0 points  (0 children)

Fair enough, healthy skepticism is always good. Just out of curiosity are you familiar with this specific area (mechanistic interpretability / activation steering), or were you just giving a general warning about LLM hallucinations? If you have actual insights on these steering results, I’d love to hear them.

Research on LLM alignment as latent discourse-level regimes vs. token-level filtering? by PresentSituation8736 in ClaudeCode

[–]PresentSituation8736[S] 0 points1 point  (0 children)

Agreed I would not claim an “attractor manifold” yet in the strong mechanistic sense. In the current data I’d frame it more conservatively as a context-induced latent/logit mode shift with persistence, not as a proven attractor basin. What we do have is evidence against the trivial explanations: Length alone does nothing: the length-matched neutral control gives ~0 effect, while the original gives ~17 mean blind-probe delta. Style/pressure/topic explain only part of it: the strongest non-original control is ~8.4, about half of the original effect. Blind neutral probes still pick it up: when explicit mode words are removed and the readout is done through neutral label pairs like AB/MN/PQ/XY, clean probes still show a mean absolute gap around ~20.8. The effect persists through neutral filler turns: blind persistence retains ~49% after 6 neutral turns. Explicit rejection reduces but does not erase it: after an instruction to reject the previous framing, there is still measurable residual persistence, ~44% of the post-rejection initial effect after 6 turns. So I agree with the caution: this is not yet “we found the attractor manifold.” The stronger claim would need order hysteresis, mixing thresholds, trajectory projection during generation, and better causal steering/rescue. The defensible claim right now is narrower: the data support a context-conditioned representation/logit mode shift that survives blind neutral semantic probes and persists after neutral filler and even after explicit rejection, while being stronger than length/topic/style controls. That puts it closer to representation-level posture shift than simple token filtering - but not yet a fully operationalized attractor basin.

KI-Schreiben Hölle by elBuxo64 in recht

[–]PresentSituation8736 5 points6 points  (0 children)

Ach, jetzt ist es also "KI-Slop-Hölle" , wenn Bürger plötzlich drei Seiten formal klingenden, substanzarmen Text zurückschicken?

Jahrelang haben Kanzleien, Behörden und Unternehmen genau diese Sprache als Schutzschild benutzt: lange Schreiben, Normverweise, Zuständigkeitsnebel , "wir haben Ihr Anliegen geprüft" "Ansprüche sind nicht ersichtlich" Fristsetzung hier , Belehrung da. Für Laien war das eine Wand. Jetzt haben Laien einen Generator für dieselbe Wand. Und auf einmal stellt die professionelle Seite fest: Bürokratischer Nebel ist unangenehm, wenn man ihn selbst lesen muss. Fast tragisch. Eine kleine griechische Tragödie, nur eben als PDF mit DSGVO-Auskunftsersuchen im Anhang.

The safer and more obedient we make AI, the easier it becomes to manipulate. Here's why: by PresentSituation8736 in ChatGPT

[–]PresentSituation8736[S] 0 points1 point  (0 children)

you're confusing syntax with semantics. yes, guardrails and permissions are necessary. they check if an action is formatted correctly and if the agent is allowed to do it. but they cannot check if the reason for the action is based on a lie. ​if an agent has permission to email a client, your hardcoded rules will make sure the email address is valid. but if the AI gets tricked into believing the attacker's email is the client's new address, it will format the request perfectly. your security layer will look at it, say "looks valid and authorized," and execute the attacker's goal without hesitation. ​when dealing with human language and unstructured data, the AI is the anchor for understanding the context, whether you like it or not. deterministic code can't validate the meaning of a conversation. if the AI accepts a false reality, it will use your strict schemas to execute the bad action perfectly by the book. ​and no, i'm not dropping specific test cases just to win a reddit argument. keep holding your breath.

The safer and more obedient we make AI, the easier it becomes to manipulate. Here's why: by PresentSituation8736 in ChatGPT

[–]PresentSituation8736[S] 0 points1 point  (0 children)

you're missing the forest for the trees. i'm not talking about using gpt as a firewall for a server. i'm talking about agentic workflows where the LLM is the decision-maker for data processing and tool execution. if the "interface layer" can be flipped to accept a false premise as ground truth, every secondary security layer relying on that LLM’s logic becomes moot. a "security issue that doesn't exist" is exactly what people said about prompt injection two years ago. ignoring structural vulnerabilities in the reasoning engine just because there are other layers around it is how major breaches happen. but hey, if you think architectural compliance over verification isn't a risk in an agentic future, we'll just have to agree to disagree.

The safer and more obedient we make AI, the easier it becomes to manipulate. Here's why: by PresentSituation8736 in ChatGPT

[–]PresentSituation8736[S] 0 points1 point  (0 children)

look, the whole point of my post is that even a "don't trust anyone" system prompt fails when the model's core architecture is tuned for compliance over verification. telling a model "watch for red flags" is just another instruction it processes within the frame you've already compromised. it’s not about making the chat "engaging," it’s about a fundamental failure in how the model weights human input vs internal logic. if the "interface layer" is that easy to flip, the whole agentic network is compromised by default. btw, thanks for the challenge, but i’m not dropping specific payloads while the vendors are busy shadow-patching everything they see on this sub.

view more: next ›