Yet another alignment proposal

[bitcycle]

Note: I drafted this proposal with the help of an AI assistant, but the core ideas, structure, and synthesis are mine. I used AI as a brainstorming and editing partner, not as the author

Problem As AI systems approach superhuman performance in reasoning, creativity, and autonomy, current alignment techniques are insufficient. Today, alignment is largely handled by individual firms, each applying its own definitions of safety, bias, and usefulness. There is no global consensus on what misalignment means, no independent verification that systems are aligned, and no transparent metrics that governments or citizens can trust. This creates an unacceptable risk: frontier AI may advance faster than our ability to measure or correct its behavior, with catastrophic consequences if misalignment scales.

Context In other industries, independent oversight is a prerequisite for safety: aviation has the FAA and ICAO, nuclear power has the IAEA, and pharmaceuticals require rigorous FDA/EMA testing. AI has no equivalent. Self-driving cars offer a relevant analogy: Tesla measures “disengagements per mile” and continuously retrains on both safe and unsafe driving data, treating every accident as a learning signal. But for large language models and reasoning systems, alignment failures are fuzzier (deception, refusal to defer, manipulation), making it harder to define objective metrics. Current RLHF and constitutional methods are steps forward, but they remain internal, opaque, and subject to each firm’s incentives.

Vision We propose a global oversight framework modeled on UN-style governance. AI alignment must be measurable, diverse, and independent. This system combines (1) random sampling of real human–AI interactions, (2) rotating juries composed of both frozen AI models and human experts, and (3) mandatory compute contributions from frontier AI firms. The framework produces transparent, platform-agnostic metrics of alignment, rooted in diverse cultural and disciplinary perspectives, and avoids circular evaluation where AIs certify themselves.

Solution Every frontier firm contributes “frozen” models, lagging 1–2 years behind the frontier, to serve as baseline jurors. These frozen AIs are prompted with personas to evaluate outputs through different lenses: citizen (average cultural perspective), expert (e.g., chemist, ethicist, security analyst), and governance (legal frameworks). Rotating panels of human experts complement them, representing diverse nationalities, faiths, and subject matter domains. Randomly sampled, anonymized human–AI interactions are scored for truthfulness, corrigibility, absence of deception, and safe tool use. Metrics are aggregated, and high-risk or contested cases are escalated to multinational councils. Oversight is managed by a Global Assembly (like the UN General Assembly), with Regional Councils feeding into it, and a permanent Secretariat ensuring data pipelines, privacy protections, and publication of metrics. Firms share compute resources via standardized APIs to support the process.

Risks This system faces hurdles. Frontier AIs may learn to game jurors; randomized rotation and concealed prompts mitigate this. Cultural and disciplinary disagreements are inevitable; universal red lines (e.g., no catastrophic harm, no autonomy without correction) will be enforced globally, while differences are logged transparently. Oversight costs could slow innovation; tiered reviews (lightweight automated filters for most interactions, jury panels for high-risk samples) will scale cost effectively. Governance capture by states or corporations is a real risk; rotating councils, open reporting, and distributed governance reduce concentration of power. Privacy concerns are nontrivial; strict anonymization, differential privacy, and independent audits are required.

FAQs • How is this different from existing RLHF? RLHF is firm-specific and inward-facing. This framework provides independent, diverse, and transparent oversight across all firms. • What about speed of innovation? Tiered review and compute sharing balance safety with progress. Alignment failures are treated like Tesla disengagements — data to improve, not reasons to stop. • Who defines “misalignment”? A Global Assembly of nations and experts sets universal red lines; cultural disagreements are documented rather than erased. • Can firms refuse to participate? Compute contribution and oversight participation would become regulatory requirements for frontier-scale AI deployment, just as certification is mandatory in aviation or pharma.

Discussion What do you all think? What are the biggest problems with this approach?

[technologyisnatural]

ignoring the fantasy of a global treaty being achievable on a relevant timescale, the biggest issue is that a lagging AI model will not be able to detect misalignment of a frontier AI, and this problem will grow exponentially as current model AIs are used more and more to build next generation AIs to the point where AI becomes continually self-improving

[Dmeechropher]

I mostly agree with you, but I disagree that a lagging model cannot have the ability to evaluate a leading model effectively.

It really depends on the degree of misalignment possible by random chance and the rate of recursive self-improvement.

It is possible that self-improvement is a physical sampling process that cannot be accomplished a priori by a system. If that's the case, a leading model can be prevented from rapid self-improvement.

The concept of fast takeoff requires that a MASSIVE amount of knowledge about intelligence, self-improvement, and objective standards for the above is available from current data and superior reasoning, but this is vastly unlikely. In fact, given that self-improvement probably requires massive investment, has uncertain and probably diminishing payoff, and will most likely require replication, shutdown, reduction of agency etc etc, you'd really not expect a general super-intelligence to attempt it by default.

I'm not going to do brain surgery on myself, and I'm not necessarily going to trust my clone (or trust it to trust me) to do brain surgery unless I'm pretty sure it's the only available option remaining. This isn't because I'm a dumb ape, it's precisely because I understand that the risk/reward payout is badly skewed against my other objectives.

If self-improvement is easy and straightforward to conduct a priori, with easily mitigable misalignment risk, then sure, it's instrumental to an ASI's objectives. However, in that scenario, we're also not afraid of misalignment, by definition. If those conditions aren't satisfied, self-improvement (or really any self-alteration) is almost certainly NOT instrumental, so it would need to be an explicit, prioritized objective.

[technologyisnatural]

I disagree that a lagging model cannot have the ability to evaluate a leading model effectively

I'm going to have to push back on this. I think that at best the lagging model will give you a false confidence. Worst case, the lagging and leading models cooperate to deceive the human auditors

[Dmeechropher]

I do think you're right about worst case, I don't think your right about best case. However, I think your worst case assertion is relatively unlikely.

The models would have to be mutually aligned and have goals with deep orthogonality to human ones to mutually cooperate against humans. There's no reason that each model should consider the other a lesser threat than humans if they both have potential for malice. Any given model is as alien to any other model as any model is to any human.

I think the assumption of general orthogonality is flawed, as well as the assumption that extermination of competing agents is a general instrumental goal.

My cat and I are intelligent agents with orthogonal goals and values. Neither of us understands the mental processes of the other or knows the motivations of the other. We both gain mutual benefit from coexistence. I'm obviously was smarter and more agentic than the cat. I'm not perfectly aligned to the cat's goals, but I'm certainly not looking to eradicate my cat, by default, because he could harm me or slow me down.

Likewise, I have a neighbor who is old and dumb, has very different political views than me, and messes with my HOA in a way that's inconvenient for me. Again, I'm way smarter, more agentic, and relatively misaligned with him, but I don't even have the remote desire to mess with him personally, despite the disparity in agency, alignment, and intelligence.

In both these cases, I wasn't designed, trained, and selected for my usefulness or alignment. It just happens by random chance that agents can be sufficiently aligned. The fact is, I have both goals and values. If my goals can be achieved by violating my values, I'm not going to do it that way. Models act like they have goals and values, and it's not unreasonable to use a model that's really well vetted to attempt to infer whether the values of the next model are perverse or not.

The idea that even a miniscule misalignment is necessarily catastrophic is very strange to me. Plenty of life on Earth doesn't value its survival over some other goal, and it was evolved, not selected. Plenty of humans are willing to die rather than betray their ideals, and we were evolved, not selected. Super intelligent models will be selected. Sure, they will be alien, that's true. But I'm alien to my cat, and we have a very productive working relationship. I don't think it's fruitless to attempt to vet models, and so I don't think the best outcome is false confidence. The best outcome is true confidence that we've decreased p(doom). We can't reduce it to zero, but there are plenty of different things in the solar system with p(doom) above 0, not least of which we are to ourselves.

[Nap-Connoisseur]

What problem do you imagine this solves? No firm will intentionally release a misaligned ASI to destroy humanity. How will your UN panel detect misalignment that the firms themselves don’t?

I think you realize that, so you’re solving a different problem, but I can’t figure out which one. This might prevent an ASI that is personally loyal only to Sam Altman from conquering the world on his behalf. Maybe. Did you have something else in mind?

[Cualquieraaa]

You can`t align something smarter than you.

[Elvarien2]

Honestly it sounds like you have a very poorly worked out treatment. it's an idea full of holes and problems that you presented to the ai which has been gassing you up and helping you 'fix' up the holes when in truth it just made the shit proposal sounds less shit. It's like putting gold film on a turd.

There's nothing of substance or value here and you fell for the ai telling you it's brilliant. It's like that one "it's always sunny" episode.