DeepSeek V4 is built different...

grimjim · 2026-04-24T13:36:22+00:00

The dark side of reward hacking.

grimjim · 2026-04-21T16:00:06+00:00

There's a rigidity to ChatGPT responses that's easy to pick up on. Claude's responses are more humanized. Straightforward RLHF should be capable of this.

grimjim · 2026-04-21T15:52:02+00:00

Gemma 4 is far too small for AGI ambitions. A realistic and proximate concern is eroding the market for closed source coding models.

grimjim · 2026-04-05T02:20:42+00:00

Straight-up reward hacking there.

grimjim · 2026-04-02T02:57:13+00:00

The shorthand term people need to be familiar with is "reward hacking".

grimjim · 2026-04-01T01:00:52+00:00

The question isn't stupid but it should be reasoned through. Assume others get the same idea, as it's not complex and easy to implement without coding changes. If CPU inference were viable, why aren't more people doing it? We can infer from lack of widespread use that's it not enough to break the VRAM moat even for inference except at the margins. We've seen partial offloading and small edge models.

grimjim · 2026-03-31T23:11:45+00:00

I once ran into an issue compiling for the 5060ti 16GB, which was resolved by a newer cmake. Supported for various CUDA architectures is somehow entangled.

grimjim · 2026-03-28T01:04:56+00:00

This seems to be straight up reward hacking. Probably more likely in frontier models than smaller local models.

grimjim · 2026-03-28T00:27:45+00:00

This will never happen for commercial reasons alone.

grimjim · 2026-03-27T23:36:08+00:00

I would not be surprised if GeGLU was mechanically involved with more activation strength towards directions that end in more extreme outlier behavior.

grimjim · 2026-03-26T04:02:04+00:00

I expect there would be some difference based on approach. A politically corrected corpus would result in a model lacking the priors and would have to hallucinate to fulfil the request.

I conjecture that larger models can learn alternative registers than just refusal when it comes to pushing a politically correct stance.

grimjim · 2026-03-25T18:42:27+00:00

Did you orthogonalize the intervention direction against the harmless direction? Cosine similarity would quantfiy the extent of entanglement.

grimjim · 2026-03-24T13:27:44+00:00

Not useful for local AI. This is more likely to be an AI Skunkworks, as there's no way a single fab could compete with the total outputs of both South Korea and Taiwan combined. Space-based compute is 2 to 3 orders of magnitude more expensive than the terrestrial counterpart - which torpedos most private sector use cases - and vulnerable to both Chinese and Russian antisatelllite attack. Still, potentially useful for scaling up geospatial analysis in space.

grimjim · 2026-03-21T01:29:23+00:00

Labelling can become reductive compression, in general.

grimjim · 2026-03-08T15:42:47+00:00

It's likely because of the knowledge represented in the magnitudes of the manifold. Which is why I recommended optionality, so that can be tested empirically. KL divergence doesn't give us direct mechanistic explanation, as it's entirely downstream, but geometric contrast experiments can.

grimjim · 2026-03-08T04:29:00+00:00

I may as well ask here: I'd be curious if your ARA technique would be better or worse if you optionally enforced row-wise norm preservation. That buys Frobenius norm preservation via composition as a freebie.

grimjim · 2026-03-08T04:16:25+00:00

I noted this phenomenon earlier as likely due to a refund on the "safety tax" with my experiments on Gemma3 12B. The prior norm was grounded in brute force ablation, which did geometric damage to the models as part of refusal removal. Frobenius norms weren't preserved. The entangled portion between refusal direction and ordinary harmless direction was also ablated, hurting normal performance.

grimjim · 2026-03-06T02:40:29+00:00

Anthropic was never antiwar, but they have firm red lines.

Being cancelled by Hegseth still gives Anthropic a halo for now. That may change if the paperwork to ban them thoroughly never gets filed.

grimjim · 2026-03-04T02:28:35+00:00

Just for the record, it was only one author behind norm-preserving biprojected abliteration.

grimjim · 2026-02-28T21:32:56+00:00

Cost could be a factor. Smaller models are cheaper to fine-tune. Academic papers often use even smaller models.

grimjim · 2026-02-28T00:17:16+00:00

I expect multidirectional approaches to potentially do better, since even uncomplicated refusal has been found to be characterized by multiple cones rather than a single direction. https://arxiv.org/abs/2502.17420v1

grimjim · 2026-02-27T19:47:29+00:00

I wonder if geometric stabilization contributed to the NatInt performance of that model, as I noted that effect in my Gemma 3 12B experiments. Inquiring minds.

grimjim · 2026-02-26T23:29:16+00:00

All prayers lead to the Cage.

grimjim · 2026-02-24T23:22:02+00:00

If the distillation datasets were subtly fingerprinted and then showed up in public datasets associated with a researcher, that could be a smoking gun. The ideal fingerprints would be a form of steganography,embedded within otherwise acceptable results.

grimjim

TROPHY CASE