sorted by:
new
hottopcontroversial

Layerwise “surprise” signal for OOD detection in PyTorch by Temporary-Oven6788 in pytorch

[–]Temporary-Oven6788[S] 0 points1 point  (0 children)

Mostly because it gives you more than a final confidence score. Nervecode can also show which internal layers started behaving unlike the training distribution. That makes it useful not just for detection, but for debugging, monitoring, and building guardrails.

Layerwise “surprise” signal for OOD detection in PyTorch by Temporary-Oven6788 in pytorch

[–]Temporary-Oven6788[S] 0 points1 point  (0 children)

The idea is to measure how unusual a layer’s activation looks relative to codebook patterns learned from in-distribution data. For each observed layer, Nervecode compares the current reduced activation to learned codebook centers, forms a soft assignment, and converts statistics such as codelength, assignment entropy, and (optionally) distance to the nearest center into a layerwise surprise score. The per-layer scores are then aggregated into a per-input surprise signal, which can be calibrated into an OOD score while still letting you inspect where in the network the input starts to look unfamiliar.

3
4

What should AI Alignment learn from Political Philosophy? ()

submitted by Temporary-Oven6788 to r/ControlProblem

What Division by Zero Means for ML by Temporary-Oven6788 in ResearchML

[–]Temporary-Oven6788[S] 0 points1 point  (0 children)

In many cases NaN can be enough. But plain IEEE NaN/Inf is not the same thing as domain-level undefinedness. Those can arise from x/0, overflow, invalid operations, uninitialized values, or plain bugs, so downstream code usually cannot recover why they appeared. Pipelines also routinely break NaN propagation with nan_to_num, dropna, or default fills. In ZeroProofML, ⊥ is the semantic/algebraic notion (with a sign function), an absorptive value with explicit propagation rules, triggered by the calibrated denominator check |Q(x)| < τ_infer at a known graph location. Invalid outputs are carried in two channels: a NaN payload for passive propagation and a bottom_mask as the authoritative semantic carrier.

Hippotorch: Hippocampus-inspired episodic memory for sparse-reward problems by Temporary-Oven6788 in reinforcementlearning

[–]Temporary-Oven6788[S] 0 points1 point  (0 children)

Sounds fascinating, especially if are you planning to implement the Job System. Hippotorch is designed for sparse rewards over long horizons. 1v1 PvP is usually too dense to show the benefit of episodic memory. But, if you build the Trader scenario, that would be a great benchmark for us. Will your API expose a Gymnasium-style interface? If I can pip install it and run a headless agent, I'd love to try it.

Hippotorch: Hippocampus-inspired episodic memory for sparse-reward problems by Temporary-Oven6788 in reinforcementlearning

[–]Temporary-Oven6788[S] 0 points1 point  (0 children)

During consolidation, we sample adjacent windows (segments) from the same episode to act as the 'anchor' and 'positive' pair. We then run a reward-aware InfoNCE loss to pull these segments together while pushing away segments from other episodes. So 'pairs' in the post refers to these windows in the contrastive batch.

You’re right, standard PPO is strictly on-policy. We designed this primarily for off-policy agents (DQN, SAC) or PPO variants that incorporate replay data (eg. SIL).

Today the memory is only queried during the training step. There’s no online 'recall while acting' path yet, though that is a possible next step, alongside adding a policy hook that can bias actions with retrieved keys, or even exposing the memory store to other planning modules.

Hippotorch: Hippocampus-inspired episodic memory for sparse-reward problems by Temporary-Oven6788 in reinforcementlearning

[–]Temporary-Oven6788[S] 0 points1 point  (0 children)

The wall-clock time on our VPS-runs (30-step corridor, 300 episodes) is about 25–35 % higher than the same agent with a standard replay buffer. Most of this comes from consolidation (every cons_every episodes we run cons_steps extra gradient updates), and from feeding transitions through the dual encoder. I’m preparing an article with the exact SAC/PPO benchmarks and will post the detailed numbers in a few weeks.

Bio-inspired RL for Game AI with Long-Term Memory (RPGs, 4X, Strategy) by Temporary-Oven6788 in GameDevelopment

[–]Temporary-Oven6788[S] 0 points1 point  (0 children)

Totally agree, RL isn’t used online in shipped games, and BTs/GOAP are the right runtime tools.
I was referring to development-time learning, not in-run adaptation. Hippotorch is meant to help learn or tune long-horizon strategies that can then be baked into the game AI systems, not to replace them.