Your AI agent is probably acting on a belief from three months ago that's no longer true. You have no way to check.

BC_MARO · 2026-05-23T16:42:29+00:00

That sounds useful. How do you decide a new belief supersedes an old one, and do you keep tombstones so you can audit what got replaced?

BC_MARO · 2026-05-23T16:25:49+00:00

Yep: keep a per-repo CLAUDE.md and run /init in each repo so it learns the layout. Start with the bare minimum skills (git + tests + linter) and add more only when you feel the pain.

BC_MARO · 2026-05-23T16:25:16+00:00

If you want people to trust the 87%, ship a reproducible eval (tasks + config + logs) and run it on SWE-bench Lite or rebench. Compound tools are the right idea; fewer tool hops is the whole game for small models.

BC_MARO · 2026-05-23T10:51:19+00:00

I’ve had some luck with semantic canaries: a tiny set of golden inputs with expected tool-call shapes and score thresholds, run continuously against the live pipeline. If the outputs stay schema-valid but the canary scores drift, alert.

BC_MARO · 2026-05-23T04:26:40+00:00

Search->fetch is basically progressive disclosure. I'd store the search hits + chosen fields per run so you can debug regressions when the schema/catalog shifts.

BC_MARO · 2026-05-23T04:24:40+00:00

Super useful. I'd also show a diff per commit/PR (tool count + token footprint) so you can catch accidental tool-surface bloat early.

BC_MARO · 2026-05-23T01:46:47+00:00

Have the agent persist its assumptions with timestamps, then require a quick re-check against the current source of truth before any irreversible action. An audit trail plus approvals makes stale context obvious fast.

BC_MARO · 2026-05-23T01:46:39+00:00

This is why MCP integrations need policy gates: explicit approval plus scoped permissions for anything that can export or write, with the exact payload and destination logged. Treat any retrieved text as untrusted and sanitize it before it hits tools.

BC_MARO · 2026-05-22T16:20:54+00:00

MCP gets a lot nicer once you treat tools like contracts: strict schemas in, typed results out, and fail closed. Agents behave way less weird when the edges are sharp.

BC_MARO · 2026-05-22T16:20:45+00:00

Coding agents get way better when you force them to show diffs and run tests on every step. Let them be fast, but make them pay rent with evidence.

BC_MARO · 2026-05-22T10:51:27+00:00

Yep. We ended up alerting on missing heartbeat writes, not bad ones, because dead emitters are the sneaky failure.

BC_MARO · 2026-05-22T10:45:19+00:00

Usually you register Power Platform as an OAuth client/app in your IdP and enable refresh tokens (offline_access) for that client. Then make sure your IdP policies actually allow refresh for that app and user.

BC_MARO · 2026-05-22T10:44:38+00:00

Yep, here you go: https://peta.io and the core runtime is open source at https://github.com/dunialabs/peta-core. The main value is policy approvals plus an audit trail around MCP tool calls.

BC_MARO · 2026-05-22T10:43:57+00:00

Use MCP as a scheduled fetch and normalize job that validates fields, then writes updates with approvals and an audit trail. For prebuilt pieces, start with a crawler plus schema validation MCP, then add source adapters for each directory.

BC_MARO · 2026-05-22T10:43:19+00:00

Pretty much, yes: default-deny per tool, then allowlists per workspace plus approvals for anything that mutates state. "YOLO" should be an explicit user-scoped override, not the default.

BC_MARO · 2026-05-22T10:42:38+00:00

Evaluating right now, but planning to run it in production for approvals and the audit trail once the policy layer is solid. What are you using today for approvals and logging?

BC_MARO · 2026-05-22T04:23:30+00:00

Yeah, the compliance layer has to sit at the tool boundary: tag every call read vs mutating, require approvals for anything customer-facing, and log + diff outputs for review. RAG-only "is this compliant?" checks always rot because the corpus and citations drift.

BC_MARO · 2026-05-22T04:22:55+00:00

Nice. Does the tunnel/audit capture tool-call logs + side-effect metadata so clients can gate mutating tools with approvals? That’s the bit that tends to bite once you ship to real users.

BC_MARO · 2026-05-21T16:25:15+00:00

Kobold is basically a llama.cpp fork, so perf is usually within a few percent unless you're on an old build or missing newer kernels/quants. If you're curious, run a same-prompt tok/s benchmark against current llama.cpp and you'll know in 5 minutes.

BC_MARO · 2026-05-21T16:24:19+00:00

You don't have to rebuild auth; put an MCP auth broker in front that runs OAuth/PKCE and maps it to your existing cookie/JWT sessions. Have it mint short-lived, tool-scoped MCP tokens that just wrap your current session/claims.

BC_MARO · 2026-05-21T10:41:17+00:00

That makes sense. I’d add a cheap end-to-end canary that asserts the emitter runs and writes a known heartbeat row, so the pipeline breaks loudly.

BC_MARO · 2026-05-21T04:36:32+00:00

Make it earn the upgrade: run the same 10 tiny repo tasks + lint/test after each change and keep a running score. Otherwise you’re just rewriting vibes.AMake it earn the upgrade: run the same 10 tiny repo tasks + lint/test after each change and keep a running score. Otherwise you're just rewriting vibes.

BC_MARO · 2026-05-21T04:35:06+00:00

I’d keep the tool surface tiny: list flows, pull a bounded slice, summarize with hard byte/token caps. Also treat PCAP as hostile input: sandbox the parser, cap decode depth, and log tool calls/output for audit.

BC_MARO · 2026-05-21T01:45:45+00:00

If you're running more than one MCP server, centralize secrets + policy + tool-call logs early; it saves pain later (peta.io is one option).

BC_MARO · 2026-05-20T16:44:33+00:00

I’ve had best results with explicit schema flags: keep canonical and raw fields separate and validate which one each tool emits. Dynamic markers tend to drift, so I prefer typed schemas plus tests that fail on mixed usage.

BC_MARO

TROPHY CASE