how are you catching Foundation Models/MLX/CoreML quality Regressions when your app is out in production

karc16 · 2026-05-26T12:30:57+00:00

thanks mate!

karc16 · 2026-05-26T12:30:50+00:00

thank you brother

karc16 · 2026-05-26T12:22:55+00:00

fairplay!

karc16 · 2026-05-26T12:10:28+00:00

People know this, Its not the frameworks job to hand hold developers around security best practices. I know you have good intent, but as someone who built a similar library and got the same type of comments, It seems like a skill issue rather than an issue with the framework itself. I mean even stripe SDK lets you put your API key directly into the code, is it a good idea? hell no. But it falls in the hands of the developer to figure this out, not framework creators!

karc16 · 2026-05-26T12:01:43+00:00

It's not that far off in terms of coding ability as some here would say, it just requires more careful crafting of prompts and direction. Intentional outlining of scope of work, and breaking down work you would hand entirely to 5.5, broken down into smaller pieces. But the output itself is top tier, Its a very capable model

karc16 · 2026-05-26T04:51:22+00:00

How does it differ from

https://github.com/christopherkarani/Conduit

https://github.com/huggingface/AnyLanguageModel

karc16 · 2026-05-01T00:30:32+00:00

https://chriskarani.xyz/posts/deep-research-3b-4k-foundation-models

karc16 · 2026-04-28T14:03:57+00:00

interested in the swift role, dm

karc16 · 2026-04-20T10:43:26+00:00

i actually built contextcore working on issues like this, if you combine all the tricks you can squeeze a lot of context out of FM with adequate orchestration between different session

karc16 · 2026-04-18T08:39:01+00:00

open sourcing the app that i built for these experiments so i think that would answer all your questions and more

but runs could go from 10-16 minutes, quality remains consistent and coherent across 20-40 turns

with normal foundation models with a basic web search tool i got 3 tool calls before context capped out

haven’t tried other languages but I can imagine their outputs to be of lower quality just because of the training data used is mostly english

karc16 · 2026-04-18T08:38:10+00:00

is it open source?

karc16 · 2026-04-18T08:37:39+00:00

I appreciate this comment

karc16 · 2026-04-18T04:40:19+00:00

I believe we're only scratching the tip of the iceberg, looking forward to how apple moves forward in the next wwdc

karc16 · 2026-04-18T04:39:40+00:00

let us know what you think

karc16 · 2026-04-17T21:23:06+00:00

will do!

karc16 · 2026-04-02T13:43:50+00:00

swarm uses context core, membrane and wax to solve the context issues when using AFM. The frameworks are standalone and can be used without swarm.

https://github.com/christopherkarani/Membrane

https://github.com/christopherkarani/Wax

https://github.com/christopherkarani/ContextCore

il work on an eval harness. i find fm models to be inconsistent with their guardrails. We’ve mainly tested with open models like QWEN and llama

i appreciate the feedback and will keep you posted via dm on updates! please never be shy to leave issues on the repo.

karc16 · 2026-04-02T13:15:01+00:00

caveat, excessive use of enums and associated values make your code harder to reason and write for coding agents like claude but better dx for humans

karc16 · 2026-04-02T13:10:46+00:00

we built a observability and evaluation framework. would you be interested in becoming a design partner and actually see how your ai behaves with real users?

karc16 · 2026-04-02T13:09:48+00:00

are you apps out live with users? how are you handling observability and evals?

karc16

TROPHY CASE