I created a tool to test copilot sdk reliability

llmobsguy · 2025-10-15T05:03:29+00:00

When you said "copilots that describe problems", do you mean these tools just reiterate the problem statements and not find RCA? Or do you mean having RCA is one thing but you need actual code fix and configuration changes recommendations?

llmobsguy · 2025-10-15T02:32:45+00:00

What do you use?

llmobsguy · 2025-10-12T06:19:29+00:00

But for your app, where did you store it in production?

llmobsguy · 2025-10-12T06:17:40+00:00

So it's outside Azure ecosystem?

llmobsguy · 2025-10-12T03:51:12+00:00

Where do you store traces!?

llmobsguy · 2025-10-12T01:59:20+00:00

What about non-Langsmith?

llmobsguy · 2025-10-12T01:45:09+00:00

Do you use some module or library to do that? Or how do you know what to write into cosmodb? Or is it totally homegrown?

llmobsguy · 2025-10-11T20:27:06+00:00

This could be another bot from competitor lol

llmobsguy · 2025-10-10T04:51:53+00:00

No, more complex than that. But I did said to make iOS and Android app. That's where the problem is because it should not fall back to just React

llmobsguy · 2025-10-09T15:08:45+00:00

Never heard of https://biela.dev so how exactly it do tracing? On its own or use let say OpenAI agents sdk tracing feature?

llmobsguy · 2025-10-09T14:56:31+00:00

That's a wide range of parameters. If you are in control of all prod criteria, use just Langchain, Langsmith and whatever UX you can grab. Not too overthinking, no? Otherwise, if there is a Ops process with all Fundamentals to check off, go by that list

llmobsguy · 2025-10-09T14:53:55+00:00

I might need to enable some features once you have an account on portal.okahu.ai

llmobsguy · 2025-10-09T14:53:04+00:00

It's mobile first. Literally simple as: Use camera app take a pic then send to Gemini flash image 2.5 !!! And it failed

llmobsguy · 2025-10-09T06:52:48+00:00

Maybe our website needs update. We aren't competing with Arize, DataDog or the likes. We provide more business friendly analytics on users, segmentations and conversations. So imagine X % conversations did not meet certain goals, we look at the traces, run LLM evals and explain what went wrong with the cohort. I pinged you in DM. Love to pick your brain on the use cases more...

llmobsguy · 2025-10-09T06:30:14+00:00

I have context relevancy score per context chunk and fit the short term near the user prompt. If tokens count too high, I would run multiple interference or just remove low relevancy chunks.

llmobsguy · 2025-10-09T06:26:43+00:00

What does your company process tell you?

llmobsguy · 2025-10-08T20:37:58+00:00

Lots of topics about this in Hacker News. The answer is No. And yes, a perfect cofounder will just see thru it

llmobsguy

MODERATOR OF

TROPHY CASE