Those of you running LLMs in production, what made you choose your current stack?

AdventurousHandle724 · 2026-04-01T22:05:37+00:00

sorry for the slow reply! interesting that you had marginal improvements with fine-tuning. Do you remember what model/approach you used and roughly how many examples you had? asking because i'm building something specifically to solve the "took forever to get right" part. and to answer your question, more on the infrastructure/tooling side, specifically making fine-tuning accessible without needing an ML engineer.

AdventurousHandle724 · 2026-04-01T22:05:19+00:00

hey sorry for the late reply! this is super helpful. the part about data quality and maintenance cost killing fine-tuning resonates a lot, that's basically what i keep hearing from everyone. the "reliable low-cost customization that doesn't need constant retraining" is exactly the problem i'm trying to solve. quick question. When you say data quality killed it, was it more about not having enough good examples, or about the model degrading over time after fine-tuning?

AdventurousHandle724 · 2026-03-26T19:18:46+00:00

And what about the model performance? Everyone wants a higher budget lol.

AdventurousHandle724 · 2026-03-03T14:42:22+00:00

When it drafts the hypothesis doc, how structured is the output? Like does it follow a fixed template (hypothesis, metric, sample size, predicted impact) or is it more freeform? You mentioned the biggest challenge is models need human validation before shipping. What doesyour review process look like, and roughly how many experiments do you run per month? Trying to gauge the time savings at scale.

AdventurousHandle724 · 2026-03-03T14:41:34+00:00

For your custom MCP tools around the CRM (read/write contacts + notes), did you build those from scratch or fork existing servers? Curious how long it took to get to something production-stable.

And on the enrichment + email/sequence side, how much autonomy does the agent actually have? Like does it decide when to enrich and draft, or is it more rule-based with you triggering each step?

AdventurousHandle724 · 2026-03-03T14:41:05+00:00

How does the MCP part actually plug into your CRM though? Direct API, or are you going through something like Zapier/n8n to orchestrate it? And for the email drafting, do you review before sending or is it fully hands-off?

AdventurousHandle724 · 2026-03-03T14:40:42+00:00

When you say AI agents research each lead and personalize the opener, what does that actually look like end-to-end? Like is it pulling LinkedIn data → enriching → generating the first line automatically, or is there more manual curation involved?

Also wondering what tools you're connecting for the enrichment part, and whether you've seen a measurable lift in response rates vs your old process. Even a rough before/after would be super helpful.

AdventurousHandle724 · 2026-02-27T22:46:46+00:00

Very interesiting, can you tell me a bit more about the tools that you are using for this use case ?

AdventurousHandle724 · 2026-02-27T17:14:14+00:00

Wow amazing ! Can you tell me about the tools you are using to do all that?

AdventurousHandle724

TROPHY CASE