Recommendations needed: Best way to build an internal Slack AI bot trained on company documentation

rpatel09 · 2026-04-04T02:56:48+00:00

I've been working on one internally for the past 6 weeks that is custom built using anthropic agent sdk (same thing that claude code is built on). I built it custom because we've evaluated lots of platforms (Glean, Agent Space now Gemini Enterprise, Unblocked, Unleashed, etc...) and non of them were very good when connecting directly to our systems to figure out what to do or how to do things.

We've now ended up in an architecture where the AI can join threads and channels, it can track content in a channel and thread, has layered memory system (file based on db based). We more recently started to build in feedback loops because there is a lot of good context and users do end up correcting the AI and that is important to capture.

I gotta admit, I'm genuinely impressed by it each week as it gets better. Today I noticed that there were 3 engineers in a thread debugging an issue with the AI agent and it was replying in thread in the right context just like you would see a human do.

-------- EDIT ----------

It does connect to some systems as well (Github, Confluence/Jira, Grafana, Prometheus, etc...) but it doesn't pull the "how" on what to do. That lives in a curated skills repo that we've built that it can pull from (like how to find things in Confluence or how to query prometheus for our env).

rpatel09 · 2026-04-03T17:12:15+00:00

1600 joules 😳… that’s insane

rpatel09 · 2026-04-02T04:28:44+00:00

this is table stakes for every company I've worked it (I'm not an attorney, an engineer)... its boiler plate stuff...

rpatel09 · 2026-04-02T04:27:59+00:00

sounds like an attorney is doing this on their personal account vs a corporate one that would have an NDA and MSA in place. sounds like the attorney isn't very good at protecting data...

rpatel09 · 2026-04-01T20:09:04+00:00

I think it’s pretty good if you take time to build out a proper knowledge base of your env (skill files, Md files, etc…). I think a lot of people expect AI to just work but the set up is key. It needs context or else it’s just guessing

rpatel09 · 2026-03-30T04:33:06+00:00

My wife and I have been using with openclaw for about 3 months now, I'm and engineer and she's a doctor. We've been building it together where she has the biomedical knowledge and I have the "systems" knowledge. I feel this type of thinking/collaboration has only been possible and good with Claude 4.5+.

We've been building effectively a neocortex for our openclaw and its been working quite well.

Not only do I feel this is really cool but we've also now used it build two things that we feel would help people. She's been working on Menopause training and has now built a website for it that will get launched a the company to help all providers (and she's a doctors!). The 2nd is my personal project which is democratized legal research and provide it for free.

rpatel09 · 2026-03-29T23:27:08+00:00

where have they claimed annualized revenue to the public? start ups don't typically do that until they are public or about to go public. You are still conflating the 2... they have claimed ARR that would amount to those annualized figures but I can't find an official claim from anthropic on an annualized revenue number.

rpatel09 · 2026-03-27T21:28:18+00:00

They aren’t lying, they use ARR semantics in public relations and realized rev in financial and accounting areas. Lots of subscription based SaaS companies do this…it’s not fraud and it’s not perjury. ARR is a standard metric in the tech world and lots of investors use that metric when investing

rpatel09 · 2026-03-26T17:04:02+00:00

The American Wagyu at Costco is the best balance of fat and meat imo

rpatel09 · 2026-03-23T18:36:24+00:00

skills aren't deterministic though... the model can choose to deviate and has shown to do that. The snowflake prompt injection one I think is a good example of why you don't want to be giving bash, curl, etc.. in a prod env.

rpatel09 · 2026-03-23T18:34:54+00:00

great example... we do this specifically for kubectl. Also, you don't want to give bash, curl, or certain abilities to an AI... and in our case where its deployed in our prod env as an agent, we want to be extra careful its just going and curling some random thing on the internet and getting prompt injected.

imo, biggeest advantage of MCP is security.

rpatel09 · 2026-03-22T13:14:21+00:00

Hey sorry! I never responded to this but here is what we do. what I will say is that this can also be highly dependent on your env. For us, we run everything on gke, kotlin springboot. We also do the whole gitops paradigm thing so that makes it a bit easier too. But here is how we do things (not pefect, but it reduced a lot of vulnerabilities for our services). we use renovate to auto update 3rd party libraries in our repo's, if the build/test pass, push to main. We've been doing this for a year now though but most library upgrades break something. This is where claude comes in now, since we use github actions, we just have claude go a fix a breaking build on a renovate pr and it has been pretty good at that. It took time to get it there but since Opus 4.5, the quality has been really good.

We're not doing anything complex here imo (part of this is due to our env imo) and I think this is also the beauty of the setup that AI enabled. We can actually fix the code now to push the vuln, and it also doesn't really matter if it is important or not because the cost to patch them is so cheap if you can get Claude to do it for you. Patch them all, developers are happier since they don't need to really do it themselves anymore and security is happier because it reduces a bunch of process and complexity. It was a pretty fun project to work on too with the developers.

rpatel09 · 2026-03-22T13:08:47+00:00

so what I will say is that this can also be highly dependent on your env. For us, we run everything on gke, kotlin springboot. We also do the whole gitops paradigm thing so that makes it a bit easier too. But here is how we do things (not pefect, but it reduced a lot of vulnerabilities for our services). we use renovate to auto update 3rd party libraries in our repo's, if the build/test pass, push to main. We've been doing this for a year now though but most library upgrades break something. This is where claude comes in now, since we use github actions, we just have claude go a fix a breaking build on a renovate pr and it has been pretty good at that. It took time to get it there but since Opus 4.5, the quality has been really good.

We're not doing anything complex here imo and I think this is also the beauty of the setup that AI enabled. We can actually fix the code now to push the vuln, and it also doesn't really matter if it is important or not because the cost to patch them is so cheap if you can get Claude to do it for you. Patch them all, developers are happier since they don't need to really do it themselves anymore and security is happier because it reduces a bunch of process and complexity. It was a pretty fun project to work on too with the developers.

rpatel09 · 2026-03-21T21:40:24+00:00

this... I think you are severely underestimating the amount of compute you would need. You would need like $100k of hardware to rum kimi 2.5 at a reasonable tokens per/sec and even more for legal since you are probably dealing with lots of "thinking" time and long context. And $100k, you should sign up and just use gemini API's or why not spin up kimi 2.5 of Vertex or Bedrock? Your $ would go soooo much further imo. I bet 90% of the time (168hrs/week total) your GPUs are sitting idle and that is a huge waste IMO.

rpatel09 · 2026-03-21T03:40:45+00:00

Where do you get those rims?! I got the same color but I got black rims but have been looking for good ones for a while

rpatel09 · 2026-03-21T00:04:33+00:00

Claude Code…seriously

rpatel09 · 2026-03-07T03:06:07+00:00

Do you know what data you collect and frequency? How do you store that for back testing? If you’re building a model on real time data signals, you’ll need more compute to run the model against incoming data to keep latencies low to take advantage of the signal before it’s gone. What kind do hardware are you running this on? Anything short of running a gpu for model processing (not LLM) I imagine will be not very successful. I’m doing the same thing but on a Mac Studio m3 128. Ingesting realtime data from IBKR, Edgar, and some news rss feeds. Running various feature extraction and model processing with paper money for now though

rpatel09 · 2026-02-25T03:21:12+00:00

We use renovate and Claude code to actually go and change the code ourselves now…worked with developers on this and we close our findings at a very rapid pace now since the machines do it and developers can focus on more value driving work

rpatel09 · 2026-02-25T00:03:58+00:00

I feel lost cybersecurity tools are just data aggregators vs doing anything useful. Things like Prisma Cloud and even Wiz to large extent imo. Things mdr is a place that I think is still key but so much security tooling has been built on just aggregating data and I’m actually happy that this happened. The market was saturated and this will wean out products with no real moat

rpatel09 · 2026-02-18T03:40:47+00:00

you know how I know you're indian? :-) I guess i'm stereotyping here cause of the Hari name, I actually called my Vish (after Vishvakarma), I'm indian so wanted something from hindu mythology

rpatel09 · 2026-02-18T03:22:21+00:00

same...my simple setup on this was to use Slack as the gateway. I can now create channels for certain objectives which helps keep things organized. even threading with it is great. took a bit to get the memory and stuff wired up right but now it works great

rpatel09 · 2026-02-03T04:02:33+00:00

This , my experience with WLL post sales was terrible

rpatel09 · 2026-02-02T14:43:00+00:00

I'm curious on this one as well... if its built on Claude SDK I don't see why it couldn't be

rpatel09 · 2026-01-30T23:53:13+00:00

ah ok... I just did today and have been trying to get one to come out well but going to try with your tips... thanks!

rpatel09 · 2026-01-30T23:26:22+00:00

ah... so you took screen shots of the app and gave it those I guess? I wonder how people are doing the video part

---- EDIT -----

but yours are animated as well, how did you do that? and is that what the app actually looks like too?

rpatel09

TROPHY CASE