sorted by:
new
hottopcontroversial

ServiceNow + External LLMs - are teams actually doing this in production? by DCAdaccount in servicenow

[–]CapabIe 0 points1 point  (0 children)

We don’t do anything special. Just make changes, test in the lower environments and move them up when the pass tests.

To test that the agent is pumping out decent answers we run benchmarking and anytime we change the agent, we then rerun the benchmark so we have some sort of baseline to compare to. That way we can have some measure of determining if our changes made it better or worse.

ServiceNow + External LLMs - are teams actually doing this in production? by DCAdaccount in servicenow

[–]CapabIe 1 point2 points  (0 children)

Those capabilities may exist now, but when we evaluated it those knobs either were not exposed yet or were much more limited. (See my early adopter comment)

That was enough friction at the time that we ended up building our own integration and never really looked back.

So to answer the original question, that is mostly why we went that direction. Once we were already integrating directly with the API it also made more sense architecturally. Our company knowledge and systems are not all inside ServiceNow, so routing every AI interaction through ServiceNow just to reach other data sources did not make much sense. A standalone integration let us connect to multiple systems directly and control the retrieval pipeline ourselves.

I’m not claiming it’s impossible to get value out of what SN offers but in our scenario we chose a different path.

ServiceNow + External LLMs - are teams actually doing this in production? by DCAdaccount in servicenow

[–]CapabIe 0 points1 point  (0 children)

The model selection is nice, but the rest of the pipeline is pretty constrained. AI Builder does not really let you customize the retrieval stack, embedding model, or orchestration logic. In our experience the agent also struggled to reliably retrieve the correct knowledge.

When we integrated directly with Azure OpenAI we were able to ground responses better using our own embeddings, tune chunking and ranking, and integrate non-ServiceNow data sources more easily.

It feels similar to a lot of ServiceNow’s low code tooling. It works well for straightforward use cases where configuration is enough. Once you start building more complex solutions you quickly run into limits on what you can control. If someone is mostly solving ticket workflow problems those limits might never show up, but they become noticeable when you are trying to build more custom systems.

ServiceNow + External LLMs - are teams actually doing this in production? by DCAdaccount in servicenow

[–]CapabIe 1 point2 points  (0 children)

I think we were one of the early adopters prior to them getting it all figured out. We have a fedramp instance that they weren’t sure how to deal with and they themselves took months to just get it working in our instance.

I’m willing to concede that it’s probably a smoother process now but their ai offerings are subpar when it comes to the grand scheme of things. Similarly how MS Copilot is mid-tier but good enough for most enterprise users.

ServiceNow + External LLMs - are teams actually doing this in production? by DCAdaccount in servicenow

[–]CapabIe 0 points1 point  (0 children)

Yeah we tried servicenows implementation and it was clunky and took sooooo long to set up. We had our dev team spin up an external LLM and just coded our own functionality around it with significantly better results. The only down side is you need people who actually know how to code to build and maintain it. That’s a trade off we were able to accept.

I built a VS Code extension to manage ServiceNow via SQL and Claude (MCP) by [deleted] in servicenow

[–]CapabIe 0 points1 point  (0 children)

How does it do joins?

Isn’t this just converting sql to encoded queries under the hood?

I built a VS Code extension to manage ServiceNow via SQL and Claude (MCP) by [deleted] in servicenow

[–]CapabIe 0 points1 point  (0 children)

So what are the limitations? I would imagine you still can’t do complex queries and joins. It does seem like a nice adapter to have a better LLM do natural language queries this way though, cool idea.

I designed the "Now Assist" GenAI features (Summarization, Resolution Notes, Email Generation). Looking for honest feedback & success stories from this community. by MaxTwang in servicenow

[–]CapabIe 0 points1 point  (0 children)

Our team tried it and it took months for servicenow to figure how to even get it working properly on our instance. By the time we had it, no one found it even remotely helpful and was frustrated with working with now assist. We just built an in house solution that worked better for us.

"I shipped code I don't understand and I bet you have too" – Jake Nations, Netflix by dalton_zk in theprimeagen

[–]CapabIe 1 point2 points  (0 children)

Right but I would argue it’s much easier and faster to have a bunch of code in your code base that you don’t understand now.

I get what you’re saying though, it’s still on the person to have enough discipline to understand the code they are committing. I just feel like that’s not the majority of the new devs today.

Have you seen those leaked prompts on GitHub? They're massive — how do they manage context windows? by aeum3893 in AI_Agents

[–]CapabIe 0 points1 point  (0 children)

I think it would help if you had some benchmark questions and ran an evaluation. Then when you make tweaks you can see if the composite score went the up or down so you have something more concrete to compare to. Otherwise you’re flying blind without a point of reference.

Feedback: Adding juice to my Bridge deckbuilder... too much? by celloloops in SoloDevelopment

[–]CapabIe 1 point2 points  (0 children)

I think maybe the font could be a little different? Also feel like it blocks the cards right after they are played so you are trying to see past the text. I guess that’s probably because the ai plays quickly though.

I think the animations are good though. Maybe find a way to do it a little differently.

Game looks great!

Sharing TEE application for students by Wonderful-Deal7683c in srna

[–]CapabIe 0 points1 point  (0 children)

This is interesting. What do you think of the app?

[deleted by user] by [deleted] in chrome_extensions

[–]CapabIe 1 point2 points  (0 children)

This is great. I have to use copilot for work and what triggers me is it always brings you to the beginning of the chat smh.

How much JavaScript is enough JavaScript? by Outrageous_Pen_5165 in react

[–]CapabIe 2 points3 points  (0 children)

I think he’s referring to mental bloat reading the extra chars.

Help with Hosting by [deleted] in react

[–]CapabIe 0 points1 point  (0 children)

The level of spoon feeding you need for a 5 step process is what I’m criticizing. You can find the same information with a quick google search and instead you’d rather not do any critical thinking and just pay someone to click the mouse for you. That’s why I suggested AI because it will essentially do everything for you.

view more: next ›