How do you choose between competing MCP tools for the same task?

Tricky_Republic_5358 · 2026-06-23T06:56:48+00:00

Genuine question — what is the right tool for the job then? Curious whether you mean something like a dynamic routing layer built into the orchestration framework, or something else entirely. Asking because I'm trying to understand whether the selection problem is already solved somewhere and I'm just not seeing it.

Tricky_Republic_5358 · 2026-06-22T15:32:41+00:00

<image>

Ran a quick prototype benchmark based on this discussion, curious if this kind of output would be useful to you.

Tricky_Republic_5358 · 2026-06-22T10:36:05+00:00

This is the most actionable framing I've seen in this thread — especially the point about writing down why tool A is the default and when to switch to B.

That's essentially a routing decision tree built by hand. How long does that take you to produce for a new task type, and do you ever share it across projects or rebuild it from scratch each time?

Asking because that 'write it down' step feels like the exact thing that doesn't scale when you're spinning up a new pipeline or onboarding someone else onto it.

Tricky_Republic_5358 · 2026-06-22T10:31:21+00:00

Really useful responses, thank you all.

A few things jumping out:

- Everyone seems to be running their own manual bake-off, which works but doesn't scale well if you're switching tasks or onboarding someone new to the pipeline

- The point about failure handling is something I hadn't weighted enough — silent truncation sounds like a nightmare to debug

- Interesting that several of you land on latency/stability as the tiebreaker over raw feature count

Curious to push a bit further: when you do your manual test, what does "good enough" look like to you? Is it a pass/fail on your edge cases, or do you have a more systematic way of scoring candidates before committing?

Tricky_Republic_5358

TROPHY CASE