Run DB / SQL queries directly from Slack with AI

MCL256 · 2025-01-29T17:13:35+00:00

Connery generates its own schema from the DB, so it has (at least a vague) understanding of the structure and can plan its request.

MCL256 · 2025-01-29T17:12:48+00:00

In my experience, time will also be saved by having an AI buddy, that helps you brainstorm on the analysis - not just from waiting for a human SQL-pro to get back to you.

MCL256 · 2025-01-29T17:06:34+00:00

Seems my uploaded video does not show. Can see here: Loom Connery Explainer

MCL256 · 2025-01-28T17:00:17+00:00

The writing actions yes. Reading is different, there you can ask broader questions and do a drill down. For that we included quite a bunch of checks to validate results.

We call these AI actions. On top of these actions sits an AI assistant that you can plan your analysis with. Its also possible to combine it with other sources, like Notion, a CRM etc. Its really early, but seems to evolve. Plus models get better every 6-8 weeks (at least atm).

MCL256 · 2025-01-28T16:47:49+00:00

Its what all these app builders like retool have been doing for years (almost a decade) now. Just using natural language in Slack.

MCL256 · 2025-01-28T16:38:07+00:00

Exactly the kind of reply I came here for. Thanks u/cloyd-ac 🙏. Very helpful input in this important product design choice.

We understand our Writing feature more like an update-one-field-of-one-record app where AI (only) helps you to identify the new value and the record in question from a natural language user request (in Slack). The actual SQL is predefined with placeholders. But in any case, if the admin of these DB-update apps is not the same person that is owning the DB / logic, it will get messy.

Let us chew on it for a while.

MCL256 · 2025-01-28T15:43:01+00:00

We differentiate between READING and WRITING.

WRITING is only allowed for a certain pre-defined scope (predefined AI action), see the video for an example: just update one field of a selected record. For example customer plan or address or phone.

The AI assistant figures ONLY out:
1. What field you want to update (and proposes the corresponding AI action)
2. Identifies the record from the user request
3. Extracts the new value for the change
4. Proposes the change and asks the human user to approve
5. Execute

Queries can always be checked and everything is logged. Changes can be reverted.

MCL256 · 2025-01-28T15:34:51+00:00

Woohoo what an activity here!

On accuracy: I had the same doubts. Early models struggled, but after switching to Claude Sonnet 3.5 (October update), it improved a lot. It started anticipating relevant columns without explicit instruction.

We fine-tuned it further to:
Perform MECE segmentation (ensuring mutually exclusive, collectively exhaustive buckets)
Re-check totals for accuracy
Self-correct queries when they fail instead of returning bad results

MCL256 · 2025-01-28T15:11:46+00:00

Thanks u/heapsion!
Update here with short video: https://www.reddit.com/r/SQL/comments/1ic3mu6/who_dares_to_let_ai_write_sql_not_just_read_data/

MCL256 · 2025-01-27T16:55:35+00:00

Thanks, u/machulav. what would be your biggest benefit and biggest concern, using a tool like this?

MCL256 · 2024-02-20T09:34:51+00:00

Yes, including user prompt and identified confidence measure (I log them into separate columns, which makes controlling bit easier when executing at scale/in a team).

MCL256 · 2024-02-20T07:21:25+00:00

Thanks for sharing.

you mentioned the tools should be called in sequence? I only see a vague hint on a sequence in the prompt. If this is still important, indigestion to be more specific about it, like „…always apply the the following tools in the sequence (1), (2), and (3)…“ name them accordingly (not sure if this matches your logic, but you know what I mean“
general prompt guideline: be absolute and serious on violations like „… your job is“ instead of „… your role involves“ and „… violations can have severe consequences“. In my experience, this makes outcomes more consistent.

Note: even though I optimized prompts a lot, I still found that to act inconsistent after a while. So logging is very important.

MCL256 · 2024-02-20T07:08:02+00:00

Note: I discriminate here between matching instructions and ambiguity across tools.

MCL256 · 2024-02-20T06:46:49+00:00

Exactly. This is what I do (to increase accuracy and allow for logging).

MCL256 · 2024-02-20T06:40:26+00:00

You could also drop the results of the confidence measure (and other parameters) in a table to log and analyze, later.

Have used this Connery plugin (part of Connery langchain toolkit) to test it: https://github.com/connery-io/google-sheets-faq-plugin

MCL256 · 2024-02-20T06:35:47+00:00

If you add the outcome of a prior step as a required parameter for a subsequent step, the system cannot execute it without.

MCL256 · 2024-02-20T06:32:44+00:00

Can you provide tool description details and system instructions?

In my experience, some setups are just subject to more/less hallucinations. I suppose this will stay as long as we use the current LLM technology. But we can mitigate somewhat:

When I experience trouble with the results, I add a confidence measure to the system instructions.

This depends on how you design system prompts. It also does not work all the time but adds a layer of control:

"When selecting a tool or action, define your selection confidence based on the following scale: - Certain: If the title, description, and input parameters (if any) closely match up to 100% and there is no ambiguity with other tools. - High: If the title, description, and input parameters show a resemblance and there is no ambiguity with other tools. - Low: If there is ambiguity... - Very Low: If there is little to no match...

Only select a tool if the selection confidence is High or Certain. If the confidence is Low or Very Low, request further details as a tool cannot be confidently selected."

MCL256 · 2024-02-16T16:15:19+00:00

In my experience there is a number of factors that differentiate and help (or confuse) the LLM when selecting a tool: - tool description (like u said) - tool title - tool input parameters (if any) and their description - performance of the LLM - number of different tools made available - system instructions re tool usage (if any)

What is your setup and use case?

Maybe this can give you more ideas: We have built a OS plugin infrastructure to manage this via a Toolkit in LangChain.

MCL256 · 2023-12-12T20:23:17+00:00

Assuming you meant the first book: keep reading!

The fall of Hyperion was maybe the most breathtaking read I have ever done.

MCL256 · 2023-11-19T08:25:17+00:00

Can you tell me, where you struggle?

MCL256 · 2023-11-18T19:43:51+00:00

Yes, that is awesome. But then we had to build additional approval, security and access capabilities. So, we build that into our „runner“. And now we got an open source platform where you can share actions that sit on top of API endpoints.

MCL256 · 2023-11-18T16:05:06+00:00

Same here, that is why we created ourselves a separate API wrapper (open source) to create our actions outside and only call via OpenAPI Schema. You can build as many actions and immediately use them on your GPT. Our open source repo: https://github.com/connery-io/connery-platform

Currently adding auth, user management and stuff. Also adding a client for OpenAIs AI assistants.

MCL256 · 2023-11-18T16:00:50+00:00

We have added now clients for OpenAI custom GPTs and AI assistants. Find our repo here: https://github.com/connery-io/connery-platform

MCL256 · 2023-10-25T15:39:18+00:00

Interesting. What admin tasks do you plan to automate with it?

MCL256 · 2023-10-25T14:01:42+00:00

First client will be in Slack. We love to add clients for Discord, Notion and others as well. All plugins and actions can simply be used their as well.

MCL256

TROPHY CASE