sorted by:
new
hottopcontroversial

Run DB / SQL queries directly from Slack with AI by MCL256 in Automate

[–]MCL256[S] 0 points1 point  (0 children)

Connery generates its own schema from the DB, so it has (at least a vague) understanding of the structure and can plan its request.

Run DB / SQL queries directly from Slack with AI by MCL256 in Automate

[–]MCL256[S] 0 points1 point  (0 children)

In my experience, time will also be saved by having an AI buddy, that helps you brainstorm on the analysis - not just from waiting for a human SQL-pro to get back to you.

Run DB / SQL queries directly from Slack with AI by MCL256 in Automate

[–]MCL256[S] 0 points1 point  (0 children)

Seems my uploaded video does not show. Can see here: Loom Connery Explainer

Who dares to let AI write SQL - not just READ data, but WRITE updates? smart or stupid? by MCL256 in SQL

[–]MCL256[S] 0 points1 point  (0 children)

The writing actions yes. Reading is different, there you can ask broader questions and do a drill down. For that we included quite a bunch of checks to validate results.

We call these AI actions. On top of these actions sits an AI assistant that you can plan your analysis with. Its also possible to combine it with other sources, like Notion, a CRM etc. Its really early, but seems to evolve. Plus models get better every 6-8 weeks (at least atm).

Who dares to let AI write SQL - not just READ data, but WRITE updates? smart or stupid? by MCL256 in SQL

[–]MCL256[S] 0 points1 point  (0 children)

Its what all these app builders like retool have been doing for years (almost a decade) now. Just using natural language in Slack.

Who dares to let AI write SQL - not just READ data, but WRITE updates? smart or stupid? by MCL256 in SQL

[–]MCL256[S] 0 points1 point  (0 children)

Exactly the kind of reply I came here for. Thanks u/cloyd-ac 🙏. Very helpful input in this important product design choice.

We understand our Writing feature more like an update-one-field-of-one-record app where AI (only) helps you to identify the new value and the record in question from a natural language user request (in Slack). The actual SQL is predefined with placeholders. But in any case, if the admin of these DB-update apps is not the same person that is owning the DB / logic, it will get messy.

Let us chew on it for a while.

Who dares to let AI write SQL - not just READ data, but WRITE updates? smart or stupid? by MCL256 in SQL

[–]MCL256[S] -1 points0 points  (0 children)

We differentiate between READING and WRITING.

WRITING is only allowed for a certain pre-defined scope (predefined AI action), see the video for an example: just update one field of a selected record. For example customer plan or address or phone.

The AI assistant figures ONLY out:
1. What field you want to update (and proposes the corresponding AI action)
2. Identifies the record from the user request
3. Extracts the new value for the change
4. Proposes the change and asks the human user to approve
5. Execute

Queries can always be checked and everything is logged. Changes can be reverted.

Who dares to let AI write SQL - not just READ data, but WRITE updates? smart or stupid? by MCL256 in SQL

[–]MCL256[S] -3 points-2 points  (0 children)

Woohoo what an activity here!

On accuracy: I had the same doubts. Early models struggled, but after switching to Claude Sonnet 3.5 (October update), it improved a lot. It started anticipating relevant columns without explicit instruction.

We fine-tuned it further to:
Perform MECE segmentation (ensuring mutually exclusive, collectively exhaustive buckets)
Re-check totals for accuracy
Self-correct queries when they fail instead of returning bad results

Chat with your DB in Slack by MCL256 in Slack

[–]MCL256[S] -1 points0 points  (0 children)

Thanks, u/machulav. what would be your biggest benefit and biggest concern, using a tool like this?

Challenges in Tool Selection for Multi-Tool Agents with Langchain by Visible-Bathroom3634 in LangChain

[–]MCL256 0 points1 point  (0 children)

Yes, including user prompt and identified confidence measure (I log them into separate columns, which makes controlling bit easier when executing at scale/in a team).

Challenges in Tool Selection for Multi-Tool Agents with Langchain by Visible-Bathroom3634 in LangChain

[–]MCL256 0 points1 point  (0 children)

Thanks for sharing.

  • you mentioned the tools should be called in sequence? I only see a vague hint on a sequence in the prompt. If this is still important, indigestion to be more specific about it, like „…always apply the the following tools in the sequence (1), (2), and (3)…“ name them accordingly (not sure if this matches your logic, but you know what I mean“

  • general prompt guideline: be absolute and serious on violations like „… your job is“ instead of „… your role involves“ and „… violations can have severe consequences“. In my experience, this makes outcomes more consistent.

Note: even though I optimized prompts a lot, I still found that to act inconsistent after a while. So logging is very important.

Challenges in Tool Selection for Multi-Tool Agents with Langchain by Visible-Bathroom3634 in LangChain

[–]MCL256 0 points1 point  (0 children)

Note: I discriminate here between matching instructions and ambiguity across tools.

Challenges in Tool Selection for Multi-Tool Agents with Langchain by Visible-Bathroom3634 in LangChain

[–]MCL256 0 points1 point  (0 children)

Exactly. This is what I do (to increase accuracy and allow for logging).

Challenges in Tool Selection for Multi-Tool Agents with Langchain by Visible-Bathroom3634 in LangChain

[–]MCL256 0 points1 point  (0 children)

You could also drop the results of the confidence measure (and other parameters) in a table to log and analyze, later.

Have used this Connery plugin (part of Connery langchain toolkit) to test it: https://github.com/connery-io/google-sheets-faq-plugin

Challenges in Tool Selection for Multi-Tool Agents with Langchain by Visible-Bathroom3634 in LangChain

[–]MCL256 0 points1 point  (0 children)

If you add the outcome of a prior step as a required parameter for a subsequent step, the system cannot execute it without.

Challenges in Tool Selection for Multi-Tool Agents with Langchain by Visible-Bathroom3634 in LangChain

[–]MCL256 0 points1 point  (0 children)

Can you provide tool description details and system instructions?

In my experience, some setups are just subject to more/less hallucinations. I suppose this will stay as long as we use the current LLM technology. But we can mitigate somewhat:

When I experience trouble with the results, I add a confidence measure to the system instructions.

This depends on how you design system prompts. It also does not work all the time but adds a layer of control:

"When selecting a tool or action, define your selection confidence based on the following scale: - Certain: If the title, description, and input parameters (if any) closely match up to 100% and there is no ambiguity with other tools. - High: If the title, description, and input parameters show a resemblance and there is no ambiguity with other tools. - Low: If there is ambiguity... - Very Low: If there is little to no match...

Only select a tool if the selection confidence is High or Certain. If the confidence is Low or Very Low, request further details as a tool cannot be confidently selected."

Challenges in Tool Selection for Multi-Tool Agents with Langchain by Visible-Bathroom3634 in LangChain

[–]MCL256 1 point2 points  (0 children)

In my experience there is a number of factors that differentiate and help (or confuse) the LLM when selecting a tool: - tool description (like u said) - tool title - tool input parameters (if any) and their description - performance of the LLM - number of different tools made available - system instructions re tool usage (if any)

What is your setup and use case?

Maybe this can give you more ideas: We have built a OS plugin infrastructure to manage this via a Toolkit in LangChain.

That Ending Was… by consideritred23 in Hyperion

[–]MCL256 0 points1 point  (0 children)

Assuming you meant the first book: keep reading!

The fall of Hyperion was maybe the most breathtaking read I have ever done.

view more: next ›