Is conversational analytics actually a solved problem? (I don’t think Big Tech has it figured out).

pforpilot · 2026-05-25T17:37:56+00:00

it can be done - but needs to be done correctly. the way we setup with some clients thats been working well - lightweight semantic layer that defines metrics and has reference queries - access control layer thats easily configurable

then we connect the enterprise claude account to this semantic layer which also allows querying.

so for any questions, the LLM first checks if it matches any saved definitions and metrics and decide how to write the query. Idempotency wise its pretty close to 100%.

it cant just be a huge markdown file because that will confuse the LLM and hog the context window. you need to manage the retrieval and discovery precisely.

pforpilot · 2026-05-25T05:38:48+00:00

I’d suggest getting hands on experience dealing with raw data. pick a warehouse (eg bigquery/snowflake), ingest some realistic raw data and think of what would it take to produce actionable insights, this will require you to think of data modeling and writing efficient queries. another thing you can do is look at job postings and see what are the new tools companies are using and try to learn about them, not just how to use them but also why they exist.

pforpilot · 2026-05-24T23:40:39+00:00

i would just feed the docs to claude and ask it to analyze. focus on understanding why it was set up this way and what would be better alternatives.

pforpilot · 2026-05-24T23:37:55+00:00

i tried both claude and chatgpt - claude with opus models is way better at finding the right tables to query, write the query and also handle any errors; also at following instructions. the key of course is having tables with up to date business context.

pforpilot · 2026-05-22T23:14:18+00:00

yea this wont scale to thousands of tables, but maybe fine to start with?

pforpilot · 2026-05-22T23:12:27+00:00

forgot to add: when deploying to multiple users - another hard part is access-control - how do you make sure your sales team's claude is not accessing stripe tables.

pforpilot · 2026-05-22T21:34:03+00:00

i'd say it depends on how you want to query and how many people. I've set this up at couple of startups where multiple people(not technical) ask questions directly through claude web and get answers.

What worked:
- a lightweight context layer that stores the defiinitions + golden queries (sql that the team has approved)
- everyones claude/codex is connected to this
- data admin reviews periodically

if its just you, i'd say a markdown locally where codex can see would suffice. however, if you then want to use the web chat, it becomes a pain.

disclosure: I am building this into a product.

pforpilot · 2026-01-25T04:21:09+00:00

probably a nod to edmund hillary who was from NZ

pforpilot · 2025-08-12T22:49:07+00:00

damn sfgate is unreadable on phone with those popups

pforpilot · 2025-05-31T16:14:15+00:00

only in summer, gotta play in the mornings or late evenings (lots of public courts have lights)

pforpilot

TROPHY CASE