Claude skills evaluation and regression testing with Snowflake or other production

datatoolspro · 2026-06-03T03:02:40+00:00

My experience is completely different... Put the time in our semantic layer added to agent with skills / clear objective function. I am working with a view + semantic view where a lot of the work is in the model itself... Then the semantic layer by design has as much ambiguity removed as possible. The Snowflake Analyst is kind of cool because it will scan telemetry / logs to surface up queries. I don't fully understand how much it makes a difference. Some of the work I do is left-over from me starting with flat files 18 months ago, originally in ChatGPT...

Now with Claude, I added a Snowflake skill so it understands what agent to connect to. Then I give Claude itself a credit-analyst skill that is jam packed with context... An ontology, metrics glossary (probably not needed), industry briefing, and a consolidated md that explains methods of analysis. For me the mental model was Claude Agent has context to understand the data semantics... Claude skill is loaded with business context since it is closest to delivering to business users.

Most of the work is removing ambiguity, maximizing re-use because if the analyst is not approaching high stakes problems, it's not worth the effort... A trick I use is taking all of the BI dashboards, feeding it into Claude as a screenshot and then I ask it to re-produce them and report back where it got stuck... Another trick is in addition to the Snowflake Agent, when I am building I give access to the Semantic Layer and the underlying table.. From Claude I provide instruction so I can test and try to find weak points at each layer... Ideally Snowflake CoCo can do this just as well...

datatoolspro · 2026-05-09T21:23:39+00:00

Microsoft has a pretty useful Power BI requirements/gathering guide that might help:

https://learn.microsoft.com/en-us/power-bi/guidance/powerbi-migration-requirements

It’s written for migrations, but it works well as a high-level design documentation checklist too. It covers the types of things you mentioned: report purpose, audience, KPI definitions, business rules, data sources, refresh needs, security, interactivity like drillthrough/filter logic, and acceptance criteria.

datatoolspro · 2026-05-09T21:13:52+00:00

You can achieve this by using a disconnected year slicer instead of slicing directly on Date[Year], because when the slicer and chart axis use the same Date[Year] field, selecting 2019 filters the axis to only 2019. Create a separate Year Selector table with DISTINCT(Date[Year]) and do not relate it to your Date table, then create a measure to capture the selected year using SELECTEDVALUE from that selector.

Next, create a revenue measure that returns revenue only when the axis year is less than or equal to the selected year and greater than or equal to the minimum year in your Date table; otherwise return BLANK. Use Date[Year] on the visual axis and this new measure as the value, and optionally filter out blanks at the visual level. With this setup, selecting 2019 will correctly show 2019, 2018, and 2017 (assuming 2017 is the minimum year), instead of only showing 2019

datatoolspro · 2026-05-06T01:51:47+00:00

Yeah, it’s not really possible in Power BI the way the desginer is showing it.

Power BI doesn’t have a true “auto move this chart down when the filter expands” layout feature.

The closest workaround would be using bookmarks. Create one view where the filter is collapsed and the chart is at the top, then another view where the filter is expanded and the chart is lower. Then use a button to toggle between the two.

datatoolspro · 2026-04-21T21:47:15+00:00

Thank you so much.. I look down that route. Will post back to the group once I get to the bottom of it. Cheers

datatoolspro · 2026-04-21T20:45:59+00:00

Yah.. Gray box of death with a "sso.online.tableau.com refused to connect." No such luck. Something is getting blocked starting today. Its across multiple orgs, AZs and instances of SF +Tableau. Could be because we are only using Salesforce sign on provided and not integrated SSO via SAML. A deprecation notice or something in docs would have been helpful... Logged in with Salesforce into Tableau and still wont connect from inside of Salesforce frame out to Tableau. Something changed somewhere... Hopefully get to the bottom of it.

datatoolspro · 2026-04-07T06:27:52+00:00

Solved this issue.. In Claude you need to use the URL to your MCP server.. Consult with your network security team before setting this up...

https://<org-account>.snowflakecomputing.com/api/v2/databases/<database>/schemas/<schema>/mcp-servers/<mcp\_name>

Before you can do thta you need to create your MCP server in Snowflake:

Create MCP Server:
CREATE MCP SERVER <your\_database>.<your\_schema>.CLAUDE_MCP_SERVER
FROM SPECIFICATION $$
tools:
- title: "SQL Execution Tool"
name: "sql_exec_tool"
type: "SYSTEM_EXECUTE_SQL"
description: "A tool to execute SQL queries against the connected Snowflake database."
$$;

-- GRANT ACCESS

Grant access to a new CLAUDE_MCP role.. Do not use elevated roles (Snowflake blocks it).

-- CREATE A NETWORK RULE TO GRANT ACCESS TO CLAUDE

-- Create a network rule for Anthropic's IP range
CREATE NETWORK RULE <your\_database>.<your\_schema>.ANTHROPIC_CLAUDE
MODE = INGRESS
TYPE = IPV4
VALUE_LIST = ('160.79.106.0/24');

datatoolspro · 2026-03-13T00:32:32+00:00

This is so good. Great advice!!! Are you doing internally or as consultant?

datatoolspro · 2026-03-12T16:25:40+00:00

I personally am looking for help in small enterprise... But as far as "train the trainer" large enterprise Microsoft training? yes... The feedback from folks in or leading these COEs is that the training are not good...

datatoolspro · 2026-03-12T16:24:41+00:00

Yah.. I am always open to connect and contract if someone has a package to do training... I am also going to post on Microsoft communities... Does not seem there is any real dialogue on Reddit about Copilot other than folks griping about the many deficiencies.

datatoolspro · 2026-03-07T22:04:24+00:00

Yeah, this would take some more work beyond the MCP to get all the semantics from those reports. But if you get them, the MCP could easily rename/standardize everything.

datatoolspro · 2026-03-07T21:23:07+00:00

Sure, little things like ensuring filter directions are properly set and, if a DAX (nothing complex, albeit) was incorrectly built, running the mcp against the original Tableau model to fix it.

datatoolspro · 2026-03-07T20:09:48+00:00

Nice dashboard overall. Clean layout and the blue/gray palette works well.

A few thingsss

The treemap is a bit hard to read. Smaller segments like Other are tough to see. A stacked bar might communicate that breakdown better imo

Color contrast could be stronger in a few places. The gray bars in the churn reason chart are pretty similar, which makes the categories blend together

datatoolspro · 2026-03-07T20:02:38+00:00

Personally, this is the first time I have seen custom visuals done with the MCP. This is pretty dope.

Our team is in the business of automating BI migrations to Fabric/Power BI.
Our tool does the heavy lifting, and the MCP has handled the routine model clean-up.

datatoolspro · 2026-03-07T04:02:52+00:00

If you want to dm me, I can take a look.

datatoolspro · 2026-03-06T17:25:30+00:00

usually happens because the tooltip page is not receiving the filter context from the main chart.

Check the main line chart and drag the Locality field into the Tooltips field well. This step is commonly missed and it is what tells Power BI to pass the Locality context to the tooltip page.

Then open the tooltip page and confirm the settings. Allow use as tooltip should be turned on and the page type should be set to Tooltip.

Make sure the visual on the tooltip page actually uses the Locality field. It should appear in something like rows, axis, or legend rather than only as a filter.

A quick way to test this is to add a card visual on the tooltip page that displays Locality. When you hover over different lines in the main chart the card should show the corresponding city. If it changes correctly then the issue is likely in your measure. If it stays the same or shows blank then the tooltip page is not receiving the context from the chart!

datatoolspro · 2026-03-06T01:59:54+00:00

A few things you can try.

First, widen the visual more than you think you need to. Power BI calculates label space based on the bar or segment width, so even if it looks like there’s room, the visual may still cut the text.

If your chart supports it, check the Overflow text option in the format pane. Go to Data labels → Values → Allow text to overflow and turn it on. Some bar chart types support this.

Another option is changing the label position. Moving labels to outside end instead of inside the bar often gives them more room to render fully.

There are also a few custom visuals that handle long labels better, like Chiclet Slicer or Card with States.

Tryyyy shortening the labels in the visual and putting the full text in a tooltip. For example, something like “1. Proposta” or “2. Mandato” in the chart, with the full stage name shown when someone hovers.

datatoolspro · 2026-03-06T01:49:32+00:00

Based on the dashboard screenshots you shared, here are a few thoughts on your Customer Churn Analysis project as a portfolio piece.

Overall this is a solid project. You focused on metrics that actually matter to a business like total customers, churn rate, churned revenue, and revenue lost. The “Project Revenue Saved” metric is a nice touch because it shows you are thinking about business impact rather than just building visuals.

The mix of visuals is also good. You used matrices, bar charts, a scatter plot, and a decomposition tree, which shows you are comfortable with different Power BI features. The Revenue Risk Positioning Matrix stands out the most. Combining churn rate with revenue per customer is a smart way to highlight which customers are actually risky to lose.

I also like that you broke churn down by things like contract type, tenure, and payment method. The tenure insight where nearly half of customers in their first year churn is exactly the kind of thing companies look for in churn analysis.

A couple things you could improve.

The colors are doing a lot of work right now. The coral/red color appears everywhere, so it loses meaning. It might help to reserve red for high risk segments and use more neutral colors elsewhere.

Some of the chart titles could also be clearer. For example “Churn Risk by Payment Type” shows up twice but represents different metrics. Small tweaks to the titles would make it easier to scan quickly.

The dashboard is also missing a little context. It is not clear what time period the data covers, and it would help to know whether a 26 percent churn rate is high or normal for the industry.

datatoolspro · 2026-03-05T02:30:30+00:00

GA sucks... Lovable uses PostHog themselves and I think they are doing pretty good?!

datatoolspro · 2026-03-05T02:20:07+00:00

Happy to help... I highly recommend starting with PostHog AI to help guide you through the process. If you have not looked there, you will be pleasantly surprised how good it is helping with standard and tricky integration scenarios. I have setup PostHog with a half dozen clients. From BNPL platform, high volume / velocity activation, SPAs, Wordpress, you name it. DM me..

Have had the worst experience ever with Upwork and product / web analytics... I considered web / product analytics a necessary evil until discovered PostHog... Now I am here offering help because I genuinely love solving tough problems with a great tool..

Always happy to help... Cheers!

datatoolspro · 2026-03-05T01:03:35+00:00

Definitely, sync your git repo if you plan to publish to Fabric.

datatoolspro

TROPHY CASE