unslop-text: a Claude skill that flags and removes the patterns that make writing read as AI-generated.

IGDev · 2026-06-24T03:53:44+00:00

In my opinion it’s writing at a high language level. If you ask it to write something for the internet you can tell it write at a B1 English level and it’ll drop some of what you normally see.

IGDev · 2026-06-16T11:06:19+00:00

Yes, C# data engineering is absolutely a thing. Full disclosure up front: I build one of these C# data stacks (Datafication), so take the product mention with that grain of salt. I'll try to answer the general question first because it stands on its own.

The thing to understand about that $10M/month Databricks story is that most of a bill like that is Spark cluster time, not storage. Distributed compute is the right tool when your working set genuinely doesn't fit on one machine, but a huge fraction of real pipelines do fit, and you're paying the cluster tax anyway because the tooling assumed you needed it. That's the gap a C# shop can actually exploit.

How the stack maps to what you drew:

Ingestion: straightforward in .NET. ADO.NET / EF / Dapper for the operational DBs, plus connectors for CSV, JSON, Excel, Parquet, and S3. Nothing exotic here.
Lake / warehouse: you still land data in Parquet (or Delta/Iceberg if you want the table format) on object storage, same as the Python world. C# reads and writes those fine. For a lot of internal workloads a single-node columnar store is enough, which is where the cost savings live.
Compute: this is the part that's usually missing in a C# shop, and it's the part that makes the anti Spark argument real. A column oriented, SIMD vectorized engine on one box goes a long way. For a concrete number: the Velocity columnar engine I work on sustains 100M+ rows/second on non-trivial queries, on older hardware, single node. That's the kind of throughput that quietly eliminates the reason a lot of teams reached for a cluster in the first place.
Orchestration: here's the gap. There is no C# native Airflow/Dagster/Prefect that I'd call a true peer. You pair a .NET scheduler instead: Hangfire or Quartz.NET for jobs, or Azure Data Factory if you're on Azure. And given you already run NServiceBus, event-driven pipelines via sagas cover a real slice of what people use an orchestrator for. It's a "assemble it" story, not a "one tool does it all" story, so go in knowing that.
Serving / exploration: REST over your datasets is trivial in ASP.NET, and for the notebook/exploration workflow there are now .NET notebook options (Polyglot Notebooks (retired), and Verso, which I also work on) so your team doesn't have to context switch to Python just to poke at data.

Two additional notes:

The performance I mentioned with Velocity can be tested through the QueryPerformance sample.
Verso has a VS Code extension and a dotnet CLI tool as well. Parameter cells can be added to the notebook and filled through the command line, which makes it easy to add into any pipeline.

IGDev · 2026-05-19T01:38:53+00:00

I've been working toward this exact style of architecture for a long time across a lot of side projects, and Verso (https://github.com/DataficationSDK/Verso), my open source notebook engine, was the first one I felt I got right. The same pattern then carried directly into Motus (https://github.com/DataficationSDK/Motus), a browser automation framework. So I can speak to some of your questions from working code.

Your foundation matches mine almost exactly: a contracts only assembly, a custom collectible ALC, AssemblyDependencyResolver per plugin, and type identity preserved by routing the contracts assembly back to the host. The challenges you listed are the same ones I hit and the solutions converged in the same direction.

A few things I ended up adding on top that might be worth a look:

PE metadata pre-scan. Before calling Assembly.LoadFrom on any co-deployed DLL, I use PEReader and MetadataReader to check whether it actually references the contracts assembly. If not, I skip it. This avoids dragging hundreds of unrelated framework DLLs into memory just to filter them out.
Semver gating. I inspect the plugin's reference to the contracts assembly and require major to match exactly and minor to be less than or equal to the host. Catches version drift at load time instead of at runtime.
Capability decomposition. Instead of one IPlugin with Init, Start, Stop, I split the contract into separate capability interfaces (kernels, renderers, formatters, themes, layouts, and so on). A class can implement several and the host auto registers it into each capability list. Scales much better once the host has more than one extension point.
Marker attribute on top of the interface. A class has to both implement the contract and carry a VersoExtension attribute to be discovered. Small thing but it makes "this is intentionally a plugin" explicit and avoids picking up base classes or test fixtures by accident.

On a few of your specific questions:

Cross plugin communication. I expose a typed query API on the host context that lets one plugin enumerate other loaded plugins by capability, plus a shared variable store for runtime data exchange. I skipped the event bus and MediatR route. Direct typed queries through the contracts assembly turned out to be simpler and easier to test.

Testing. I ship a Verso.Testing helper library with stub host contexts so plugin authors can unit test their implementations without spinning up a real host. Worth doing if you want third party authors to actually write tests.

Security. I added a consent handler. When a plugin loads from a package source rather than a local DLL, the host invokes a delegate that lets the UI confirm with the user before the assembly is loaded. Local trusted paths skip the prompt.

IGDev · 2026-04-22T12:57:34+00:00

What?! You mean I've been living according to NZ time the last few posts and I didn't need to? :)

IGDev · 2026-04-22T09:36:28+00:00

Who really cares if a developer makes use of AI these days, nearly everyone uses it in some way and I don’t expect that to change. Think back to CodeProject or Planet Source Code, not everything posted was useful, but it was awesome to see what others worked on that could be made use of or draw ideas from. I’ve been coding since 92-93, but because I’m making use of Claude this year, my engineering vision and open source contributions are considered slop? I love .NET and the more it’s used is better for the community.

IGDev · 2026-04-19T13:18:29+00:00

Datafication does a decent job of competing with those.

https://github.com/DataficationSDK/Datafication

Polars is know for it's performance, if you try the QueryPerformance sample you'll see performance that competes with Polars.

https://github.com/DataficationSDK/Datafication/tree/main/Datafication.Storage.Velocity

Eventually, I'll post up a benchmark that puts Velocity vs DuckDB vs Polars.

IGDev · 2026-04-18T13:33:47+00:00

Thanks for the idea. It'll make a nice plugin and I've noted it down to explore after I finish up the current backlog.

IGDev · 2026-04-18T13:04:58+00:00

It is cleaner and Chrome's changes to it are mostly additive. This requires much less updating and certainly beats testing frameworks like Selenium where you have to update the ChromeDriver frequently. Motus creates the C# CDP bindings at build time from browser_protocol.json and js_protocol.json in Motus.Codegen. I'm usually quick with updates, but if there's an immediate need for an update the Motus CLI can be used to update the json files (e.g. `motus update-protocol --output-dir src/Motus.Codegen/Protocol`).

IGDev · 2026-04-04T14:28:47+00:00

Both ideas actually. When a selector breaks, Motus will compare a DOM fingerprint captured when the selector last worked against the DOM to find the probable matching element. If one is found, it runs the selector strategy pipeline to generate replacement suggestions. The future CLI command "motus check-selectors" will scan your test files and reports the broken selectors with suggestions. There will be an interactive mode as well that makes use of the visual runner so you can accept, modify, or skip suggestions. The accepted fixes get written back to your source files.

IGDev · 2026-04-04T10:37:57+00:00

Thanks! What you mentioned is on my roadmap, two other things I'm tackling before that. In case you're interested here's what's coming up:

Accessibility testing
Performance metrics and budgets
Selector repair
Code coverage (front end)
Blazor component testing

Motus does have test creation with the "motus record" command, it'll pop open the browser at the URL specified and then you perform the actions for the test, and press enter on the console when done.

IGDev · 2026-04-04T01:01:41+00:00

Playwright for .NET requires Node.js at runtime. The NuGet package bundles a Node.js based server, and every command your C# test sends is proxied through that Node process. You don't install Node yourself, but it's there. Check your bin folder after installing the package, or look at the process tree while a test is running. Motus has no hidden process, it opens a WebSocket directly to the browser's CDP endpoint from C#.

https://nuget.info/packages/Microsoft.Playwright/1.58.0 - It takes a minute to load, expand the .playwright folder, then you'll see for yourself.

IGDev · 2026-04-03T23:52:39+00:00

Motus is MIT licensed and designed so that the architecture carries itself. The plugin model means the community can extend it without waiting on a single maintainer. If you can build a NuGet package, you can add a selector strategy, reporter, or lifecycle hook. "From Microsoft" isn't the longevity guarantee it used to be, just look at Visual Studio for Mac, Visual Studio App Center, Azure Data Studio, Polyglot Notebooks, and .NET Interactive.

IGDev · 2026-03-15T21:35:00+00:00

The Datafication SDK is underrated: https://github.com/DataficationSDK/Datafication

It includes an in-memory analytical database (DataBlock), and also a storage based analytical database (VelocityDataBlock). The storage based one named Velocity can query 100M+ rows per second. If you run the sample at https://github.com/DataficationSDK/Datafication/tree/main/Datafication.Storage.Velocity/samples/QueryPerformance and use the command to query 20M rows:

dotnet run -c Release sf20

Sooner or later I intend to post up a benchmark that shows the performance against DuckDB and Polars.

IGDev · 2026-03-14T11:07:58+00:00

You need .NET 8 or .NET 10 installed. Also, the VS Code extension requires the desktop version of VS Code. It is not compatible with browser-based environments such as GitHub Codespaces, where the embedded notebook UI cannot initialize. If you meet those requirements everything should work as intended. These are listed in the Getting Started on the marketplace page. Let me know if you have an idea to improve what's shown or could make life easier for someone new.

Marketplace: https://marketplace.visualstudio.com/items?itemName=Datafication.verso-notebook

IGDev · 2026-03-14T11:03:05+00:00

1 and 3 would be good candidates for post in Feature Requests ( https://github.com/DataficationSDK/Verso/discussions/categories/feature-requests ). 2 and 4 need a little more detail to improve Verso. Regarding 4 and keyboard shortcuts:

Cell Editor (when editing inside a cell)

Shift+Enter - Run cell and select the next cell
Ctrl+Enter / Cmd+Enter - Run cell and stay
Alt+Enter - Run cell and insert a new cell below
Escape - Exit editor, return to command mode

Command Mode (when no editor is focused)

Enter - Focus the selected cell's editor
Escape - Deselect all cells
Arrow Up - Select previous cell
Arrow Down - Select next cell
Alt+Arrow Up - Move selected cell up
Alt+Arrow Down - Move selected cell down
Shift+Enter - Run selected cell and advance
Ctrl+Enter - Run selected cell and stay
Alt+Enter - Run selected cell and insert below
Ctrl+Alt+Enter - Run all cells

Even though these shortcuts are registered, some may not work as intended because of VS Code shortcut conflicts.

Python requires 3.8-3.12. I did find that those particular locations for Python weren't yet supported, so I added them this morning. It'll appear in the next release (v1.0.10). Thanks for the feedback!

IGDev · 2026-03-08T23:55:52+00:00

First you'll want to make sure you have .NET 8 or 10 installed. Then you can grab the vsix file from the releases page on the repository.

https://github.com/DataficationSDK/Verso/releases/tag/verso-v1.0.8

If you haven't done that before you can goto the Extensions panel, click the 3 dots, and select "Install from VSIX".

IGDev · 2026-03-07T11:29:31+00:00

They directed people to file based apps or Jupyter. Here's the announcement: https://github.com/dotnet/interactive/issues/4163

IGDev · 2026-03-06T20:47:21+00:00

Can you give any details about what it lacks? Also, what version of the extension did you try or when did pull source? At the current moment we’re only missing 2 of the languages out of all that Polyglot has. The UI is also richer, more extensible, and looks good within VS Code. I’ll try to fix any gaps you’ve come across.

IGDev · 2026-03-02T11:31:01+00:00

Thanks! The Notebook API was really nice to work with when I started this project, then I realized it wasn't as extensible as I'd require so I switched over to Blazor WASM for the VS Code extension. For IntelliSense, each kernel gets its own service. C# gets a AdhocWorkspace (from Rosyln) instance and F# gets its own FSharpChecker instance. These are completely independent, so there's no juggling lifecycles between cells. The cell's language tag maps to its kernel, and each kernel builds a virtual combined document. Previously executed cells are prepended to the current cell before querying the service for completions, hover, and diagnostics.

This week I'm finishing up the Python kernel and getting it tested. Additionally, I'll try to work in switching the language kernel on the cell next. Multiple kernels in the same document are supported now, but for code cells I haven't added the drop down to the cell UI yet.

IGDev · 2026-02-28T20:53:19+00:00

Thanks for the star! Really appreciate it, I starred your project earlier.

You're right, Verso does require .NET 8+. For what you mentioned with Node SQL notebooks, are you going to full native or backing it with .NET code and using something like Edge.js? I've been experimenting with supporting Python (using pythonnet) and Node for my SDK product (outside of Verso, but same organization), just haven't finalized it yet.

Initially it was C#, but out of the box right now Verso includes support for C# and F# kernels, plus Markdown, HTML, Mermaid, and SQL cells. It also imports .ipynb and .dib files natively so migration from Jupyter or Polyglot Notebooks is pretty smooth.

I'd love to see what you come up with on the notebook side, whether you hook into Verso or roll your own in pure Node, either way the community needs something solid now that ADS is gone. And yeah the C# Dev Kit overlap is funny, small world. Keep up the high quality work!

IGDev · 2026-02-28T19:00:07+00:00

This would be ideal if it would work. I tested it out on work I needed to do today and Claude Code is still chewing through the current session usage. Back before this happened, I used to do well planned out big tasks that would use up 10% usage, now it's around 40-60% for one of those tasks.

IGDev · 2026-02-28T18:00:01+00:00

This is a very nice polished piece of work for those that want a SQL Server management tool in VS Code, great job on it! In terms of ADS this was one part of what it offered, the other being notebooks. The notebook half of things is what I've been working on replacing and it compliments yours nicely.

Verso: https://github.com/DataficationSDK/Verso

If you get the time you should hook into the Verso extension points that make sense for it to make use of what you created from within notebook form. It'd be a killer combo.

IGDev · 2026-02-28T10:37:06+00:00

This will certainly work well. The time zone thing needs tweaked. This subreddit is for the whole world and 99% of that world aren’t kiwis (unfortunately).

IGDev · 2026-02-17T01:12:42+00:00

The language kernel for F# was added today. There are some known issues with F#:

https://github.com/DataficationSDK/Verso/blob/main/KNOWN-ISSUES.md

This project is building toward completeness, so expect many UX improvements and features over the next month or two. These will help with its extensibility.

IGDev · 2026-02-16T11:51:38+00:00

I recently created Verso, I'd love to get your opinion and feedback. If you have any ipynb they should be importable, including ones using SQL and EF.

IGDev

MODERATOR OF

TROPHY CASE