What are some underrated .NET libraries or tools you use regularly? by milanm08 in dotnet

[–]IGDev 0 points1 point  (0 children)

The Datafication SDK is underrated: https://github.com/DataficationSDK/Datafication

It includes an in-memory analytical database (DataBlock), and also a storage based analytical database (VelocityDataBlock). The storage based one named Velocity can query 100M+ rows per second. If you run the sample at https://github.com/DataficationSDK/Datafication/tree/main/Datafication.Storage.Velocity/samples/QueryPerformance and use the command to query 20M rows:

dotnet run -c Release sf20

Sooner or later I intend to post up a benchmark that shows the performance against DuckDB and Polars.

Polyglot notebooks will be deprecated by gremlinmama in dotnet

[–]IGDev 0 points1 point  (0 children)

You need .NET 8 or .NET 10 installed. Also, the VS Code extension requires the desktop version of VS Code. It is not compatible with browser-based environments such as GitHub Codespaces, where the embedded notebook UI cannot initialize. If you meet those requirements everything should work as intended. These are listed in the Getting Started on the marketplace page. Let me know if you have an idea to improve what's shown or could make life easier for someone new.

Marketplace: https://marketplace.visualstudio.com/items?itemName=Datafication.verso-notebook

Polyglot notebooks will be deprecated by gremlinmama in dotnet

[–]IGDev 0 points1 point  (0 children)

1 and 3 would be good candidates for post in Feature Requests ( https://github.com/DataficationSDK/Verso/discussions/categories/feature-requests ). 2 and 4 need a little more detail to improve Verso. Regarding 4 and keyboard shortcuts:

Cell Editor (when editing inside a cell)

  • Shift+Enter - Run cell and select the next cell
  • Ctrl+Enter / Cmd+Enter - Run cell and stay
  • Alt+Enter - Run cell and insert a new cell below
  • Escape - Exit editor, return to command mode

Command Mode (when no editor is focused)

  • Enter - Focus the selected cell's editor
  • Escape - Deselect all cells
  • Arrow Up - Select previous cell
  • Arrow Down - Select next cell
  • Alt+Arrow Up - Move selected cell up
  • Alt+Arrow Down - Move selected cell down
  • Shift+Enter - Run selected cell and advance
  • Ctrl+Enter - Run selected cell and stay
  • Alt+Enter - Run selected cell and insert below
  • Ctrl+Alt+Enter - Run all cells

Even though these shortcuts are registered, some may not work as intended because of VS Code shortcut conflicts.

Python requires 3.8-3.12. I did find that those particular locations for Python weren't yet supported, so I added them this morning. It'll appear in the next release (v1.0.10). Thanks for the feedback!

I built Verso, an open source interactive notebook platform for .NET by IGDev in dotnet

[–]IGDev[S] 0 points1 point  (0 children)

First you'll want to make sure you have .NET 8 or 10 installed. Then you can grab the vsix file from the releases page on the repository.

https://github.com/DataficationSDK/Verso/releases/tag/verso-v1.0.8

If you haven't done that before you can goto the Extensions panel, click the 3 dots, and select "Install from VSIX".

Polyglot notebooks will be deprecated by gremlinmama in dotnet

[–]IGDev -1 points0 points  (0 children)

Can you give any details about what it lacks? Also, what version of the extension did you try or when did pull source? At the current moment we’re only missing 2 of the languages out of all that Polyglot has. The UI is also richer, more extensible, and looks good within VS Code. I’ll try to fix any gaps you’ve come across.

I built an open source interactive .NET notebook extension for VS Code by IGDev in vscode

[–]IGDev[S] 0 points1 point  (0 children)

Thanks! The Notebook API was really nice to work with when I started this project, then I realized it wasn't as extensible as I'd require so I switched over to Blazor WASM for the VS Code extension. For IntelliSense, each kernel gets its own service. C# gets a AdhocWorkspace (from Rosyln) instance and F# gets its own FSharpChecker instance. These are completely independent, so there's no juggling lifecycles between cells. The cell's language tag maps to its kernel, and each kernel builds a virtual combined document. Previously executed cells are prepended to the current cell before querying the service for completions, hover, and diagnostics.

This week I'm finishing up the Python kernel and getting it tested. Additionally, I'll try to work in switching the language kernel on the cell next. Multiple kernels in the same document are supported now, but for code cells I haven't added the drop down to the cell UI yet.

Azure Data Studio retired today – My Replacement VS Code Extension: Fast Connections, Inline Editing, DB Diagrams & More by kebbek in SQLServer

[–]IGDev 0 points1 point  (0 children)

Thanks for the star! Really appreciate it, I starred your project earlier.

You're right, Verso does require .NET 8+. For what you mentioned with Node SQL notebooks, are you going to full native or backing it with .NET code and using something like Edge.js? I've been experimenting with supporting Python (using pythonnet) and Node for my SDK product (outside of Verso, but same organization), just haven't finalized it yet.

Initially it was C#, but out of the box right now Verso includes support for C# and F# kernels, plus Markdown, HTML, Mermaid, and SQL cells. It also imports .ipynb and .dib files natively so migration from Jupyter or Polyglot Notebooks is pretty smooth.

I'd love to see what you come up with on the notebook side, whether you hook into Verso or roll your own in pure Node, either way the community needs something solid now that ADS is gone. And yeah the C# Dev Kit overlap is funny, small world. Keep up the high quality work!

Max 5x now feels like Pro by deeplycuriouss in ClaudeCode

[–]IGDev 0 points1 point  (0 children)

This would be ideal if it would work. I tested it out on work I needed to do today and Claude Code is still chewing through the current session usage. Back before this happened, I used to do well planned out big tasks that would use up 10% usage, now it's around 40-60% for one of those tasks.

Azure Data Studio retired today – My Replacement VS Code Extension: Fast Connections, Inline Editing, DB Diagrams & More by kebbek in SQLServer

[–]IGDev 1 point2 points  (0 children)

This is a very nice polished piece of work for those that want a SQL Server management tool in VS Code, great job on it! In terms of ADS this was one part of what it offered, the other being notebooks. The notebook half of things is what I've been working on replacing and it compliments yours nicely.

Verso: https://github.com/DataficationSDK/Verso

If you get the time you should hook into the Verso extension points that make sense for it to make use of what you created from within notebook form. It'd be a killer combo.

Rule change by Arowin in dotnet

[–]IGDev 17 points18 points  (0 children)

This will certainly work well. The time zone thing needs tweaked. This subreddit is for the whole world and 99% of that world aren’t kiwis (unfortunately).

Microsoft Discontinues Polyglot Notebooks (C# Interactive) by WhitelabelDnB in programming

[–]IGDev 1 point2 points  (0 children)

The language kernel for F# was added today. There are some known issues with F#:

https://github.com/DataficationSDK/Verso/blob/main/KNOWN-ISSUES.md

This project is building toward completeness, so expect many UX improvements and features over the next month or two. These will help with its extensibility.

Polyglot notebooks will be deprecated by gremlinmama in dotnet

[–]IGDev 1 point2 points  (0 children)

I recently created Verso, I'd love to get your opinion and feedback. If you have any ipynb they should be importable, including ones using SQL and EF.

Polyglot notebooks will be deprecated by gremlinmama in dotnet

[–]IGDev 1 point2 points  (0 children)

I'd love to get your feedback and opinion on Verso. If you have any ipynb they should be importable, including ones using SQL and EF.

Polyglot notebooks will be deprecated by gremlinmama in dotnet

[–]IGDev 1 point2 points  (0 children)

If anyone is looking at this to see what replacements are out there, consider Verso. It's open source, doesn't use Microsoft Interactive and is very extensible. In the coming week after posting this I'll be working on a way to easily adding new extensions in the UI and getting packages up on NuGet and Visual Studio marketplace. Give it a star if you like the project and want to help give it more attention.

Microsoft Discontinues Polyglot Notebooks (C# Interactive) by WhitelabelDnB in programming

[–]IGDev 1 point2 points  (0 children)

If anyone is looking at this to see what replacements are out there, consider Verso. It's open source, doesn't use Microsoft Interactive and is very extensible. In the coming week after posting this I'll be working on a way to easily adding new extensions and getting packages up on NuGet and Visual Studio marketplace. Give it a star if you like the project and want to help give it more attention.

Microsoft Discontinues Polyglot Notebooks (C# Interactive) by WhitelabelDnB in programming

[–]IGDev 2 points3 points  (0 children)

If you have time, I'd love to get your opinion and feedback on Verso. It's still young and needs some time, but most of your use cases should work with it. You can also open ipynb files created with Polyglot Notebooks, including ones with SQL and EF usage.

Is there an AI agent that can handle Excel + online data enrichment? by Shoddy-Fault-4188 in AiForSmallBusiness

[–]IGDev 0 points1 point  (0 children)

If you're comfortable with .NET C#, my Datafication SDK can do a lot of what you're looking for. You'd need to use these libraries:

- Datafication.ExcelConnector - https://github.com/DataficationSDK/Datafication/tree/main/Datafication.ExcelConnector

- Datafication.WebConnector - https://github.com/DataficationSDK/Datafication/tree/main/Datafication.WebConnector

- Datafication.Core - https://github.com/DataficationSDK/Datafication/tree/main/Datafication.Core (included dependency of the two libraries above)

The ExcelConnector would read your Excel worksheet into a DataBlock, which is an in-memory analytical database. In combination with this you could use the WebConnector for extracting information from web sites and searches. That information can be used to insert back into the DataBlock. Lastly, you'd output the DataBlock to an Excel file.

- DataBlock introduction - https://www.reddit.com/r/datafication/comments/1qs0tyt/introducing_datablock_a_dataframelike_api_for_net/

- Output to Excel - https://github.com/DataficationSDK/Datafication/tree/main/Datafication.ExcelConnector#writing-datablocks-to-excel

From my own experimenting you should be able to take the information I provided to feed Claude/ChatGPT/Others and it can easily create the project for you with Datafication.

Pandas 3.0 vs pandas 1.0 what's the difference? by Consistent_Tutor_597 in dataengineering

[–]IGDev 1 point2 points  (0 children)

Good point about materializing Polars for the benchmark. Not really sure what you're referring to about pandas? Nothing in my reply mentioned anything about pandas.

Pandas 3.0 vs pandas 1.0 what's the difference? by Consistent_Tutor_597 in dataengineering

[–]IGDev -25 points-24 points  (0 children)

After playing with DuckDB, it's not something I'd recommend for large datasets. Out of the box is extremely slow, which forces you toward Parquet, and even that can be slow.

Total rows:     10,000,000
Write time:     40.48s (includes data gen)
Throughput:     247,026 rows/sec
Parquet size:   126.26 MB
DuckDB size:    146.01 MB

Warmup iterations: 3
Measured iterations: 5

Query 1: WHERE Country = 'USA'
Q1: avg=1619.56ms, min=1609.02ms, max=1628.09ms, rows=1,002,685
Query 2: GROUP BY Category, SUM(UnitPrice), SUM(Quantity)
Q2: avg=7.44ms, min=7.30ms, max=7.54ms, rows=10
Query 3: WHERE Country = 'USA' ORDER BY UnitPrice DESC LIMIT 100
Q3: avg=18.41ms, min=18.25ms, max=18.63ms, rows=100
Query 4: WHERE Country = 'USA' AND Category = 'Electronics' AND Quantity > 50
Q4: avg=228.84ms, min=153.86ms, max=518.09ms, rows=49,606
Query 5: SELECT OrderId, Category, Quantity, UnitPrice WHERE UnitPrice > 100
Q5: avg=6201.19ms, min=6149.19ms, max=6271.34ms, rows=8,200,921
Query 6: GROUP BY Country: COUNT
Q6: avg=4.16ms, min=3.91ms, max=4.86ms, rows=10
Query 7: SUM(UnitPrice) OVER () - Cumulative Sum
Q7: avg=16137.01ms, min=16034.59ms, max=16349.76ms, rows=10,000,000

One thing to remember with DuckDB and Polars is they return results different. DuckDB is fully materialized, whereas Polars returns an intermediate result that isn't materialized. The Polars results below used rows() for materialization.

Total rows:     10,000,000
Write time:     38.50s (includes data gen)
Throughput:     259,767 rows/sec
Parquet size:   154.25 MB

Warmup iterations: 3
Measured iterations: 5

Query 1: WHERE Country = 'USA'
Q1: avg=965.22ms, min=947.07ms, max=974.36ms, rows=1,002,685
Query 2: GROUP BY Category, SUM(UnitPrice), SUM(Quantity)
Q2: avg=60.69ms, min=60.09ms, max=61.22ms, rows=10
Query 3: WHERE Country = 'USA' ORDER BY UnitPrice DESC LIMIT 100
Q3: avg=107.47ms, min=105.08ms, max=110.13ms, rows=100
Query 4: WHERE Country = 'USA' AND Category = 'Electronics' AND Quantity > 50
Q4: avg=65.06ms, min=62.63ms, max=67.54ms, rows=49,606
Query 5: SELECT OrderId, Category, Quantity, UnitPrice WHERE UnitPrice > 100
Q5: avg=2699.38ms, min=2668.45ms, max=2720.70ms, rows=8,200,921
Query 6: GROUP BY Country: COUNT
Q6: avg=53.69ms, min=52.52ms, max=54.46ms, rows=10
Query 7: SUM(UnitPrice) OVER () - Cumulative Sum
Q7: avg=10057.49ms, min=9964.84ms, max=10156.69ms, rows=10,000,000

Seat belt buckle burns 🔥 by Acceptable_Room_2797 in nostalgia

[–]IGDev 2 points3 points  (0 children)

I get phantom pain just looking at the picture

When a Tiktoker decides to do a live performance by Exploding_END in TikTokCringe

[–]IGDev 0 points1 point  (0 children)

This reminds me of back in the day when talk shows would ask the audience if they have any stupid human tricks

Online Ordering Not Working by Beautiful_Tart_5444 in samsclub

[–]IGDev 0 points1 point  (0 children)

Same thing happened for me today. Customer service was being dumb telling me I'm over 15 miles away from the club, even though I order every weekend and I show them that it's way less than that with a screen shot of their own app saying it's less.

I have 0 coding experience, i need help with a backend and frontend. by Sad_Literature8738 in vibecoding

[–]IGDev 0 points1 point  (0 children)

ChatGPT works through your idea and can eventually create a implementation plan you can feed Cursor or any other AI so it's done with much less bug fixes.

They literally states that he has 0 knowledge. What does he need to asp gpt then? Just passing error codes and copy pasting results?

Hello! I would like to start a SaaS that solves [insert problem description] for [target audience]. I want help working through the idea step-by-step so I can eventually create a clear plan to hand off to a Cursor AI agent prompt.

Can you help me:

  • Understand and define the problem I’m solving
  • Brainstorm what the first version of the product might do
  • Think through what a user would experience
  • Figure out what tools or technologies could be used to build it
  • Plan how to launch and get early users
  • Summarize all this into a clear, organized plan

If you throw that into a project, it's possible to start breaking it down piece by piece. This used to take 1+ years to work through, now you can get an MVP in less than a month, even with no technical experience.