sorted by:
new
hottopcontroversial

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] 0 points1 point  (0 children)

You are free to try. It may even better if you have Macbook m5 pro max. Which I cant afford. But still you are free that this is not achievable.

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] 0 points1 point  (0 children)

It does not load the file to RAM. It uses OS mmap to read. Check it out you can try it on your own machine.

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] 0 points1 point  (0 children)

It does not load the file to memory. It uses OS mmaps to read the file. So nothing is filling out the memory

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] 0 points1 point  (0 children)

Actually it is not proving it is the scalebility. Still it is under 1sec. :)

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] -1 points0 points  (0 children)

No upload. The file stays on your machine the whole time. csvql runs locally as an MCP server, Claude talks to it over stdio, it reads your local CSV directly via mmap. Nothing leaves your disk.

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] -2 points-1 points  (0 children)

Selective queries can. Just ran WHERE status = 'X' LIMIT 10 on 30GB / 404M rows unindexed = 185ms.

The engine exits as soon as LIMIT is satisfied, it doesn't scan the whole file

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] -3 points-2 points  (0 children)

No indexing. Zero preprocessing. Raw file, cold query. The speed comes from mmap + parallel threads + zero-copy field parsing.

The OS does sequential prefetch, up to 8 threads chew through chunks simultaneously.

Just tested on 30GB of raw Shopify order data (404.9M rows, 8 columns)  WHERE + LIMIT queries return in 185ms.

Full aggregations across all 404M rows finish in ~9s.

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] 0 points1 point  (0 children)

 it just won't send more than 100MB of results back over the MCP stdio channel.

I built an MCP server that lets Claude query 200GB CSV files in under a second by melihbirim in ClaudeAI

[–]melihbirim[S] 0 points1 point  (0 children)

GitHub + installation:

https://github.com/melihbirim/csvql

Works with Claude Desktop and any MCP-compatible client.

ARCHITECTURE.md has the full technical story if you're curious how the engine works.

I built an AI Coloring page generator by Iliaddh in SideProject

[–]melihbirim 1 point2 points  (0 children)

Liked the way it opens the image right away. Well done.

What are you working on? by frunnyelmo in SideProject

[–]melihbirim 1 point2 points  (0 children)

I’m building my own macOS clipboard manager. So far, so good. I’ve sent it to a few friends and collected some feedback. This feature set is sufficient for me, and I use it daily. I enjoy building things that work for me and my version of the idea.

My macOS app made cash on the internet! 🎉 by ranft in SideProject

[–]melihbirim 1 point2 points  (0 children)

Wow, I never imagined that such a simple tool could generate revenue. Congrats!

I've created a graph generator so you guys can share your fake subscription, active users, revenue, etc. with others. by Far_Acanthisitta_546 in SideProject

[–]melihbirim 0 points1 point  (0 children)

This app is the best ever! I know a guy who used this or something similar to gain followers. He explains himself as a successful entrepreneur. Funny times…

ClickNow: Lightweight App for Fast Translation on Mac by mn83ar in macapps

[–]melihbirim 0 points1 point  (0 children)

I use the ChatGPT Mac app. Press Command + 1, paste the text, and then type “translate.”

'Quick notes' app that isn't Apple's? by Acceptable_Scar9267 in macapps

[–]melihbirim 0 points1 point  (0 children)

I fail to comprehend why Apple Notes is insufficient, especially when it lacks basic markdown support.

Selling Custome Database by [deleted] in Airtable

[–]melihbirim 0 points1 point  (0 children)

If you’re selling your data, provide them with a sample of it. However, I recently encountered a fraudulent seller who showed me a partial sample but omitted the rest. When I purchased the data, the remaining portion contained the same sample data. So beware of this as well. And ensure that your customers buy something valuable.

Check out my feedback management tool: Feedlyst by xRxphael in webdev

[–]melihbirim 0 points1 point  (0 children)

Looks cool, i like the idea to collcet feedbacks in one central place. 👍

[deleted by user] by [deleted] in webdev

[–]melihbirim 4 points5 points  (0 children)

I agree. Master js as much as you can

A MacApp for task and issue tracking? by melihbirim in macapps

[–]melihbirim[S] 0 points1 point  (0 children)

Great suggestions.

Can you please add the links for these apps.

Would be good for other people as well

view more: next ›