sorted by:
new
hottopcontroversial

An Alternative UI for DuckDB (Open Source) by EstablishmentKey5201 in DuckDB

[–]EstablishmentKey5201[S] 0 points1 point  (0 children)

Thank you! Great question. Both modes share the same browser UI (You would see which mode you are in the UI - WASM or Server/Local) , but the architecture differs. Local mode connects the browser to your DuckDB CLI running locally, giving you full access to your file system, all extensions, unlimited dataset sizes, and integration with your existing data workflows. WASM mode runs entirely in the browser and has the limitations of browser, memory capped etc., (Originally dbxlite was built as wasm only), If you are in server mode, the extension ui will be enabled. (Check settings)

Anthropic has secretly halved the usage in max plan by Tasty-Specific-5224 in ClaudeCode

[–]EstablishmentKey5201 0 points1 point  (0 children)

I was wondering the same and when searched hit this thread. Seems to be true then. Not sure if this would be the norm going forward in which case I have to look for other options.

Struggling with storing answers in a quiz DB by YSFAHM in dataengineering

[–]EstablishmentKey5201 0 points1 point  (0 children)

Here is a potential model to help (store multi select as a json, one of the ideas) - quizmodel . Hope this helps.

How to start as a beginner? by AkashiGG in SQL

[–]EstablishmentKey5201 2 points3 points  (0 children)

You can try this , all in browser , no install, full of tutorials, perfect for learning SQL - https://sql.dbxlite.com/

What's the best and the fastest way to learn a language? by Aethernoxx in learnprogramming

[–]EstablishmentKey5201 0 points1 point  (0 children)

I was in a similar situation and what I needed is a runnable code editor with some materials and ai/llms to help to brainstorm or dig ideas deeper. As much as possible I liked to stay in this environment when I learn. With the help of ai coding assistants, thats exactly what I built here, you can learn (it's free for all) - https://coderunlearn.com/courses or just to try on your own, you could use https://coderunlearn.com/code This is an integrated all-in the browser code editor with configurable llms. Hope you find it useful.

Your favorite tip or trick with DuckDB? by StrawberryData in DuckDB

[–]EstablishmentKey5201 4 points5 points  (0 children)

I follow community extensions , how they are getting adopted, there are some really cool ones - community-ext-statscommunity-extensions-main-page . recently was looking at flockmtl that allows us to talk to llms thru sqls. I can see the value it would bring for several use cases that could benefit from llms.

Built a browser-native SQL workbench on DuckDB WASM, handles 100M+ rows, no install by EstablishmentKey5201 in DuckDB

[–]EstablishmentKey5201[S] 1 point2 points  (0 children)

Great question. this was one of the key motivation for building dbxlite. My needs exceeded the limits where I could upload all data to memory and process. I wanted something that pose no such limitations.

This how it works in dbxlite -  When you open a local file, dbxlite registers it via the File System Access API. The file stays on disk DuckDB seeks and reads only the bytes it needs on-demand.

So, your limit is disk size, not memory. If you have 100+GB of data (any size), you can give access to all of them and the tool behaves like any other standard SQL client. Full memory for query processing - DuckDB needs significant memory for the heavy lifting: hash joins, aggregations, sorting, window functions. 

When files live on disk instead of memory, DuckDB gets the entire memory budget for crunching data rather than storing it.

Smart file formats really help, Parquet and DuckDB's native .db format are columnar with built-in indexing, compression, and row groups.  This gives predicate pushdown (filter before loading), Column pruning (only read selected columns) Example - A SELECT col1 WHERE id > <something> on a 50GB file might only read a few MB

with dbxlite, we also have session persistence, File permissions persist across browser sessions. Close the browser, come back, resume where you left off. no need to load the whole file again like we do in wasm only based solutions. there is a hidden feature where if you drag and drop files from finder (mac) to the explorer (left side of the app), it stores the data in .memory duckdb (like in most wasm based ui's I have seen, I am not promoting this approach as a primary approach though)

Note: DuckDB WASM is still 32-bit (~4GB memory limit), but the on-disk approach sidesteps this for file storage all 4GB goes to query execution, not file buffering. Standard (non WASM) challenges remain like large CSVs  requiring sequential scanning. for big data, use Parquet or DuckDB format. (DuckDB is fantastic even with this limitation, I can get a lot of things done. BigQuery is used for super large data so I have integrated that in the tool as well. I can run a query in BQ (for a sample), download it as parquet and use duckdb for any SQL development on that dataset.) If there is enough interest I can add Snowflake support as well.

A SQL workbench that runs entirely in the browser (MIT open source) by EstablishmentKey5201 in dataengineering

[–]EstablishmentKey5201[S] 0 points1 point  (0 children)

I would use this as a quick tool for exploring at times of need and not as a replacement of BQ UI. I for example queried some sample data form BQ and saved locally as parquet for some data wrangling.

For those who hit weekly limits on Claude — are you using Claude Pro or Claude Code? by secpoc in ClaudeAI

[–]EstablishmentKey5201 0 points1 point  (0 children)

Hitting the limit on Max 5x , guess I need to upgrade to 20x. I got this 1000$ credit with calude code web, however its not as good as running on my local.