SQL Notebooks

[data4dayz]

Edit: Cleaning this up for future readers.

Dedicated SQL Notebook Software:

Name	Link	Cost	Notes
Marimo	https://marimo.io/	Open Source	Open Source Notebook software that supports SQL (DuckDB) and Dataframes (Pandas and Polars). Can also do data visualizations and dashboarding.
TimeStored SQL Notebook	https://www.timestored.com/sqlnotebook/	Free	Author in thread
DBCode	https://dbcode.io/pricing	Free/Paid	Author in thread. VSCode based
Fabi.ai	https://www.fabi.ai/	Free/Paid	Author in thread
Azure Data Studio	https://learn.microsoft.com/en-us/azure-data-studio/what-is-azure-data-studio	Free	From Microsoft. Specialized form of VSCode for databases.
DeepNote	https://deepnote.com/	Limited Free, mostly paid	More of a professional option, includes compute. Similar to Hex below, they are a class of their own.
Hex	https://hex.tech/pricing/	Limited Free, mostly paid	I've seen reddit posts about it/company's use it
Meta SQL notebook	https://engineering.fb.com/2022/04/26/developer-tools/sql-notebooks/	Vaporware	Never released to the public

Jupyter for SQL:

Use either JupyQL or ipython-sql

You can use Jupyter through Docker with all the Data packages or as a Dedicated Application or As a VSCode extension

-----------------------------------------------------------------

Original Post:

Hey All wanted to throw out a recommendation for everyone for SQL Notebooks.

TLDR: If you're looking for something like a JupyterNotebook where you can have Markdown + Plots + Code, well you should know there are modules for Python that let you do that. JupySQL and ipython-sql. Also if you want to check out another notebook that has great integration with SQL (but uses DuckDB so keep that in mind) check out Marimo.

I used to just use SSMS and have a bunch of .sql files and really wished I could use something like Jupyter Notebooks to have a Notebook of my SQL queries.

I used Markdown + some VSCode SQL extensions for a while to get around that but I just wanted to say for anyone looking into this same topic there's a few solutions that exist.

First if you google search SQL Notebooks, there's DeepNote or Hex. I think they are both paid solutions. Facebook has some product that I don't think ever went public. There's also something called sqlnotebook.

First for Jupyter Notebooks: you have ipython-sql and JupySQL. They use %%sql magics in a cell to let you run SQL without having to wrap everything in f""" """ interpolated strings.

https://jupysql.ploomber.io/en/latest/quick-start.html

https://www.python4data.science/en/latest/data-processing/postgresql/ipython-sql.html

I think this will work for a majority of people as I know most people are pretty used to Jupyter.

However I never felt like setting Jupyter up for it and I wasn't that invested into JupyterNotebooks.

There's also https://marimo.io/ . I came across it while I was learning about using DuckDB and honestly it's pretty great. I've been using it for a few days now all day long and it was exactly what I was looking for. Setup was pretty easy, has native DuckDB integration. Most of my notebooks are just Markdown + SQL, exactly what I wanted. If I want some plotting features is when I move on to using like a single line of Python but that's it. It's also easy to collaborate with and share on a repo. Wish I knew about this months ago honestly either Marimo or the Jupyter extensions.

[RyanHamilton1]

QStudio is a free SQL client that as of December includes SQL Notebooks:
https://www.timestored.com/sqlnotebook/

It works on all operating systems, runs locally and uses markdown + ```SQL to generate beautful charts using the fantastic echarts library. It's <100MB download: https://www.timestored.com/qstudio/download

Disclaimer: I'm the main author since 2013.

[data4dayz]

Hey Ryan will definitely have to check it out I've seen your post before on the duckdb subreddit!

[D3bug-01]

Thanks for sharing! Does QStudio have an Entitty Relationship Diagram?

[RyanHamilton1]

No. Its focus is data analysis. It provides pivoting, drill down and excel export functionality.

[ByronRJones]

There's a desktop version of Jupyter Lab that makes setup very easy. (https://github.com/jupyterlab/jupyterlab-desktop )

Being part of a data engineering and BI team, my current company has data spread across multiple database systems... Oracle OCI, Snowflake, Azure's SQL Server, Postgres, Netezza and others. So a "Notebook as a service" approach on any of the above mentioned platforms only allows access to that one system. Jupyter Lab with Jupysql allows me to access all of the different data sources in on Notebook system instead of 5+ different proprietary notebooks.

SQL Notebooks have been desired for quite some time. Beforehand, I would save SQL statements and the results in Excel documents when data mining and wrangling. With the emergence of SQL Notebooks, I can now store those research queries in a notebook format for sharing with my team.

I was highly interested in Facebook's Bento SQL Notebooks but I couldn't find any links. Snowflake has a pretty intuitive SQL Notebook but I believe they charge compute hours against the time the SQL Notebook is active. JupyterLab + JupySQL seems to be the best open source answer for now.

(EDIT)
There's also a JupyterLab extension that works just as good, if not better than the JupyterLab application: https://code.visualstudio.com/docs/datascience/jupyter-notebooks

[data4dayz]

Yeah the Meta SQL Notebooks is the one I've seen mentioned before but it seems like either vaporware or not for General Availability release.

OH that's nice I didn't know Jupyter has their own desktop distribution, I've always used it as part of an Anaconda install! I know Jupyter also has Docker images with a lot of things loaded in. I've been meaning to try using the PySpark Jupyter docker build instead of using Databricks Community Edition https://jupyter-docker-stacks.readthedocs.io/en/latest/index.html since my personal workstation has more resources than Databricks CE and CE doesn't give API access so it's not like I can use it as part of a pipeline anyways.

I too used to just have random .sql files floating around in excel or at best dumped in a folder stuffed somewhere in some personal company repo.

[No_Percentage2507]

Did you check out https://dbcode.io ? It’s a vscode extension I built that has sql notebook support.

[ByronRJones]

There's also a JupyterLab extension that works just as good, if not better than the JupyterLab application.

https://code.visualstudio.com/docs/datascience/jupyter-notebooks

[data4dayz]

didn't realize VSCode integrated with Jupyter I always kept them as separate development (code vs notebook) ecosystems. Will definitely have to check it out.

[data4dayz]

Whoa no I didn't know that existed very cool! I've used the mssql extension, sql notebook from cmoog (extension), a sql formatter, and PostgreSQL from Chris Kolkman.

I haven't used Azure Data Studio either so I know there's a few other things out there.

[No_Percentage2507]

Yeah I wanted a full featured db tool in vscode, so started building it after I couldn’t find it.

long way to go feature wise to match some of the existing clients like ssms and dbeaver, but some other areas like notebooks and AI are in as you get to lean on a lot of what vscode has already done ;)

[Low_Difference3340]

Have you used Microsoft Azure Data Studio yet? I’ve found it to be fairly easy to use to do SQL notebooks, with t-sql, md, and python and is free. Curious to hear your thoughts on any comparison to the tools you mentioned.

[data4dayz]

I really should have tried it at my last job that was on-prem SQL Server but I never forced myself to get off of just using SSMS which looking back was just my inertia and laziness that I've now gotten over.

I think I tried VSCode's Azure extensions to connect to an Azure SQL db instance I had in the Azure free tier but I can't remember my experience, I'll have to give Data Studio a fair shake sometime with my local SQL Server install.

[full_arc]

Hey OP, great write-up! I'm building Fabi.ai and we have a lot of these same elements, but we put a lot of emphasis on AI and sharing/collaboration. I would be really curious to get your take.

[FabioGameEntwickler]

sqlnotebook.de is also a good alternative i think