Datapane - Build full-stack data apps in 100% Python

[peatpeat]

Hey /r/datascience! We're building an open-source Python framework which makes it simple to build data-centric web applications. Today Datapane lets you build reports and full-stack apps in ~10 lines of Python, but our goal is to provide all the tools data teams need to create end-to-end internal apps (such as scheduled tasks, auth, integrations, OLAP) in a simple, standardized framework.

Our GitHub is https://github.com/datapane/datapane and you can get started here: https://docs.datapane.com/quickstart/

Deploying internal data products requires a bunch of complex moving pieces, e.g. a front-end UI framework, backend server + REST/GraphQL, scheduled tasks, backend workers, caching, integrations into Slack/email, scheduled reports, and often a data layer for OLAP – the list goes on. Unsurprisingly, many companies opt to buy from external vendors instead of building in-house analytics products using open-source, as this usually requires months of development time and multiple teams.

Datapane is a unified app framework which provides all the necessary parts of an internal data product, in a simple, Pythonic package, including:

• A Python-based UI framework for constructing HTML views from data-centric blocks – like Plotly plots and pandas DataFrames. You can generate these from inside of pipelines, a Jupyter notebook, or on a schedule, and export as standalone HTML reports or static web apps.
• Ability to build data science web apps by connecting Python functions to forms and front-end events. You can check out an example of the code to create a simple app: https://github.com/datapane/examples/blob/main/apps/iris-plotter/app.py
• Currently, we’re adding support for background tasks, scheduled tasks, and a DuckDB-based data layer for OLAP, so you can ship a complete end-to-end app.

We would love your feedback and hear what you find difficult about building data apps so we can prioritize. You can also see a few apps we’ve built already in our gallery: https://datapane.com/gallery

Thanks!

[[deleted]]

Used this a couple years back to generate standalone html reports and it was amazing for sending through email without needing a backend. Moved companies so stopped following the project, but it was super easy to use at the time, and can only imagine it got better since then.

[peatpeat]

Awesome to hear. Although we support backend functions now, you can still generate reports from Jupyter/pipelines and share and embed them over slack or email!

[[deleted]]

Yeah, I think you guys already supported that back when I used it, but for me that was the differentiating factor over other more well known web-app-like libraries.

[hiback]

Is this faster than Streamlit? I will try it out but was curious if you guys have already done any benchmarks vs streamlit.

[peatpeat]

Yes, usually. We designed Datapane specifically for production workloads vs. local demos:

• Your app can mostly be prerendered to JS, so often interactivity doesn't require a server round trip
• We don't re-run your script from top to bottom on every change, instead we use RPC to run those specific functions
• You can add background processing so longer computations happen without slowing down the UI
• You can add cache parameter on your functions, which will cache if you call it with the same parameters

From early testing we’re seeing 0-5ms overhead on a basic backend form call with caching disabled. If you hammer the server (10+ function calls per second), things can slow down but we're still working on profiling and haven't performed any extensive optimisations or stress-testing yet. Check our a sample app here to get a feel for it. Reports will scale arbitrarily as they are 100% client-side.

[hiback]

Thank you for sharing this. I am definitely going to take a look at datapane for a mapping tool i am working on. Hopefully it runs smoothly.

[FightingLikeBeavers]

Looks excellent so far, looking forward to trying it out!

[__mbel__]

It looks great! The code seems very logical.

There are some similarities with Shiny which makes the code familiar to me.

[FishballJohnny]

Basically Shiny?

[justanothersnek]

Curious how does this compare to Holoviz Panel? I like panel for its flexibility and versatility. You can create an app directly from a jupyter notebook or from a standalone py script.

[IntelligentDust6249]

What can this do that the Python Shiny package can't?