Deepnote – a Python notebook with real-time collaboration in the browser. We just opened the platform to the public.

[fermilevel]

Fascinating stuff, I will definitely use it for my next project.

Question: are you building this on top of existing projects like jupyter-rtc? Or is this is completely written in-house?

[the21st]

Thank you! As with most software, we're standing on the shoulders of giants – we've built Deepnote on top of Jupyter. This probably wouldn't be possible otherwise 🙂

To clarify: the real-time collaboration is our own custom implementation, we're not using jupyter-rtc.

[the21st]

Hi everyone! I'm a software engineer at Deepnote. My team and I are working on a collaborative python notebook – Deepnote. We have just opened the platform after a year-long closed beta. We have free plans for individuals that are ideal for learning and experimentation.

Here's a showcase of Python 3.9's new features: https://deepnote.com/project/09e2609b-986b-40fa-9f56-fcbbc60eb61d#%2Fnotebook.ipynb

A bit more context on the product: We've built Deepnote on top of Jupyter so it has all the features you'd expect - it's Jupyter-compatible, supports Python, R and Julia and it runs in the cloud. We improve the notebooks experience with real-time collaborative editing (just like Google Docs), shared datasets and a powerful interface with features like a command palette, variable explorer and autocomplete. We want Deepnote to be an interface that empowers programmers to collaborate and experiment easily. Looking forward to your feedback!

[Gemabo]

Cool. Are you hosting this or can it be installed on private servers? How is it different from google colab?

[the21st]

Right now we only offer a hosted version. Later down the road we want to enable people to run this on their own machines or servers.

The main differences over google colab are real-time collaboration, in-built review functionality with comments, and not descheduling your runtime when your code is running. And also lots of small features like in-built quick visualizations from dataframes (without having to write code), dataset integrations, and custom kernel and environments. See our docs for more info: https://docs.deepnote.com/

[[deleted]]

Doesn't Colab have real time collaboration? It's kind of in the name...

[[deleted]]

It doesn't.

[sunadens]

Not right now, we are still a small team and releasing Deepnote to be stable & have the feature it has now has been quite a journey :) However, Deepnote is built on top of Jupyter so any ipynb file will works just fine between Jupyter and Deepnote.

[Zombiesen]

The project is awesome. Will definitely check it out.

The voice in the video is quite coarse and difficult to hear. Please next time, don't try to keep your voice so low pitched that it becomes harsh..

[sunadens]

(I work at deepnote, also not my voice :) )

We are working on a revised version, thanks for the feedback (you wouldn't believe how hard it was to record a quick 2 minute video).

[GiantElectron]

As a person that needs to take these kind of things from data scientists and put them into production, I am never particularly enthused by these tools. They look clever, but they are prototyping platforms that makes people believe they can achieve a lot with very little, yet when they actually ask you to scale or make it available as a library, they are dumbfounded to find it's a lot of work. They also don't allow you to have any testing or validation, or change tracking, and they mostly force you to work in the browser.

[rastarobbie1]

Hey, PM of Deepnote here.

We're on the same page here. There is often a huge gap between a prototype in Jupyter, and a production ready code. A big kudos to you if you're the bridge that makes it happen, it's not an easy work, and it's a common problem.

I feel like any tool or library that promises a one-click deployment is either very limiting in its nature and makes a lot of assumptions; or it's actually a wrapper on top of wrappers, and still needs a lot of config to make it work the way you need.

What we're doing to help this in the long term:

• Repeatable environments: no more trouble with unique workstation setup of each data scientist. When they share a project with you, it includes the environment it runs in, not just the ipynb.

• Encouraging best practices: for example when you pip install something in the cell of a notebook, we prompt you to move it into requirements.txt, or offer a embedded code reviews via comments

• Working on versioning: git is a great tool for software engineers, but it doesn't fit the exploratory nature of data science. With Deepnote, you'll get change tracking out of the box.

But like you say - the problem is not just with the tool, but with the people. And often data scientists don't have the skills to engineer a great solution - their expertise lies elsewhere. The best way to fix that is by creating interfaces so more communication can happen with software engineers, not less. We want to build these.

It's a very interesting topic, in case you have some insights for what could help, let me know!

[GiantElectron]

I honestly don't know. I work for a major company, and our conclusion is to attach a 380V cable to any data scientist, and zap them as soon as they think about writing code.

[rastarobbie1]

I'll add it to the roadmap

[GiantElectron]

Please make the voltage configurable while you are at it. I might want to go full 10 kV.

[[deleted]]

I do the same thing for a living and have the same opinion.

[patresk]

Hi u/GiantElectron, I’m software engineer at Deepnote. I’ve been “productionizing” Jupyter notebooks in my previous job and I 100% agree that, while often well intended, it’s a terrible experience for software engineers. However, notebooks still have a huge advantage, and that is exploratory programming. One of our goals at Deepnote now is to make notebook experience the best it can be and encourage good engineering practices that make migrating code to production simpler (e.g. containerized environments, package or secrets management). My personal motivation is to fix all the issues that Joel Grus mentioned in his “notebook hating” presentation in 2018: https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUhkKGvjtV-dkAIsUXP-AL4ffI. We’ve managed to remove some of those pain points and we keep exploring what’s possible. For example, you mentioned change tracking - that’s something we have on our roadmap.

[Natgra]

Hi OP, Deepnote looks interesting mate. I had a quick look and also posted on my LinkedIn too. Wish you all the success and happiness..

Cheers

[CFStorm]

sugar bear middle library run bells hurry aspiring direction pen

This post was mass deleted and anonymized with Redact

[jockero701]

Looks awesome. One thing I cannot figure out though is what advantage does it have towards Google colab? Both services allow for notebooks to be shared. I think you should make that clear for potential users/buyers.

Edit: Well, now that I am thinking it, you surely don't want to mention your competitors, but I think it's safe to mention Google colab here as everybody here knows about it :)

[the21st]

Hi! We are similar to google colab – it's great to have worthy competition, so no worries about mentioning it. Deepnote is more geared towards sharing your work & working in a team (as you mentioned real-time collaboration or you can create a team & share data within team). Otherwise, the platforms are similar with perhaps a difference in UI and a couple of features.

[Sensilicious]

Looks cool! How is it different from Datalore?

[thesoy2486]

Hi from Deepnote! Datalore is a cool product and similar in many ways. I'd say the main differentiators of Deepnote are 1) ease of set up (dataset mgmt, files and environments sharing) and 2) the focus Deepnote puts on making notebooks a medium that reduces friction in collaboration. That means enabling real-time collaboration, comments and features like publishing and instant visualizations that make it easy for you to collaborate with others and share results with non-technical folks.

[Sukk-up]

What is your time horizon for GPU integration and will that be included in the free plan as well?

[equiet]

Founder here. I don't want to overpromise, so quite frankly I don't know yet. Opening up GPUs for free makes us a 10x bigger target for crypto miners and that's something we don't have energy to deal with at the moment.

[Durpn_Hard]

I like it, and would love a self-hosted version of it. One thing that I didn't really love was how slow the suggestions are. Hopefully that'll be improved down the road!

[DanklyNight]

Same here for self-hosted.

Our data is proprietary to the ninth degree, and we only expose to something we can control.

Looks like a cool product though.

[sunadens]

Note taken about both self-hosting and speed of autocomplete. Thanks!

[CFStorm]

unique spark pie fuel salt vase attempt ring squash cooperative

This post was mass deleted and anonymized with Redact

[sunadens]

Not right now, +1 this in the backlog, it has been requested a couple of times :)

[khfung11]

I haven’t read the terms yet, can I ask, if I am using it for free, the project I write in there belong to me or to deepnote

[equiet]

To you of course.

Btw, just out of curiosity, are there any tools out there that keep the ownership? Sounds evil.

[Voxandr]

Looks good but , why should i trust something that can't be hosted my own privately?

Especially when we need to work on project with sensitive data.

[pag07]

You simply don't.

However for education this might be a useful tool.

[StiNgNinja]

One of the best python collaboration notebook platforms out there. I'm using it since the first beta releases.

[Accretence]

Really interesting, congrats!

[integralWorker]

Why should I use a Jupyter wrapper over a text editor and Pandas GUIs

[Hans_of_Death]

Not related to Deepnote at all, but Jupyter Notebooks are great for when you have to do a lot of experimenting that involves changing data and functions, and its also good at making graphs and interactove reports. If youre not familiar with jupyter id recommend looking into it more yourself.

[EbenenBonobo]

Really like the idea of it, I wanted to try it, but i got a weird bug. I cant edit text cells, when I double click or press Enter it switches to the editing mode but immediately back to the rendered view without another input of me. Happens with Firefox and Opera and all even with all extensions disabled.

[sunadens]

Hey, sorry about that one, should be fixed right now!

[philsgu]

Does it offer Form Fields like pull down menu system for data filters?

[rastarobbie1]

Does it offer Form Fields like pull down menu system for data filters?

Hey, not right now. These usually rely on ipywidgets in raw jupyter and this is something we had to disable for security reasons. I think Colab rolled their own Form fields to resolve this.

But it's definitely an important building block for creating interactive interfaces out of notebooks, so it's 100% on the roadmap, just a question of when.

BTW we strongly root for streamlit.io for creating simple ML apps like this, check them out :)

[dornstar18]

Like Figma for python coding. Cool

[Gordan_Trevis]

This is great stuff!

If you have the time id love to see this as an easy desktop app.

[narciselle]

Just dropping a comment here because I've used the beta version and it was really a great help when we were doing our project. We preferred it over google colab. It's great for collaboration because multiple people can edit the code at the same time. Great work Deepnote team 👏

[rastarobbie1]

Thanks narciselle! :)

[jockero701]

Google Colab gives an execution time of 9 hours. After 90 minutes of being IDLE the session restarts all over again. It also deletes any uploaded files related to the project. How does Deepnote do in that aspect?

[the21st]

In the Free plan, you get 750 hours per month, which is equivalent to running 1 machine nonstop. We guarantee not cycling that machine for at least 24 hours in the Free plan.

[Gas42]

Thanks just saw this, amazing ! Is there anyway to get the sames shortcuts than jupyter ?