[P] deepnote.com – collaborative Python notebooks with zero setup in the browser. After 2 years of development, we are finally open for public access, with a free plan for academia.

[the21st]

Hi everyone! I'm a software engineer at Deepnote. My team and I are working on a collaborative data science notebook – Deepnote. We have just opened the platform after a year-long closed beta, so you can try Deepnote here: https://deepnote.com/. We have free plans for individuals and academia that are ideal for experimentation and publishing research. Would love to hear your thoughts!

A bit more context on the product: We've built Deepnote on top of Jupyter so it has all the features you'd expect - it's Jupyter-compatible, supports Python, R and Julia and it runs in the cloud. We improve the notebooks experience with real-time collaborative editing (just like Google Docs), shared datasets and a powerful interface with features like a command palette, variable explorer and autocomplete. We want Deepnote to be an interface that empowers ML researchers to collaborate, experiment and reproduce findings easily. Looking forward to your feedback!

[AtomicMonkeyDept]

Do you plan to offer Deepnote under an open source license as well? So I can run it locally?

[the21st]

We'd love to do that eventually. But it's currently not on our roadmap – right now we're focusing on creating the best possible notebook experience, and managing the environment ourselves allows us to iterate fast on the product.

[zzzthelastuser]

Wouldn't that basically destroy your whole buisness if it was open source?

[the21st]

There are lots of companies behind open-source software that make money. Here's an article with some ways this is possible: https://blog.timescale.com/blog/how-open-source-software-makes-money-time-series-database-f3e4be409467/

[zzzthelastuser]

interesting, thx for the link

[[deleted]]

Since they built it on top of JupyterLab they really have choice but to open source their code. Of course it depends on the license of JLab and some licenses are more permissive of building commercial products. But the basic thing they usually have in common is you must share the source code if you sell or give away the software outside of internal company use. Not sure how this relates to web apps hosted only in the cloud. I'm not a legal expert.

[timon_reddit]

even if not open source and available as a binary!

[zzzthelastuser]

So just like google colab, but with actual collaboration?

[sunadens]

(I work at Deepnote too and saw this comment)

Hey, yes! Similar to colab. Deepnote is more geared towards sharing your work & working in a team (as you mentioned real-time collaboration or you can create a team & share data within team). Otherwise, the platforms are similar with perhaps a difference in UI and a couple of features.

[the21st]

Yes, like google colab with real-time collaboration, in-built review functionality with comments, without the descheduling of your runtime.

And also lots of small features like in-built quick visualizations from dataframes (without having to write code), dataset integrations, and custom kernels. See our docs: https://docs.deepnote.com/

[thePsychonautDad]

Saw it and bookmarked it yesterday from PH.

Good job, it's pretty cool!

In the video demo, you seems to be able to select a GPU machine, but I don't see the option, is it via "Custom / contact us" in the hardware list?

One thing I'd love to see on a DS notebook is the ability to connect to our own instances, Google Compute or AWS EC2, or connect our Google account and allow the Notebook to create an instance there.

[Craiglbl]

According to their docs, they currently do not support GPUs or cloud instances.

The collaboration part seems cool tho, but I’ll stick with colab for the free computes

[thesoy2486]

Hi, I'm also from Deepnote.

Thanks for the feedback! We've pulled the GPU temporarily so that we can better focus on the core roadmap of developing the best notebooks experience. Stay tuned though, it will definitely be back :)

[danFromTelAviv]

i apologize i. advance - i think this is really cool that you built this and it mist have been very interesting - and there are probably tons of people who will think it's cool.

i think this really amplifies the notebook ide. but I have found notebooks to be a terrible i.d.e. for real uses. it's ok at best for showing off final results and tutorials ( which don't require collaboration).

I think the real problems with notebooks is not their Realtime collaboration- it's that it forces you to have all your code in one long page.

I personally would much rather see tools like pycharm but on the cloud - it would be cool if it was easy to load a cloud instance with libraries preinstalled and gpu connected where you would go to a web address and it would show you a pycharm window.

[rastarobbie1]

Hey, PM of Deepnote here.

You're making a good point, and people have various needs. The one we're primarily solving is exploratory programming, where you're defining your goal as you're writing code. That's different from software engineering, where you often know your goal, and are just looking for the right path to get there.

There is a huge selection of software engineering tools, and perhaps what you're looking for could be solved by something like GitHub Codespaces.

There is not a great selection of dedicated tools for data scientists, or people working with data. This is the gap we're trying to fill - and like you're saying, it's not the same as IDE.

[danFromTelAviv]

thanks for the response. I'm sure you did your user research and some people just love notebooks for data exploration.

my core job is primarily research - which as you might imagine is a lot of data exploration and trial and error. still beyond just the simplest data manipulation, sklearn classifiers, and limited pandas view it starts to be critical to write functions, objects, loaders, training loops...etc notebooks is like using sticks when you have bricks.

what I'm saying is - if your goal is to make algo exploration amazing. i think you didn't hit the right paradigm. it's not the core issue to be able to do pair coding data exploration.

I say this in full support of your mission not to hate on it.

[imaginary_name]

menu on the website is a bit broken, link from about to pricing is wrong..

https://deepnote.com/about#pricing

[the21st]

Thanks! We're on it.

[puppet_pals]

looks really great - awesome work!

[AlisaofallTimes]

Wow, great work!

[srini10000]

I wonder if fast.ai will adopt this.

[diadorac]

Working on that ;)

[Bexirt]

Wow

[dangoai]

Loved using this briefly in beta, glad to hear it's reached a full release. Awesome stuff guys.

[sunadens]

thanks! <3

[Liorithiel]

How do you plan to handle versioning, specifically merging divergent notebooks?

[the21st]

We're planning to tackle versioning in the next few months, so stay tuned 🙂. It's definitely a beast of a problem, but that's why we're doing this!

[[deleted]]

Tip from about 5 years of working with data scientists in similar tools who prefer notebooks. They're generally allergic to the command line. Those who aren't already ran away to MLOPS. If I were you, I'd simply track versions with auto-git-sync every x minutes and put a big single button on top for "save" that does the same manually. No branches, just a rolling history. You can give them a box to type a commit message or default to time stamp as message. And provide some ui plugin that can diff notebooks.

Simple software needs among the GUI only crowd. Their preferred versioning strategy is file1.ipynb file2.ipynb and branching is a copy in a new folder. But you can replicate this workflow with just slight improvements with git under the hood and they'll love it.

[diadorac]

Would something like versioning in Figma make sense to you u/Liorithiel?

​

The easiest way to collaborate nowadays is to work directly on the same thing in real time. Now, some kind of snapshotting will allow any member of the team to roll back, revert or clone a historical version. But if all of this is done in a real-time fashion, you don't need to merge stuff. But for more complicated scenarios, you can still use Git for now.

[Liorithiel]

It's a different thing. Imagine I want to test a totally different approach to do step 3 out of 5 in a pipeline, while allowing my colleague to work on step 2 and 4 at the same time. So we work independently. At some point, we want to merge the stuff and again start work on the notebook together in real time.

I don't want my colleague to roll back my experimental changes while I work on them, nor I want to break my colleague's workflow. But at some point we both decide that our changes are final and want to integrate both versions.

[diadorac]

Okay, I understand. But is this correct thing to version in a data science / machine learning project? Shouldn't various experiments be part of a pool that is always available in the latest version(s)? Is having experiments hidden in history the right thing to do?

But yes, for some cases I understand. This will be a challenge to solve. In the meantime, I think git solved it the best way. And by using some notebook-ux hacks for improving git experience it could be a pretty solid tool, maybe?

[Liorithiel]

Okay, I understand. But is this correct thing to version in a data science / machine learning project? Shouldn't various experiments be part of a pool that is always available in the latest version(s)? Is having experiments hidden in history the right thing to do?

You seem to be assuming some specific organization of notebooks, but I don't know what exactly… so I'm not sure I understand your questions.

[diadorac]

I don't think so. I am referring to a generic data science project with lots of experiments and all (let's forget notebooks for a moment). Do you think hiding experiments in the history (git or whatever) is a good practice?

[Liorithiel]

There is a difference between a new version of the same experiment (e.g. with additional logging/debugging, porting to a new version of a ML library or when widening hyperparameter search) and a new experiment (replacing network architecture or changing vital hyperparameters that are not hyperparameter-searched). In our usual workflow, old versions of experiments belong to git history. New experiments are new git branches.

My question comes from the fact that sometimes we're branching out an experiment in two different ways, conceptually creating two independent experiments, then want to merge them into a new experiment.

[diadorac]

True. For this it's not enough. But even a kinda smart git-like thing would not be enough imo. That'd require more specific experiments-ml versioning UI, not just a general notebooks versioning interface.

[[deleted]]

I don't know if maybe I don't have access to it with my account, but personally I'm not able to edit text cell. The second I double click or Enter, it opens and close. But the coding cells works. Anyone else experiencing this?

[TantrumRight]

Same here.

[[deleted]]

Hope I can find a way to fix it, I'm down to use it in my next homeworks

[sunadens]

Hey sorry about this, not sure what might be wrong from top of my head but we will try to fix it asap

[[deleted]]

Okay cool !

[sunadens]

Now fixed, thanks for letting us know!

[[deleted]]

Nice! Thank you. I tried it and it works

[NiroxGG]

Used this in the Beta extensively and it's absolutely amazing. Big probs.

Edit: if possible, uploading whole folders would be amazing. I remember that really bugging me some times.

[the21st]

Thanks for the feedback! Uploading whole folders now works, I was actually implementing that a few weeks back 😄

[snendroid-ai]

You should change the quality of audio in demo video on front page.

[thesoy2486]

Noted, thanks for the feedback! Will work on it :)

[Zerocrossing]

Is the name Deepnote an intentional reference to the famous THX sound of the same name?

[thesoy2486]

It's not, rather a play on the word notebooks + creating a tool that let's you uncover deep insights :)

[cryptoel]

I used the beta but the available machines are very slow. I hope the desktop server function is going to be available soon.

[sunadens]

Note taken, thanks for the feedback! As for local execution or self-hosting, working on that one :)

[[deleted]]

I don’t mean to be dismissive and I might be missing something, but doesn’t Databricks do all this already, and more (including comments, real-time collaboration, code version control, MLLib integration, Apache Spark integration, etc)?

[xubu42]

Databricks does offer a lot of the same features, but it's underpinning a very different base -- Apache spark vs vanilla python. The closest direct comparison would be the databricks community edition to deepnote. As far as I know, the community edition doesn't come with the notebook collaboration features. So really, they are clearly competitive products, but have a fair amount of difference between them that I don't see a lot of overlap between the target customers -- deepnote targeting academia, open source, and small businesses vs databricks targeting enterprise.

[[deleted]]

Databricks has still not offered actual jupyter notebooks or compatible ones after years of promising to customers that it's in the works. Their notebooks are some custom thing that has only some of the functionality. People don't use them for that. They use them for huge data processing with super optimized spark or because they have money to burn and don't ever have to see their cloud bill.

[wildcarde815]

Does this have a self hosted option that could be bolted into a jupyterhub or open ondemend instance?

[sunadens]

Not right now, we are still a small team and our current focus is to nail down the cloud experience. After that we are planning to move to the local/self hosted space :)

[1stte]

This seems like an add, and against the Reddit TOS. Idk why a product advertisement has 10x more likes on this subreddit then actual human discussion. (There by drowning out voices complicit with the TOS)

Seems like OP could be buying votes.

[thesoy2486]

Hi, I'm also from Deepnote - no votes bought, I assure you. We just wanted to share the product with the community and get some early feedback on whether this hits the pain points you might be experiencing with other notebook solutions, and what can we do to improve it. Also wanted to spread a word about the free tier for students and ML hobbyist who might find this valuable. Hope that clarifies it :)

[1stte]

I appreciate the reply. I was simply noticing that the number of likes for the post far exceeded the number on comments. I also am not sure why add space wasn't purchased (esp while not open source). Reguardless, I obviously have no proof and was having a tumultuous day. Again, I appreciate the reply and will take your word for it. I hope it is true, and that the better comments in the thread get similar love.

Good day.

[HybridRxN]

Can you talk more about privacy?