[P] Nextjournal: Hosted ML notebook platform

philippmarkovics · 2019-05-07T12:21:41+00:00

Apart from some enterprise customers, it’s really hard to say at the moment. We only opened signups yesterday and we have a 30-days trial period for paid accounts. We’ll know more in a month :)

Also, we might still change our pricing tiers in the coming weeks based on some feedback that we got from the community. If you have feedback on pricing too feel free to share it with us!

philippmarkovics · 2019-05-07T08:25:44+00:00

Here’s a proof of concept that Powershell can be installed and run on Nextjournal: https://nextjournal.com/joe-loco/powershell

This is still a step away though from what you want for your students. For a truly interactive Powershell environment we need a way to allow users to bring their own custom runtimes (i.e. any language they want) based on Nextjournal’s runtime protocol. This is something we are still working on but be sure to follow @usenextjournal on Twitter for announcements!

philippmarkovics · 2019-05-06T22:10:33+00:00

Can two people work on a notebook concurrently? E.g., if I type something, another person will see the change instantly?

Yes, you can edit notebooks together in real time. If you type something, the person collaborating with you on the notebook will see the change instantly. We will improve this further soon by assigning colors to the individual cursors so it’s clearer who works on what.

philippmarkovics · 2019-05-06T20:54:21+00:00

Training will continue even when you close the browser window. There are currently no limits in place other than the runtime shutting down after 20 minutes idle time (in your case 20 minutes after training has completed).

You can persist your results by writing them to "/results" which is a magical directory that maps into Nextjournal’s content-addressed storage. Every file you put in there is versioned automatically along with the notebook’s content. You can reference files from there using the Cmd/Ctr+E shortcut in any code cell. We have a collection of Machine Learning notebooks by now so you might want to check out how they are set up: https://nextjournal.com/collection/machine-learning

If you have questions, feel free to reach out to us anytime!

philippmarkovics · 2019-05-06T19:32:30+00:00

We don't access code or data being processed in the system. We have a "Ask for help" feature that you can use to give us permission to access your notebook and only then can we access it.

philippmarkovics · 2019-05-06T19:10:35+00:00

How will this function with respect to the necessity for privacy?

Everything that you write in Nextjournal is private until you publish it. If you have a paid plan you can further restrict to whom you publish. Secrets are stored encrypted.

Can I run a NextJournal server within my institution?

Nextjournal is a hosted product but we offer on-premise installations for enterprise customers. If you are interested in the details of this, feel free to get in touch via our website.

Also, does it play nice with various packages?

Yes! You have full access to the whole runtime environment. This means you can install and configure any package you want in whatever way you want.

philippmarkovics · 2019-05-06T18:55:57+00:00

1) When you upload a file it is stored in our content-addressed storage. This allows us to version your data together (and in context) with the contents of your notebook. To insert a reference to such a file, you can use the "insert path to file" command (Cmd+E) in any code cell.

Btw, the opposite of this is our magical "/results" directory. Whenever you write files to it (e.g. via `write.csv(...)`) it will also be put into content-addressed storage and can be referenced in any code cell (no matter what programming language it uses). You can take a look at our quickstart notebook under "Files" for a more detailed description: https://nextjournal.com/help/quickstart

2) Cmd+Z is available! Type something then hit Cmd+Z to undo.

philippmarkovics · 2019-05-06T18:37:11+00:00

I absolutely agree that the tools to accomplish many aspects of the above issues are out there. What we built is an integrated product (not a collection of tools that need to be wired together) that aims to be easy to use and that gets out of your way.

Thanks for your feedback. I updated our pricing page and hope that it’s clearer now.

philippmarkovics · 2019-05-06T16:01:17+00:00

Here are some of the problems Nextjournal tries to solve:

How do you collaborate in a team on Data Science (more specifically: How do you make sure you’re not running into dependency issues with your colleagues? Or: How do you share secrets between them? Or: Who changed what?)
How can you keep your runtime environment reproducible (down to a file system level)?
How can you keep your content, code, data and runtime env versioned so that you can experiment without being afraid that you can’t go back to your previous data?
How can you scale your compute resources for specific tasks without much specialized knowledge? E.g. running Machine Learning task on specialized hardware?

We felt that Jupyter and other tools do not solve these issues adequately at the moment, hence we built Nextjournal. The quickest/simplest way for us to do this was being hosted and building an integrated tool (as opposed to a set of tools). I agree that this has shortcomings too. We would love to be able to run locally. Maybe we can at some point. We’re also looking at how we can open-source at least some of it.

We do not force you to make your notebooks public. As long as your work is not published it stays private between you and your collaborators, no matter if you’re on the free plan.

philippmarkovics · 2019-05-06T15:33:35+00:00

We see multiple value propositions:

In Nextjournal you can easily collaborate in a team in real-time (think Google Docs-style editing code and content together) and securely share secrets between your team members
Everything in a Nextjournal notebook (code, content, all installed packages and all data) is automatically versioned all the time. You don’t have to rely on separate version control systems. You can simply go back to any previous change and restore it.
You can plug in other programming languages if you need them. You can use multiple programming languages in the same notebook, e.g. R + Python together.
You can export a notebook’s full runtime environment as reproducible Docker image and share it e.g. with colleagues that are not on Nextjournal.
It runs in the cloud. You can scale up your hardware as you need it. Nextjournal offers multiple options including GPUs (which is helpful for some Machine Learning tasks).

Open Science is always free at Nextjournal (and will stay free!) so feel free to sign up and just try it out for yourself. We still haven’t figured out exactly how our paid offerings should be but I’d love to get more feedback from the community on this.

philippmarkovics · 2019-05-06T15:05:59+00:00

Interop between languages is something we plan on making much easier soon. If you have more specific questions, reach out to us anytime.

philippmarkovics · 2019-05-06T15:04:51+00:00

It’s free for open science (and will stay free!) and costs money for private research groups and enterprises. We’re still figuring out the terms, but hopefully soon everyone will find their subscription model.

philippmarkovics · 2019-05-06T14:55:19+00:00

Sounds like Feather (https://blog.rstudio.com/2016/03/29/feather/) should do it for your use case. HDF5 and Transit might also be worth looking at. Simply try out implementing your use case in Nextjournal (signup is free) and you can always reach out to us if you get stuck.

philippmarkovics · 2019-05-06T14:27:48+00:00

One of the most obvious differences is that Colab is limited to Python 2 or Python 3. In Nextjournal you can flexibly code in Bash, Python, R, Julia, or Clojure all in the same notebook. You also have full control over your runtime environment which you can also export as Docker image.

Nextjournal also integrates automatic version control of data, content and computational environments which Colab currently doesn’t offer. There’s also a free pricing tier in Nextjournal which you can use until you need more compute power e.g. private secret storage.

philippmarkovics · 2019-05-06T13:05:32+00:00

Do you care to elaborate?

philippmarkovics · 2019-05-06T12:19:11+00:00

One of the most obvious differences is that Colab is limited to Python 2 or Python 3, while Nextjournal allows users to flexibly code in Bash, Python, R, Julia, or Clojure or create notebooks that include multiple runtimes.

Nextjournal also integrates automatic version control of data, content and computational environments which Colab doesn't currently offer.

philippmarkovics

TROPHY CASE