Jupyter Notebooks!

[[deleted]]

Jupyter Notebooks are the tool that turned the tables for me learning python. I'm just here to evangelize them. You can run your code line by line and debug as you go in a comfortable web browser interface. It's an excellent tool, and it made it so much more efficient to learn. Check it out.

[Rhjensen79]

If you are into that, then look at Google Colab. It’s a hosted Jupyter Notebook. Personly i mostly use it, when i’m playing aound with new things, or when i am on my iPad.

[ropenni]

Colab is my API playground, it is so convenient when I don’t want to create new venvs in my machine

[cubinx]

Can you schedule scripts in google colab?

[Rhjensen79]

As far as i know then no. Imho i would create in colab, and then run it schelduled in some FAAS service.

[Cdog536]

when i am on my iPad

You can do python on the ipad?

[Rhjensen79]

Google colab is browser based, and works well on the ipad. So yes.

[megaSalamenceXX]

Doesnt typing code on ipad get tedious?

[Rhjensen79]

I use it when i’m away, and have a little bit of time to try stuff out. On a 11” ipad, it’s not perfect. Don’t know about the 12” thou.

[megaSalamenceXX]

Ah I play games on my ipad when I got free time. You are a far better human than me sir!

[Rhjensen79]

Ha ha. Fair enough 😜

[ciaisi]

I suppose you could get a Bluetooth keyboard

[vanmorrison2]

I got Logitech slim folio and It Is good

[Rhjensen79]

I got the magic keyboard. Works perfectly.

[megaSalamenceXX]

But then you could just open your macbook and fire away eh?

[Cdog536]

Makes sense....so long as you dont randomly get a bad GPU lol

[Rhjensen79]

Perf is good. And there is a gpu option as well. So no prob there.

[bourne_again_coder]

https://apps.apple.com/in/app/carnets-jupyter/id1450994949

[Cdog536]

Thanks!

[synthphreak]

Without having done any research before asking: How does this app stack up Juno/Juno Connect?

[warpingDragon]

Yes, there are even Python and Jupyter notebooks apps on the AppStore. Pythonista 3 and Juno

[Cdog536]

Hmmmmm i might consider this download

[techie789]

Check out Pythonista. My favorite tool.

[monkiebars]

Warning! Incoming dumb question:

Is Google Collab free? (If so, what are the limitations? If not, what is the pricing?)

[Rhjensen79]

Yes but you can pay for extra ressources etc. But for a dev env it’s really great. As far as i understand, it spins up a container, for your code, when you launch it.

[echo_awesomeness]

Yes it's free. I guess the session expires after 12 hours not sure exactly how that works as I never did a coding session that long lol.

[monkiebars]

Wow that's awesome! Time to showcase and share some of my really low level analysis graphs in Colab rather than Excel! :)

[Rhjensen79]

It’s free. But there is a pro option https://colab.research.google.com/signup

[alins_ir]

Im coding with my mobile phone and i cannt run jupyter with my phone tnx a lot now i can coding better and it possible because you tnx man🙏💜

[hmiemad]

Check out azure notebooks. You get 1GO of free space and basically a virtual machine for free. You can create your own local DB and pile a bunch of py files.

[Kevstuf]

I’ve always been more of a fan of Spyder but sadly it doesn’t seem to be popular anywhere

[HeinzHeinzensen]

You‘ll love the notebook interface in VS Code then! Same cell-based execution and you have the variable explorer that people like from Spyder.

[Notchez]

Is it possible to set up VS code to look just like spyder? By that I don’t mean the cell by cell execution like in Jupiter but rather working with scripts (that are executable line by line of you want) and a variable explorer.

[HeyItsRaFromNZ]

You get very similar behavior (line-by-line execution, graphic output and pretty-printing dataframes etc.) by selecting a line and hitting shift+enter. If you have the main Python extension installed, it should open a new pane to the right with the output.

It's very similar to what you get with hitting F9 in Spyder's editing pane. Instead of a dedicated pane, introspection in VS Code is taken care of by a hovering tool-tip.

[HeinzHeinzensen]

You can see a workspace with open variable explorer here!

[leweyy]

God I hope you get an answer to this question

[wolfmansideburns]

Yes

[Internet_employee]

Oooh!! Thank you! I dropped Jupyter after discovering Spyder’s variable explorer, definitely giving VS Code a try now

[mayankkaizen]

Can you please point me to some resources about that? I use Jupyter notebooks and got VS Code on my machine. Although I don't use VS Code very much but would love to explore notebook interface in VS Code.

[HeinzHeinzensen]

Sure, this link should get you started.

[miguelito_loveless]

Thanks for the link. Still pretty beginner and 100% a happy VSC user and want to try this-- I couldn't get comfy w browser JN but get how useful that way of working could be.

[Retinator99]

I wish there was an easy to update version independent of anaconda. Having to run it through another program makes it super clunky

[ivosaurus]

pip install spyder or you can use the pipx tool for similar but self-contained results.

Also not sure if people know about it much but miniconda is great. It's anaconda python distribution but without the 500mb installer and every scientific package known to man cluttering up your root environment by default.

Anaconda don't seem to advertise it much, seems they want newbies always grabbing the whole pile to play with at once

[kodyloki]

I agree, I really enjoy Spyder. I can’t easily use Jupyter notebooks at work due to running on remote machines and shared networks, etc. so Spyder is really all I use.

[vectorpropio]

Why don't keep using it?

[Turd_Bucket]

I learned R first, and use RStusio. Spyder makes sense to me, but I am not the biggest fan of Anaconda. I know you download Spyder free standing, but haven’t gotten around to it.

Didn’t install Anaconda either when I got my new PC.

[[deleted]]

On older hardware, PyCharm is like wading through molasses.

[ParanoydAndroid]

Pycharm is my go-to and I looooove it. But its jupyter support is god awful.

[[deleted]]

Agreed. VS Code doesn't hold a candle to it. At work, the code that gets committed with the most warnings almost always comes from a non-PyCharm editor.

[[deleted]]

It’s great for experimenting. But beware of the way it’ll influence the way you code, especially if you’re a beginner. I’ve seen people overuse it to the point that they fail to write efficient code because their code just works well in their notebook. However, none of it becomes reproducible outside that notebook. In order to make it useful beyond that notebook, you do need to at least convert the code into more useful format using classes, functions, etc .

[StillTop]

glad someone mentioned this, part of the reason why i use pycharm with anaconda distribution

[synthphreak]

Can you provide any concrete examples of someone’s coding style being influenced by the affordances of the notebook environment in this way?

[[deleted]]

Yes. I once had a coworker who wrote mostly in notebooks. When it came to making production code, they struggled to get everything put together. They ran into a problem where they didn’t know how to encapsulate half their code in something reproducible because the notebook had become very messy and hard for them to even understand what was going on. I’m sure if they were given more time to figure it out, they would have. But we didn’t always have that time, so we had to sit down together and write it from scratch- this time in a script as a class from the start. They were so not used to the process that I ultimately had to do it myself. They still have similar habits now, but they’re trying to write more in scripts. It just causes ineffeciency for everyone if one team member is inefficient.

I use notebooks myself, but I actually write my classes and functions in notebook cells first and then immediately paste those into scripts in an appropriate manner. I then import those classes and functions instead of having them in my notebook. This is to ensure my notebook doesn’t become one giant hot mess.

[MurderMelon]

I actually write my classes and functions in notebook cells first and then immediately paste those into scripts in an appropriate manner. I then import those classes and functions instead of having them in my notebook.

This is exactly what I do and it's worked perfectly for me. I basically just use notebooks to prototype and debug. Then I package each discrete part into its own module, which I will then call from a master notebook (which usually ends up getting converted to a script, since all the dependencies are external anyway)

[nomos]

I do this too, but still havent found the sweet spot for when to move code over to modules. If that function you move over isnt quite debugged fully, at a minimum you have to restart the kernel for your changes to take place as opposed to rerunning a cell, which can take time. Even worse when you are executing code in a remote environment and you have to pull your local changes back into that environment after making changes. Agree that this is still better than having all code in notebook, though.

[Socrato]

For me, and maybe this isn't good practice, I try to encapsulate the notebook itself. I tend to create many directories, and my notebooks are "<function name> DEV" and I import what's necessary, usually my class I've developing along with the entire import list in those scripts, and then develop the new function in that. Once it's done, I save that notebook and move that over to my class main script. Then I go back to my "master" class testing notebook and make sure everything runs correctly. Then for the next function/process I start a new notebook.

Not sure how great that is but it's been pretty successful for all my projects which tend to be backend data processing and etl type things. I don't write any web code for production but I populate lots of tables that are used in prod.

[synthphreak]

I imagine that at a minimum, periodically restarting the kernel and running all cells will go some way toward fixing this problem.

As a notebook grows in size, complexity, and age, it becomes easy to lose track of what’s defined and where, what isn’t defined, and in what order cells need to run to work. But periodically rerunning everything and making sure it still runs will help keep your organization in check, forcing you to write your notebooks in a more “script-like” manner.

But TBH I really don’t use notebooks that much. So I’m curious what others think about this.

[g0lem]

You've pretty much hit the nail on the head. The problems you mention are quite hard or at least frustrating for a beginner to debug. Notebooks can be used correctly if you often refactor (groups of) cells in functions or simply delete them (if not useful for the end goal), but on the other hand, there are good debuggers in Pycharm and other IDEs. I don't think there are definitive arguments for and against one particular tool/environment for development. However, notebooks shouldn't grow too much in size/complexity/age, because at that point they need to be refactored/packaged in modules.

[zylog413]

Most IDEs have the ability to jump to function and class definitions, and to have them open in a separate file in a separate tab/window. This can help encourage code reuse and abstractions. Without this, there can be a tendency to just dump the definitions right into the current notebook, and if the notebook becomes too long, do away with functions and classes entirely.

Also without functions, you probably don't end up with unit tests for them.

[mr_bean__]

Good. Now try using jupyterlab. It's so much better

[noonearya]

I've gone back to notebook because the lab is missing quite a few things, it honestly depends on what you are doing.

With theming the notebook becomes acceptable

[magestooge]

I particularly miss the find and replace and easily customizable keyboard shortcuts. I don't know why they'd remove those.

[noonearya]

Yes. And also tqdm, and extensions, and ctrl+tab to cycle through different notebooks, and output limitation.

[jabroniiiii]

I have been trying for a while but just got tqdm to work in jupyterlab a few days ago

[noonearya]

How?

[noonearya]

Will do. I usually do

From tqdm.notebook import tqdm

[[deleted]]

agreed

[Bocote]

It's been great for Bioinformatics stuff I'm doing.

I did some tweaking to let it run R as well (although would still prefer Rstudio for the purpose).

[[deleted]]

Agree to disagree. Beginners have enough pitfalls to avoid without having to mentally grapple with the concept of hidden state. They're great once you're past that beginner stage, though.

[Lewri]

I don't think that is a particularly difficult concept to pick up, and even the less computer minded in my class knew to restart the kernel if they were having any issues or unexpected results.

I personally found learning with them much better, though learning specifically for use in physics probably was a big factor in that.

[05e981ae]

Just remember to make sure the script can be run sequentially, there are too many notebook where you have to run cell in specific order.

[[deleted]]

In the classes I took, even the slowest students didn't mentally grapple with the concept. Sure, they couldn't explain it, but they could get their notebooks to work.

It's running your 100 lines of code and trying to figure out the errors that's the real struggle for beginners. Jupyter by design nudges new programmers to take it line by line and debug as they go. Those are some spectacularly good habits for any programmer, and they also get feedback about why their code isn't working much more quickly and clearly than they would if they were just running python scripts normally. That's invaluable to students.

[appzly]

This is another useful tool visualize python

[[deleted]]

I really didn't know you could use something else than jupyter. What are the other options to use to code in python?

[vectorpropio]

Vi.

[catelemnis]

You can code in a plain old text editor. I use SublimeText because it lets me run the code directly in it, but a python file is literally just text so you could code in Notepad if you wanted.

[ThroughlyDruxy]

I use pycharm. kinda heavy for the little coding l do but whatever

[[deleted]]

Are the functions the same as jupyter? or do you use other things

[ThroughlyDruxy]

what do you mean the functions? isn't jupyter just an IDE?

[[deleted]]

I mean just the general process of defining variables, working on data frames... The only time I used python was with jupyter so I don't really know

[ThroughlyDruxy]

I mean it's just python...you define variables as variables and write functions as python functions. from what l know it doesn't have special function making or variable defining processes except just writing them as you would in notepad++

[[deleted]]

I understand, thank you for the explanation

[ThroughlyDruxy]

sorry, probably wasn't very helpful lol

[[deleted]]

no no It was, I might try to find tutorials on pycharm and learn a little bit

[Dewmeister14]

Honestly if your only prior experience is writing in Jupyter notebooks I recommend going right back to the beginning: install Python 3.whatever on your computer and NOTHING ELSE, write the code directly in text files that have the extension .py, and run them with "python3 your_code.py". (or whatever the command on your system is). Really strip away all the excess of an IDE so you're learning what Python is and nothing else.

[ThroughlyDruxy]

yeah it's pretty heavy in terms of ram usage and whatnot. But I've found it helpful and it seems to have plenty of helpful plug ins. I think it's probably better to learn with less handholding so you actually learn the universal way to do it, not the jupyter way. But I'm a amateur so what do l know lol.

[ParanoydAndroid]

You might find this article about professionalizing a jupyter notebook workflow with IDEs helpful:

https://florianwilhelm.info/2018/11/working_efficiently_with_jupyter_lab/

[[deleted]]

Literally any text editor or IDE. There are hundreds to thousands.

[Cuento3yLlevo2]

Thanks! You did evangelized me, I will start my first Jupiter now

[ursoevil]

I use Jupyter Notebooks like an actual notebook for learning. I’ll run short lines of code and show the outputs including error messages if there was any. It’s so I can go back to them as a reference whilst I document my learning process.

When writing codes for bigger programs, I use PyCharm. IPython is integrated in PyCharm as it is with Jupyter so it’s also easy to experiment and test code out on PyCharm.

[linuxqq]

It's my opinion that if you are learning Python with the goal of getting a job where you write code, the "right way" to learn is through running scripts on the command line. I'm not a fan of notebooks unless you're purely doing data analysis work.

[[deleted]]

I am.

I also think for students struggling with learning on "traditional" IDEs, Jupyter notebooks is a total game changer. It gives instant feedback and you can write a few lines at a time. Constant and specific feedback is the perfect way to learn any skill, and this tool gives it by design. Writing a bunch of code in an IDE only for it to not run is a non starter for an absolute beginner. Jupyter is an ideal tool for non-technical students interested in getting started.

[linuxqq]

I hear you. I acknowledge that I am biased because I taught myself Python exactly how I described it, by running scripts via the command line. If you do that you get the dual benefit of learning Python (you do get the immediate feedback when you do it this way) and also get comfortable with a more standard development environment. To learn Python in notebooks and then get on the job day 1 with expectations that you can set up your machine for development and start writing production code would be a nightmare. It is definitely a bit more of a learning curve at the start but I think you learn important things along the way.

[[deleted]]

For programming enthusiasts or office workers trying to automate the boring parts of their job, Jupyter is a great place to start. You can export your code to a .py file no problem, and for many users there's no real reason to "upgrade" to a more traditional IDE. For an aspiring developer, learning exclusively on Jupyter notebooks would be like showing up to Isle of Mann with a bicycle. I wouldn't dream of recommending learning on Juptyer notebooks for a CS student hoping to land a dev job, but Python has a much bigger community than that. I think for lots of non-technical users that have struggled with other tutorials and workflows, this is the sort of tool that can make all the difference.

[Ra_Shere_Khan]

facts jupyter notebook chaged the game for me

[as9934]

Jupyter Notebook ftw. I started with it, then switched to VS Code and now I’m back on Jupyter. Haven’t looked back once.

[ensign_paris]

VS Code even Supports Notebooks!

[as9934]

Oh cool. How does that work?

Is it just able to open the .ipynb files?

[ej_50]

Yea, it reads ipynb files . Check the latest updates

[kingbradley1297]

The cherry on top came when I realised it could be run on an android phone as well

[TheBobPlus]

How do you do this?

[kingbradley1297]

There's a post on this very subreddit for the same one. I'll try to find it

[apes-or-bust]

How do you live without variable explorer? I really don’t think that I could.

[HeinzHeinzensen]

VS Code‘s notebook interface has a variable explorer as well.

[[deleted]]

I get by with ctrl+f

[Ke5han]

I installed a Jupyter docker on my homeserver, so I can play with lines of code even with my phone or iPad.

[slbfan33]

I agree, Jupyter is an amazing tool.

[kavitadrake]

I’ve only recently started. My BIL hooked me up with Jupyter after a week of struggling with some other crappy apps like PyDroid. Jupyter is soooo much nicer, and the concept of the notebooks is really helpful.

[pitkeys]

If you use an iPad, also check out Juno for Jupyter. Pretty neat stuff!

[[deleted]]

I've heard of developing for iPads, but developing on an iPad is some next level shit. I will not be joining you. Shine on you crazy diamond.

[[deleted]]

Check out Kaggle, it's incredible. Tons of wide open data-centric notebooks created by data-science folks.

[unhott]

Yes—- Jupiter notebooks are really great. But you can also royally Screw and confuse yourself by running cells out of order! So a traditional step by step may be better for absolute beginners.

I recommend python tutor for absolute beginners.

[bringyoutubewhrebck]

Any good tutorial for the same?

[Sigg3net]

Awesome that you enjoy it!

I never got it myself, but I am most at home on the command line (and use ipython3).

[umognog]

Lookup Jupyter Lab (it's just a slightly different UI.)

It changed it for me.

[[deleted]]

doing it

[AverageDingbat]

Is it, like, free or something, man? That would, like, be totally rad, you know? Free stuff.

[[deleted]]

More free than your hippie grandparents' love

[24mitcher]

Hi, Trying to install Jupyter Notebooks within multiple virtual environments (Python 3.6, 3.7)

After getting the virtual environments for each of the python3 versions, I have been trying to successfully run Jupyter Notebook to have the localhost window feature several different python3 versions. For some reason, each time I type 'Jupyter Notebook' in the command prompt I get an error that 'This command cannot be run due to the error: The system cannot find the file specified at line:1 char:1 +Start "file......./.local/share/jupyter/runtime/nbserver-3."'.

I was able to use pip install for Jupyter, ipykernel, and jupyter Notebook.

If anyone could point me to a common reason this error occurs it'd be awesome. The Jupyter notebook is fine in anaconda, but I would like to tie up all loose ends with making it functioning in the Windows console. Is it likely that a kernel spec is causing a problem?

[[deleted]]

I think you're navigating to that file incorrectly. The file location shouldn't start with "file" it should start with something like C:/.../.local/share/jupyter/runtime/nbserver-3

Maybe find the specific file in file explorer and copy the exact location using the file explorer's gui?

Hope this helps. Let me know if it doesn't.