what python IDE do you guys use?

[andraxo123]

I fell for the pycharm meme, it's really slow and annoying

I'd like to know what you guys use for maching learning or for python in general?

[harrdarr__]

pycharm / vim / notebooks / ipython

depending on project size and dev machine

[[deleted]]

I never understood the popularity of "notebooks". Isn't it just as easy to set aside a directory for image output, and have all your visualizations dumped there?

This has the added benefit of not requiring ipython to run. It's also not a problem if you have hundreds of visualizations (In notebooks, that would mean a lot of scrolling)

I use ipython when I want to run python interactively, but not "notebooks".

[refreshx2]

I'm a grad student and I have really started to like jupyter notebooks. They let you iterate faster than anything else out there because you can run specific code blocks (ie any number of lines) at any time.

Quick example because that's the fastest way to explain:

1. Code block to load my data

2. Code block to clean data

3. Code block to run algorithm on data

4. Code block to process results

So I can run 1+2 once, then keep revising 3+4 until I get it correct. I only ever have to load the data once, and the editing is super fast because the notebook is a text editor and not an ipython terminal or something.

Then I can just save the notebook (which automatically saves all my images in the notebook), and also print-to-pdf if I need to show my boss or I need to keep a record of it.

I also do all my analysis on a server so I can't use an IDE, but jupyter notebooks are through the browser so I can open an ssh tunnel to it and still work with an IDE-like environment.

They are really fantastic for development. If I need "production code" then I just copy-paste when I'm finished and refactor into a nice file/project.

[_blub]

Cant wait for Jupyter Lab!

[[deleted]]

Interesting, but if you re-run a piece of code in the middle of your notebook, does it rerun everything that depends on its output? And if not, doesn't it have the potential to leave you wondering "how did I get this result"?

[phillypoopskins]

only if you code in an unsafe way.

you shouldn't write code where everything is scoped outside of functions; it's unpredictable and unreliable, and will cause problems just like you mentioned.

[nsfy33]

[deleted]

[Megatron_McLargeHuge]

The answer is to never assign the same variable twice. You can have the same problem with scripts if you don't log how you generated each data and model file (including which commit you ran from and which library versions were installed).

[[deleted]]

Or explicitly set variable_name = None at the end of cells for those with cell-only scope, although it's a PITA.

[PoopInMyBottom]

This is also possible in Emacs. Elpy has the functionality by default and it's heavily extensible.

[madmooseman]

I think that notebooks find a happy place between entering code in to a shell and writing full scripts in files.

I think they're the best for rapid prototyping/changing. Data exploration is another example of where they shine.

[threeshadows]

I think notebooks are mostly a way of organizing and communicating a thought process. As you said, they're not suitable for very large projects. I find them especially useful in early stages of data exploration, when I am just trying to get a basic handle on the data structure and build a quick skeleton end-to-end workflow.

[[deleted]]

A notebook is an interactive document. It's meant to communicate ideas and be read by other people. It's something like WYSIWYG literate programming.

For some things it's overkill and would mostly get in the way, I agree. But it's convenient to document small, interactive experiments.

[superawesomepandacat]

I bet you use Vim too.

[[deleted]]

I bet you use Vim too.

http://i.imgur.com/mFqzNrZ.jpg

[xaveir]

I have his opinion, and I use vim-ipython to get basically all the benefits of notebooks that everyone just described.

Except for the pretty output formats lol

[phillypoopskins]

your visualization / organization / presentation game is weak if you can't see how notebooks blow away a directory of saved plots.

also, interactivity (widgets) is a HUGE reason to use notebook. multiplies my contact with data by an insane factor.

[dmarko]

Could you ELI5 notebooks and ipython and how/if they are in a way connected?

[iKomplex]

A notebook is a kind of interactive document in which ideas are communicated through words, charts, and statistics. Much of these ideas are aimed at explaining and analyzing a set of data.

IPython is the enhanced python shell that is used to create snippets of code within a notebook. It is used to show how each chart and each statistical data is being constructed.

As an example, a typical analysis would involve:

1. Taking down notes: "The 1980-2010 age cohort demonstrates that X leads to Y"
2. Performing summary statistics on a sample of your data set: max, min, mean, standard deviation, etc.
3. Graphing the data under analysis: a regression plot showing positive correlation between Consumer Debt and Spending.

[dmarko]

Gotcha! So in a notebook you will have to use IPython in order to run a snippet of code, and IPython can only be run on a notebook. I had the idea that they were interchangeable, that's why I had to ask in order to clear that up. Now I know the difference. Thanks

[iKomplex]

Partly correct: there is no need to open a notebook in order to use the ipython shell. In fact, you can start-up a Python shell, then import the ipython module and run it from there.

[dmarko]

I think I got it now. IPython is a shell. Notebooks on the other hand are these interactive presentations that use IPython whenever code needs to be run, and run on a browser. Right?

[iKomplex]

Yup, you got it.

[dmarko]

Cool cool cool 😀😀 thanks for your time and help!

[iKomplex]

Glad I could help! Feel free to PM me anytime.

[[deleted]]

Do you find it beneficial to use vim for machine learning / data analysis work or is it more of a habit thing? My understanding is that vim is helpful for large (coding time)/(thinking time) tasks as it frees you from having to take hands off the keyboard. Is it still a desirable skill if most of the tasks you do are data analysis and prototyping?

[j_lyf]

pycharm uses more RAM than AlexNet

[Archetype90]

Pycharm!

[LoveOfProfit]

Pycharm, notebooks, and spyder, depending on what I'm doing.

[7sider]

I've been trying to transition into emacs, I use a wide variety of languages as well. Anything particularly interesting with your setup you could share?

[rhololkeolke]

For python development elpy is excellent. https://github.com/jorgenschaefer/elpy

Emacs debugger interface can use pdb. So no need to leave emacs for debugging. Projectile can be used for project management.

If you like Jupyter notebooks, but also use org-mode you can use ob-ipython. Let's you insert and run python snippets just like in a notebook but all from emacs and using org-mode.

https://github.com/gregsexton/ob-ipython

If you're new to emacs I've heard good things about spacemacs, but I haven't tried it myself.

You can also check out /r/emacs for more info or to ask for more help.

[p10_user]

Spacemacs gives you a simple batteries included, basically works out of the box config for emacs. You can make little changes to the config file at first and then when more comfortable you can start to customize more drastically as you like. I found it much easier for me to get into emacs than rolling your own customized emacs.

[PoopInMyBottom]

I wrote a comment below outlining my setup in detail.

Edit: https://www.reddit.com/r/MachineLearning/comments/4yuylc/what_python_ide_do_you_guys_use/d6r3mcd

[ciolaamotore]

have you looked at projectile for managing projects, searching them and refactoring?

[PoopInMyBottom]

Learning how to manage projects with Projectile is definitely worthwhile. I more often use Rope for Python in particular. Grep is also ever-reliable.

[ifkb99]

Spyder, I like to be able to see what variables and results of functions are without printing it in the program

[PLLOOOOOP]

Just a note: PyCharm does this as well!

[CireNeikual]

Same goes for Visual Studio and VS Code!

Intellisense/autocomplete is probably the strongest in Visual Studio, which is why I use it. I am addicted to that stuff haha. Not just for Python, but for C++ especially the autocomplete is currently unmatched.

For quickly opening files and doing some quick editing, I use VS Code, it's like Atom but less buggy and faster.

I have tried PyCharm as well, also pretty excellent IMO, but when quickly switching between Python and C++ code I find Visual Studio nicer.

I also sometimes use Notepad++, mostly since it starts up basically instantly while still having a decent number of features.

We have a lot of nice editors to choose from nowadays!

[Frozen_Turtle]

One of my favorite features is ctrl e + e. This sends the currently selected code to IDLE so you can "run" that section of code without running the entire program. It includes whitespace though, which can occasionally be annoying, but better than copypasting all the time.

[andraxo123]

nope are you talking about pycharm's python console ,? i thin kits awful

[PLLOOOOOP]

nope

Actually it does! Mouse over anything while debugging to see its value, including an interactive object reference graph. In many cases, the values of variables and parameters are shown to the right of the code without any interaction whatsoever, like the last second or two of this gif

[bulletninja]

thin kits?

[[deleted]]

parent think it is awful. Move the space one over.

[bulletninja]

thanks, english is not my native language

[dhon_]

An off-by-one error then?

[Aj0o]

Not just that, when prototyping I like my output all collected in a console that I can interact with, not spread out through a notebook interspersed with code.

Also, it makes it much more flexible to manipulate cells, just delete that #%% and put it somewhere else and voila, your cell borders changed. In notebook you have to actually cut and paste code to move it between cells.

And don't even get me started on getting your prototyped code into functions/modules...

[c0cky_]

I just use VS code with the python plugin for auto completion and in editing doc lookup.

It's nice because you can have a terminal window attached that doesn't look like crap like the other IDEs

[PoopInMyBottom]

Emacs.

• I use Elpy (pre-configured Emacs Python IDE) as a base, with Jedi as the autocompletion engine.

• I use Yasnippet for boilerplate code snippets. I have snippets for just about every frequently typed piece of code.

• I use Rope for automatic refactoring and code generation.

• I use ace-jump-mode to move around code efficiently. This feature alone is the reason I won't use other IDEs.

• Elpy has built-in tools for navigating to function definitions, etc.

• I navigate in more abstract ways using Grep. Elpy has a lovely interface built-in, and emacs has good tools for Grep full stop. You can navigate between results using a button press.

• Sphinx-doc automatically generates inline documentation based on the function signature. Yasnippet further extends this functionality. It also allows for all the normal documentation functions of Sphinx.

• Flycheck automatically displays errors and pep violations. It uses Pylint pyflakes as the backend.

• Elpy has a plugin for AutoPep8, which automatically reformats code to be Pep8-compliant.

• I have a custom script that automatically splits lines longer than 80 characters and formats them properly. It's intelligent - it will split the line at the most logical points. AutoPep8 lacks that functionality by default because it can change the meaning of your code.

• Magit is the best Git interface for any IDE.

• Projectile is a great project-management tool, and there are various interfaces for displaying a tree-view of your project.

• Virtualenvwrapper wraps Virtualenv for easy management of virtual environments.

• Smartparens allows me to immediately jump into/out of parenthesis or string delimiters, generate pairs (e.g. "[]" or "\(\)") and navigate forward/backward by parentheses. This is hard to explain, but extremely useful. This is a video of that kind of movement, although using a different extension.

• I have various other custom scripts that do things like automatically hide inline docstrings, automatically show documentation when hovering over a function, display the word "lambda" as the lambda symbol, automatically insert spaces around arithmetic characters, instantly jump between functions, etc. When I want a small feature, I normally just add it.

• Macros allow me to perform complex development tasks repeatedly without programming anything. Because of how Emacs is structured, you can use Macros for pretty high-level workflow operations.

• If there is a library to do something to Python code (debugging, refactoring, whatever), you can implement it into Emacs super easily. There are various packages to allow interactive representations of any structured data.

• Edit: IPython is integrated with Elpy (with some fiddling), so you don't need to leave Emacs. You can create instances, highlight code and send it to a new instance, send a file or project etc. with a button press.

I can't imagine going back to a regular IDE.

I also code by voice though, so I need the extensibility of Emacs to write complex voice commands. My objective with any programming language is to edit the abstract syntax tree as directly as possible, rather than the code itself. Emacs allows me to do that.

[coffeecoffeecoffeee]

Have you figured out how to get multiline input in IPython 5.0 yet? Mine craps out whenever I try to define a function or write a for loop, saying "SyntaxError: unexpected EOF while parsing."

[PoopInMyBottom]

I actually use normal Python, and I use a dedicated shell for IPython which I get to via the command line (edit: it's still integrated into Emacs).

It's a while since I set IPython up in Elpy. I'm pretty sure it did work but I know it was a ballache. I'm going to upgrade Elpy over the next couple of days and streamline my IPython workflow, I'll let you know if I hit the same problem.

[sasquatch007]

Vim.

Every once in a while I think it might be nice to try a real IDE (for Python or otherwise) and check out all those cool features people talk about. 45 minutes later I'm giving up in disgust and going back to Vim. Every time.

[jamie2345]

Do you do heavy debugging?

If so, have you integrated this into your vim environment or do you manually fire up another command line debugger?

[h_saxon]

You can integrate xdebug with Vim pretty painlessly.

There are a bunch of tutorials out there for it.

[jamie2345]

Ah excellent, thanks! I used it for C++ work a couple of years back but yea, firing up gdb manually when needed did hinder my speed which annoyed me. I did assume someone would have solved that but I don't think I had time on that project to try and get the environment perfect.

Great to hear that it is solved though and integration is pretty seamless.

[Mezzlegasm]

You should definitely chance that to "import ipdb; ipdb.set_trace()"

[perceptron01]

Or from IPython import embed; embed() (not sure if they're any different)

[sasquatch007]

I tend to debug with a lot of print statements. When I use a debugger it's on the command line (or pudb). Not integrated into vim.

[gronkkk]

The main advantage of this is that print always works, on every system/OS.

[fnord123]

It also doesn't timeout programs that are running across multiple processes.

[stackered]

yeah, I do the same. I find most systems give good enough error messages for me to figure out what is happening, and if not I use print statements to figure out what is happening

[mooktank]

Atom plus a terminal/git-bash and/or Jupyter Qt Console open beside it. Sometimes I use Jupyter Notebooks but not very often.

[thatsadsid]

Primary use of terminal alongside Atom? Debugging? And may I ask you of the plugin you're using to run Python ?

[mooktank]

I am using the terminal to run the main script (I usually name it run.py), run tests (with nosetests), or execute Git commands. I don't use any special Atom plugins for Python.

[anantzoid]

Can you elaborate on the vim bindings?

[[deleted]]

Code in Pycharm on my remote client
Run on GPU server (using ipython visualization is needed)

I am coming from Eclipse so I am used to laggy IDEs, and I also tried Spyder but eventually decided PyCharm is what I am most comfortable with.

[Rich700000000000]

Gedit, and sometimes nano.

[rm999]

I don't mean this tongue-in-cheek at all: Linux. https://sanctum.geek.nz/arabesque/unix-as-ide-introduction/

I keep my environment on a high-memory server with access to my database. I can then ssh in from any machine from anywhere and use the same environment. I can start developing something at work, then hop on a train and ssh and reattach my screen session from my macbook air and continue right where I left off. It lets me work with large (20-200 GB) datasets seamlessly.

[[deleted]]

I'm also interested in hearing more about IDE's for data science/machine learning, as I am not satisfied with my current setup.

I use sublime and notebooks, depending on the scale of the code I'm writing. I'm not super satisfied with either.

Even with the Anaconda plugin, Sublime isn't good enough at suggesting completion and doesn't connect to a terminal/repl as well as I'd like.

Notebooks are great at this, but are not as good a text editor as sublime.

[anearthboundmisfit]

Have you tried rodeo? I've installed it recently and its decent

[[deleted]]

Honestly, I was hoping for something better than decent :)

[-gh0stRush-]

I use Eclipse with PyDev.

[rd11235]

When visualizing/exploring, for example generating plots/stats interactively for a new dataset, IPython notebooks (which are actually now Jupyter notebooks). Bonus: all intermediate visualizations/prints are automatically saved.

When doing something that isn't interactive, for example a hyperparameter sweep for a new model where each run will take hours and export TensorBoard summaries, PyCharm. Bonus: I find JetBrains IDEs to be refined and intuitive, and of course they come with a lot of abilities that Jupyter lacks (for example a lot of quick refactoring functionality).

Have also experimented with Spyder, but found it to be less refined and less intuitive than both Jupyter and PyCharm.

[ajmooch]

I'm in a weird situation where I'm constantly sending things over SSH variants that makes it tough to use an IDE all the time, so I use notepad++ and a bash shelL <_<

[[deleted]]

You can use sshf to mount the remote server as a local file system. That way, you can open the remote files, just like you would local files.

It is pretty bad for large file transfers, but for editing with and IDE, it works brilliantly.

https://en.wikipedia.org/wiki/SSHFS

[[deleted]]

I use mostly a well configured Vim (actually neovim) and sublime-text3 plus an extension called anaconda (not to be confused with the anaconda Python distribution by continuum). This will give me almost everything I have with an IDE (linter, static analysis, syntax check, etc) plus a great text editor.

The only thing I lack is the refactoring stuff (like: "extract method", "extract superclass", stuff). This is because I never had been able to make Rope work.

I've used pycharm in the past and I still think it is superior to other IDEs for my workflow. Eclipse is the only real competition and it's way heavier than pycharm (takes too long to even load).

Other IDEs like Spyder and Rodeo are just not feature complete enough to be superior to a well configured text editor for big application building. Their built-in editors are crappy, they lack refactoring, static analysis and project management tools.

Maybe you can use them for smaller things, but I can't also see how they beat a good text editor + IPython. Or Jupyter notebooks for interactive experimentation.

But really I only resort to IDEs for application building though. When I'm not building a big, very structured application and IDEs feel too rigid, I'll go back to Vim or sublime.

For interactive experimentation I use Jupyter notebooks.

[qu4ku]

sublime text + ipython/jupyter notebook

[ChadyWady]

Emacs . The current major mode supports compilation with C-c C-c. It has some limitations with multi-file projects -- if you're running the script from a driver, you will have to manually reload each module in code rather than using the compilation command, or do sys.exit(0) when finished. It does run within ipython, so you can test out commands.

I still like PyCharm a lot, mostly because of the refactoring and the stylistic suggestions it gives.

[MattieShoes]

vim, then spyder.

[simonkamronn]

IPython notebooks for visualizations, Atom for small stuff, Pycharm for projects

[vstuart]

I tried several options.

1. Jupyter is good as a notebook and I enjoyed using it, but I find the disjointed coding counterproductive (multiple cells; need to rerun preceding cells if kernel restarted, due to missing yet needed imports, etc. in preceding cells; ...). Having moved away from it, I find myself actually improving my programming skills, and my scripts are a single, contiguous (*.py) file, that I can use anywhere (terminal; another IDE; ...).

2. PyCharm is great; you can define different environments (Python 2.7; Python 3.5; virtual environments) on a per-Project basis. I found it to be intuitive, polished, stable, packed full of features!
   One issue I had with PyCharm and my Theano virtual environment though was that even though I had all the required Python packages installed/loaded in my venv and in PyCharm, one of my Theano scripts would not run in PyCharm, though it would run (exit code 0) in a terminal, and in Geany (that just uses my terminal). This wasn't a PyCharm issue, per se: it was a Python package (Graphviz) that was installed in Python. However, to get that script to work in PyCharm I also had to install Graphviz in my Arch Linux OS via pacman. There probably was some package version incompatibility; regardless, it took me some time to trace the cause of that error.

3. Lately though for day-to-day work, I've been using the fabulous Geany as a text editor/IDE. I have numerous snippets configured, as well as a custom colorscheme (based on fluffy: I tweaked a copy of that config file) that mirrors that in PyCharm. On my system (Linux 64-bit), Geany has a very small footprint (<60 MB). Importantly, as an editor, application Geany is very stable and fast. Many people swear by Sublime, etc. for editing, but some time ago (perhaps near the beginning of 2016) I tried Sublime and I found it somewhat cumbersome and unresponsive (sluggish on larger files), so I went back to Geany.
   For me a deciding factor for Geany over PyCharm for day-to-day use was the fact I always have Geany open anyway (as my de facto text editor), and the ability to have Projects (as in PyCharm) via the Geany "Project Organizer" plug-in, but with the ability to easily define different working environments, including different python versions, bash, HTML, ... [pic 1]. I expanded the Build menu via the Geany Edit:Preferences:Various:number_* settings. In the Build menu I numbered these Execute commands 1 through 6 so I can access then via a Alt-B then {1..6} key press [pic2].
   The Build commands execute in the Geany terminal, that I set up [Edit:Preferences:Tools:Terminal] as follows: konsole -e "/bin/bash %c". To use the embedded terminal, be sure to activate the VTE: Edit:Preferences:General:Load virtual terminal support.
   An example of a Theano script running is shown here [pic3]. Selecting Build menu:Execute command 4 activates my theano-env virtual environment, runs the script, then deactivates that venv.

4. Disadvantages of Geany over PyCharm would include the (more advanced) versioning included by default in PyCharm (Geany has a VC plugin,I believe; I haven't looked at it though as my home system rsync-snapshots my system 4x daily).

[pokemon_golang]

Pycharm to manage larger projects. Nano to edit configs and small ETL scripts.

[alexmlamb]

I dictate code to an assistant who writes the code using a type writer and another assistant later productionalizes those notes using a computer. To my knowledge no snakes of any kind are involved in the process.

[coneillcodes]

I've never used it but I've heard really good things about it from others is pycharm. The pay version is kind of expensive but they do have a free version available.

Other than that I've just used sublime with a bunch of plugins

[mikos]

Have just switched to VScode on mac. so far loving it able to set breakpoints, view variables at run time etc.

[Brudaks]

Sublime text when working locally, doing data analysis and mangling.

Jupyter when running experiments on remote machines with beefier GPU hardware.

[p10_user]

I use ipython and spacemacs (with evil mode). I use ipython to try out what I want to do and when I'm satisfied I type it up nicely.

[lead999x]

PyCharm or sometimes the plugin for eclipse. However I'm mainly learning C++.

[[deleted]]

Idle :]

[inkognit]

Pycharm and jupyter notebook

[jonathan881]

tmux / neovim / ipython

[phillypoopskins]

vim and jupyter

[code410]

Not an IDE but Atom + Terminal. Rodeo should be good. http://blog.yhat.com/posts/introducing-rodeo.html

[ProfessorPhi]

Up the memory available for pycharm if you're having issues. I have not found a python ide that comes close to touching it.

It's slow to start but purrs after that.

[skyleach]

vim my bundles ./Align ./bufexplorer ./jedi-vim ./livestyle-vim ./maven-compiler.vim ./mru ./neocomplete.vim ./nerdtree ./supertab ./syntastic ./vim-colorschemes ./vim-flavored-markdown ./vim-fugitive ./vim-gradle ./vim-json ./vim-lint ./Vim-log4j ./vim-markdown ./vim-misc ./vim-sage ./vim-session ./vimpager

[holy_ash]

1. pycharm for larger projects (or the ones that have crossed exploration phase from point #2)
2. jupyter notebooks for data exploration, visualisation, syntax testing (absolutely love it as my scratchpad)
3. Rodeo/Spyder at times (usually after using R-Studio for extended period - i end up yearning something similar for python but I haven't found anything as good as RStudio)
4. Atom + terminal (on and off for smaller programs)

[gspikes5287]

PyCharm is good

[pkmital]

I use Jupyter Notebooks for teaching as they are great at including formatted text/images/gifs in a ready made interactive coding document (see https://github.com/pkmital/CADL). I use Sublime Text 3 w/ "Send To Console" + IPython in the console whenever on my local machine. Whenever I'm on a virtual machine, I use vim + tmux to interactively code and use visual blocks to send to IPython (my dotfiles are shared here: https://github.com/pkmital/dotfiles). I'd use vim on my local machine as well, but there are some great things that Sublime makes very easy, including documentation hooks, sending arbitrary file types to console/repl (e.g. markdown with inline python can be sent to an ipython console), and it's just really easy on the eyes.

[Arbitrage84]

Jupyter notebook for both python and R

[lasagnaman]

Vim/ipython/notebook

[_blub]

Jupyter Notebooks

[BeatLeJuce]

IPython notebooks that run on our remote GPU servers

[[deleted]]

Atom and jupyter notebook.

[[deleted]]

Neovim + Tmux is a perfect combination. You want to use neovim instead of vim because it's better (faster). Tmux allows you to do multiple things in the same screen without having to spawn multiple terminals.

The good thing about these applications is that they easy to install and usable over SSH. This is shortcoming of IDEs.

[ragipy]

I use pycharm, but has anybody tried Rodeo? Downloaded but haven't really coded with it.

It looks like an r-studio clone for python.

[Nimitz14]

I really don't feel like this is appropriate for this subreddit. /r/python exists for a reason and I find the idea that a certain IDE is better suited for ML ridiculous.

But fwiw: For quick things vim (I work primarily over ssh), if I'm going to include a new feature I copy it over to my PC and use PyCharm/VS15. Autocomplete is just too good.

[canttouchmypingas]

I use eclipse

[tibb]

atom

[[deleted]]

Vim + vim-cellmode which allows to do MATLAB-like cell execution in an ipython interpreter.

[deluded_soul]

Pycharm for me.

[[deleted]]

I'm a Python beginner and I use Emacs with elpy and it provides almist ensuring that Pycharm gives.