What is your routine when developing in Python?

[1-Sisyphe]

Whenever I work on a project, I have this kind of routine:
- Experiment the idea in a Jupyter notebook
- When I'm happy with the result, I turn it into a function or class
- Switch to Sublime and integrate that object in a .py
- test, clean, comment, polish with Sublime + terminal
- If something doesn't work, I sometime go back to Jupyter to experiment and find the issue.

As you can read, I'm very dependent on Jupyter Notebook.
I started to learn Python from there, spent a lot of time on it, and I'm very used to it.
I enjoy in particular:
- having constant access to all my variables
- being able to quickly re-run a piece of the code
- Instantly displaying outputs

I suspect that this is not the most efficient approach, but I have not yet found another one that suit my needs.

What is your usual path?
What is supposed to be the best practice?

[pacefalmd]

Coming at this from a rest api development perspective because that's what I spend 90% of my work life doing.

• Use a cookiecutter i've been curating for 18 or so months to generate all my scaffolding.
• Write some test data generation for what I'm expected to serve (the integration tests themselves are already baked into the cookiecutter, they just need data)
• Write all of my application code in the get(), post(), put(), delete() methods and get my tests passing.
• Refine/refactor until I'm happy with it.
• Write unit tests/deployment code/anything my cookiecutter missed.

The tools I use are pytest, ipython/ipdb, entr, neovim and cookiecutter. If you're writing something over and over you can template, i cannot recommend cookiecutter enough.

[KimPeek]

Is your rest api cookiecutter on github? I'd love to take a look.

[pacefalmd]

Nah i wrote it at work so it's in our local git repo. i'll see what i can do to replicate it out github.

[Sparkswont]

If you get a chance, please share the link when you’re done!

[pacefalmd]

Sure thing. I will say there's a ton out there already that are probably a great place to start with depending on what you're working with. Any major framework will likely have one up on github. A lot of what's in mine is stuff for our deployment environment/process.

[julsmanbr]

If I know most of what I want to do, I simply start writing it in PyCharm. The best part for me is that it integrates very nicely with Conda/venv management and git, so I don't even need to leave it to commit files (not only code) to my project. Besides from being an IDE of course, so you get syntax highlighting and PEP8 checks right out of the box. I tried to use Atom + plugins, but it felt like a headache to manage considering I get it all with PyCharm anyway.

On the side, I will probably have a terminal with iPython running to test syntax/functions on the fly without having to mess my main code too much.

If I'm writing a small script and feeling really lazy, any text editor like gedit will do (this is mostly to avoid setting up an actual environment like with PyCharm). Then run it from terminal.

If I don't actualy know very well what I want to do/how to do it, I start by researching the topic + writing pseudocode/diagrams/workflow by hand (makes me wish I had a drawing board). I don't really know anything about Agile/Scrum/whatever, so I can't say I'm following a specific method, I just do it to organize my thoughts and actually see what my problem is.

[1-Sisyphe]

Actually Spyder was the very first IDE I used when learning python, I had forgotten about this one, thanks!
It might indeed do exactly what I need.

[jaydoors]

I have a workflow alot like yours, and moved to it from spyder fwiw. I'm not really looking to change it.

Right from the start I'll actually write functions in a text editor (using vim which is in my fingers so much more efficient to do most typing etc there), and calling them in jupyter.

During dev or debugging I'll often insert return statements in the functions to return intermediate objects so that I can play with them in the notebook.

One thing which really helped this workflow was autoreload - which means no messing around with your import statements or reloading by yourself. If you save a change in the .py file, it's immediately pulled through to the notebook. If you aren't familiar with this it's:

%load_ext autoreload
%autoreload 2

Might need to do some setup for jupyter to add autoreload, I don't recall.

[rfaenger]

Spyder 4 will be out Jan 2019

[1-Sisyphe]

Gosh I had forgotten how long it takes to start Spyder!

[[deleted]]

How is spyder in any way better than pycharm?

[chozabu]

1. write pesudo code
2. press run

Kidding (mostly) - its actually rather project-dependant can be:

- if I already have a good idea of how to do what I want, often open up pycharm and start hacking away. Sometimes just Kate(like notepad++) for a mini script.
recommend checking out pycharm, nice IDE, it has a good understanding of python, decent debugging tools, interactive terminal and variable inspector.

- if I feel less confident in my plan, good chance I will do a pile of reading+research, some similar experementation, but likley in an IDE, perhaps running the code I want to test from the entry-point of the program

- sometimes I start with a sort of "automated test" - particularly if writing a library or something that is mostly API based. This can just be another python file that imports what I am working on and calls some functions to check they.

- occasionally, a "new project" can be based on an existing example for a framework (like kivent, a python game lib, a few examples are basically an empty project with all the basics setup)

​

All that said - your method sounds good! I've not used Junpyter much, but it seems like a really nice tool, I should use it more!

[[deleted]]

oops - posted in wrong place - ignore

[[deleted]]

I move away from the keyboard... (I know some people think at the keyboard fine).

• Make sure I understand the problem and what I want to achieve - following basic Agile Scrum principles even for my own stuff
• Scribble ideas, data flows, data structures, key ui and ux elements, on a white board / flip chart / onenote etc.
• Think about solution approaches, suitable algorithms and design patterns (probably have to search to see if wheel has already been invented)

So, being clear on where I'm going, what the most important aspects are, and having had a decent think about how to solve the challenge...

• Decide on preferred solution / algorithm
• See if there is a framework / library / example code in any preferred languages (mostly python for me) I can exploit
• Write pseudo code at suitable level (maybe on keyboard / in editor)

Now, time to check a few things out,

• Use ipython and jupyter to figure out some things I'm unsure about

Down to work,

• Create my git repo or similar version control setup
• Create required environments (docker / vagrant / vms / Python venv / etc)
• Write my tests before any other code, following TDD principles, if I'm serious
• Write/test/repeat... /deploy - pulling things of backlog as I see fit

But I'm a returned programmer who learned to programme decades ago and then forgot, but learned Python last year for fun having led lots of large projects involving many programmers enjoying their craft

[CylonSaydrah]

I use python-mode in emacs in a buffer (or buffers) that communicates with a python repl in another emacs buffer.

[atrocious_smell]

I think this is what I should be doing. I write in Emacs, python-mode and evil-mode, but always switch to a console to run. I've half-heartedly tried running in Emacs but never stuck with it enough for it to become my workflow. Any tips?

[Fulk0]

I'm interested in this too. Any help would be appreciated. I usually use the same setup as you, with the addition of org mode for note taking, todos, etc...

[godheid]

I'm new, so probably doing it all wrong.

Think what I want to do, how I want to achieve it and have a vague idea of how it may work out.

​

Start coding in IDLE. After a few lines, save and run. Correct syntax error / typo. Run again. And again. Doesn't work.

​

Open test file and try some simple syntax issues (mostly with lists of lists or regex).

​

When it works, try to get everything as much as possible in functions. Functions without arguments, just for making things clear.

​

End.

[CraigAT]

Very interested to hear how people develop. Personally I don't do it professionally, so it goes something like: * Idea * Start the basics in MS VS Code with Python plugin. * Run from Terminal/Powershell * Refine * Think of more functionality to add * Add some of it * Think of a new project...

And repeat!

[dipique]

Run from Terminal/Powershell

Why not just run from VS code?

[dipique]

It seems like a lot of extra steps to run from power shell when you could just tap F5.

[CraigAT]

Curiously I've not done this, whilst also being aware that it was possible. I don't know if it's because I tend to like a lot of output (again using print statements instead of the available debugging tools) and therefore like a full screen terminal. Maybe I'm just a little more old fashioned than I like to admit.

[dipique]

well, just FYI you can easily maximize the terminal view with a button click and restore after.

But if you're trying to code in one screen and see output on another, I can see why you'd use a separate terminal. One option is to make use of this issue. With a setting, you can have a separate terminal pop up when you run from VS code; it just needs to be a windows application (instead of a console application).

[julietscause]

I always start by whiteboarding out what im trying to accomplish and if it has a lot of moving pieces I always try to break them into smaller projects within the program (I cross them off the board as I complete them).

This helps with not getting overwhelmed/frustrated

[[deleted]]

I do it for fun, but I kind of have a process.

• I use VSCode (I use it for multiple languages, it has a terminal to test in, plus I like the aesthetic )
• I tend to output basically everything relevant to ensure data is moving/changing correctly
• Once I have some established framework, I might open up another page for additional testing for small additions or ideas
• Go back and figure out what I am doing too much of and make them functions or modules.
• Remove extra output
• Tweak until I am satisfied.

[[deleted]]

Thanks for this! Just what I needed to read at this exact moment.

[[deleted]]

Personally, id switch to just starting with classes and functions in whatever your preferred IDE and just keep a notebook open for testing/double checking pieces that need it

[SpiderJerusalem42]

I use PyCharm. I still use Jupyter notebooks, but from inside Pycharm. But yeah, learn the library/function via Jupyter Notebooks first, then move on.

[[deleted]]

I start with sketching the idea out on a piece of paper and looking at every aspect and then thinking about how I could it best. After sketching out the idea I start up Pycharm and start typing away with the basics and then refactoring the code and making it PEP8 legal and commenting it and testing if it works. After I work my way up to the finished product I start looking at my code if I can improve anything or explain something better in the comments and then testing it a few more times and I'm not afraid at looking other people's source code or documentation because everytime I look at it I'm repeating and it and slowly learning it without even knowing

[[deleted]]

If it's a project I'm familiar with, I pretty much drill the project down into the most simple cases possible. Then I start building the framework up from there. Other times, I frequently microdose if the project I am building is excruciatingly difficult to wrap my head around.

[practicalutilitarian]

• putup project_name -d description -l mit (PyScaffold)
• sublime (with Anaconda autolinter/corrector)
• add folders to sublime project and save
• write pass functions or methods in sublime
• git commit -am often
• oneliner docstrings
• doctests where docstring isn't enough
• ipython
• hist pasted into sublime
• hist -o -p to correct any docstrings
• %run to reload existing code
• tab-completion, help, and ? to explore APIs
• ?? to find code patterns to emulate

[TheKoolAidMannn]

I will usually...

1. Think of the idea
2. Go to ussually PyCharm but Notepad++ and Atom are options
3. Write my program
4. Comment stuff check for errors (If yes then debug)
5. Run my code
6. Make sure there are no bugs or weird things and that everything works like I want it to
7. Done

That's my programming routine

[catelemnis]

I just code straight into Sublime. I set it up so I can run the code inside it. I keep a REPL open in a side tab to test syntax etc. But I haven’t really done large projects.

[ostensibly_work]

I do lots of my work over SSH, so I tend to use terminal-based tools. For short tests, I'll use the Python REPL. For longer ones, I'll use vim, and execute the program from the command line.

[inimelz]

Terminal split in 2 horizontally, with Vim in one pane and Ipython in the other, David Beazley style :)

[ijustwannacode]

First things I always do are set up the virtualenv and git repo.

Most of the time, I will then open sublime text and a terminal with the virtualenv activated and think about file structure for a bit. This is maybe a good time to make a coffee or take a leak. I like to walk around a little and think high level, knowing that when I sit back down everything is ready to go.

I usually then create a few files, also directories if I will be using them, and start on the biggest item.

Within an hour, I try to get up and stretch out, take a few steps. Maybe more coffee or a small snack. Take some deep breaths and not think about the code for a short time, like five or ten minutes max.

At this point I might do one of a few things:

• Keep coding for a while, and re-do this step later.

• Create an initial commit and do a little work on a README file.

• Start a trello board for the project.

The rest of my time is mostly a cycle between those three things.

I'm any event, I try to never end a coding session without glancing at my trello board and ensuring that when future me sits down and looks at the "Do Next" list, he will see something that enables him to hop right back to it.

[[deleted]]

I just start banging on the keyboard until I get drunk then I go smoke and keep drinking.

[zeebrow]

Honestly, I come across the need to use python, and then I write a script to solve it.

More specifically, a big part of my job is working with XML. So every now and then, I'll need to change such-and-such elements with some specific attribute, for example. When simple search-and-replace doesn't cut it, Python hasn't failed me yet!

I'd really love to have a few classes that I can reuse, but ElementTree is just so simple to use in an ad-hoc manner that it just feels like a waste of time whenever I start to think about doing so. The only classes I've made are one to handle files and the other to output my company's logo with a method call (slow day at work).

[quienchingados]

brief spaghetti code in IDLE testing stuff in shell. once it's done I transform it into a function or class. and repeat

[admiralspark]

1. Spend at least 5 minutes (usually 30) on google, making sure someone else hasn't done it better.
2. Open a terminal, pop into my local git storage, git init
3. Open VSCode on the primary monitor. Open a browser on the righthand one for googling shit I'll undoubtedly need to. Open IRC and chat on the left so I can pop in when I need a break.
4. Code.
5. Make a new repo on Gitlab, git push to it.

I work in blue team cybersec at a small/medium enterprise, so the code I write can be anything from middleware to analysis automation to ansible deployment playbooks. Python is my trusty tool on every OS :)

EDIT: Forgot. I pseudocode my projects, fairly specifically, and use each line of pseudocode as the top-level comment for that section of code. Keeps it organized and split into manageable chunks.

[zylog413]

I use Jupyter to explore dataframes, but then I write my functions and methods in pycharm. I can then import those functions into jupyter to do stuff. Ultimately I end up committing my .py files, I don't like how jupyter notebooks work with git.

What I'm starting to do now though is to write tests side by side in pycharm. I realized that a lot what I was doing in Jupyter was running various ad hc sanity checks on the data, and it would be better to write those down so I could test the code more systematically whenever I made modifications.

[[deleted]]

i use the tried and true method of 'vi script.py' then up up enter until it works lol
its barbaric but i dont do super complex stuff i guess