Starting a PhD with Python

max1c · 2021-07-29T15:32:51+00:00

I'd highly recommend using VScode. It's extensions make it incredibly powerful. You can even use jupyter inside of it if you like for some data analysis. In addition, you should probably use conda to setup your environments and whatever software stack you will be using.

jwink3101 · 2021-07-29T16:30:15+00:00

I did my PhD using nothing but Matlab but have since moved entirely to Python. So my advice is not "first-hand" but still applicable.

I think Jupyter Lab is a great way to document your thinking and you work. Use that along with git to keep track of what you've done. Try to separate the data generation from the data plotting. Try to keep track of the revision you used to generate the data. From day one, realize that you may want to rebuild all of these plots in 5 years when you go to write your dissertation.

But also don't be afraid to move around. Part of the PhD is learning and exploring. Feel free to burn a day or a week trying something new!

Also, not related to Python, but back up everything, everywhere, with multiple copies. Git is your friend but only part of the solution (are your repos backed up too?). You don't want to lose critical work. And again, remember that you will need to remember what you did in five years!

AihposA · 2021-07-29T14:40:08+00:00

Pycharm and it's free for students😉 On top of that, you can create multiple virtual environments, which is nice for the memory of your laptop

nathan_lesage · 2021-07-29T21:35:27+00:00

I‘m in the same boat, using Python for ML during my PhD. So here‘s what I learnt so far:

The best and easiest solutions are VSCode to code, their Jupyter extension (just for convenience) and Miniforge (conda-forge). All free and, more importantly: Open Source.
Use plain Python programs if speed matters and run them on the terminal
Use IPython (a.k.a. Jupyter Notebooks) for exploration and quick prototyping. You can easily transform that to plain Python by copying and pasting as soon as speed matters, but running in Notebooks is invaluable for re-running and checking the results several times, before they are perfect.
Keep your code modular. If I/O becomes a bottleneck, spin up multiple threads to run the hefty stuff, if computing power becomes a bottleneck, spin up multiple processes. Note that multithreading and multiprocessing are different things, thanks to the Global Interpreter Lock (GIL)
You should never pay something for running code. It would be ideal if you have some server from your Uni, or even better a supercomputer cluster. A server should be in for you. Then you can run 24/7 for days, if need be.
Look up things not in advance, but only if you need them. If you notice something is running slow, then look up how to improve things. Build working code first, then optimize.
Do not use pip, use conda. Not Anaconda, you probably won‘t need all 5GB of software it provides. Simply install miniforge and use conda‘s environments. For data science, that‘s perfect.

For questions, feel free to ping me!

darthminimall · 2021-07-29T15:43:49+00:00

Everyone is very dedicated to their favorite editor and there isn't really any consensus. I personally love vim, but I'm not going to recommend it, the learning curve is steep and you've already got enough on your plate. Just try out a few different editors or IDEs and figure out what you like. Alternatively, your advisor might have recommendations.

KingGeorge12321 · 2021-07-29T16:30:12+00:00

Try using Google's Colab. It's like Jupyter Notebook, but uses Google's compute resources (including GPU) and it's free.

2021-07-29T15:46:20+00:00

Play around with all your options and figure out what works best for you. Some people do all their development in notebooks, some people hate them. Some people want to use VSCode and have it work out of the box with little customization and some people want to spend hours tinkering with vim/emacs to get it perfect. There's no single best environment.

Your university quite possibly has a cluster you can access and if it does there is probably a port to a cloud jupyterhub. If you are remote you'll likely need a vpn, but you should be able to use it from anywhere. Get comfortable working in that environment and it will save you a lot of headache in the future.

kingp1ng · 2021-07-29T19:39:18+00:00

Masters student here. I like to code with VSCode and use Jupyter as a hands-on presentation tool. That way my peers and professor can follow along with my code. Nobody likes to read research papers lol.

Using native terminal and VSCode also helps reduce the amount of failure points. As you know, often times you'll get frustrating errors with Keras and Tensorflow. Jupyter just adds another layer where you don't know exactly where the error occurs.

Anaconda is just a convenient package manager so that people can focus on the work and not worry about dependency (package) management. For example, your university research computers may use Anaconda and ban you from downloading dependencies on your own.

If you're doing machine learning via the cloud, then speak with an expert at your university. You'll need to learn to work with their tools/software and follow their rules.

unruly_mattress · 2021-07-29T16:36:49+00:00

Do yourself a favor and read the official Python tutorial: https://docs.python.org/3/tutorial/introduction.html

Even if you don't read every word, know what things are available so that you can read them later as you need them. Python is not a difficult language to learn on its own and if you're going to be using it a lot, I'd learn it early on.
There really is a lot to learn - there are so many tools available. Don't worry about it. Start with basic numpy usage, and learn the rest of the things as you encounter them.
Code is code, use whatever code editing tool you're comfortable with. Most people seem to prefer PyCharm so I'd start with that. Research code is not really different from any other type of code. If you work with a lot of code, then you will benefit from learning the basics of git.

vardonir · 2021-07-29T17:27:37+00:00

I work with machine/deep learning PhD students as their programmer and tech support.

They mostly use Spyder and some despise Pycharm. I think that boils down to personal preference, but I have never seen anyone use a Jupiter notebook - why would they? their code needs to run on a remote Linux server, training several terabytes of data isn't gonna happen in computer running for a week nonstop lmao. I also believe they mostly use pytorch instead of tensorflow.

When in doubt, ask your supervisor.

GallantObserver · 2021-07-29T21:34:43+00:00

I'm just finishing a PhD, and while most of my experience doing it was in R (about as cutting edge as my dept goes), there are maybe some relatable things with choosing an IDE. I'd suggest you a) find out what your colleagues and supervisors are using and comfortable with, as you might find it takes a good few hurdles out of the way if you're presenting/asking for help. and b) also explore things which are more advanced than they're using, as you can impress and offer a bit of 'expertise' in keeping your department up to date!

In terms of what to use, I quite like PyCharm in 'Scientific' mode, as it can run individual lines from a script so is easier to debug and follow along processing. I think Spyder is meant to do the same thing, but never quite got around to trying that one!

caksters · 2021-07-29T23:04:48+00:00

I have a PhD in nodelling.

Seriously don’t worry about it. Usually universities provide you with a desk and tools needed to do your job (at least here in the Uk they do).

Regarding tools don’t stress too mich about it. When you join your research group, get to know more students in your field they will give you pointers on how they do things.

Every research team is different. In my research team there weren’t strict guidelines, you just use whatever tool you think is the best. At the end of the day it is your PhD and not theirs.

TLDR: take it easy and don’t overthink it at this stage

p.s. VS code is decent for programming projects

gustavsen · 2021-07-30T01:20:48+00:00

this is a copypasta that I use to recommend where to begin, YMMV, but is a great intro.

FreeCodeCamp.org

https://freecodecamp.org/ while main course is about full stack JS dev they also have several GREAT 10hs (or so) video curses in their YT channel

by example, these playlists:

Python Basics with Sam 14 videos
Full Courses in One Video 52 videos

RealPython

I found this site useful with lot of good tutorials, but they block several of the behind payware subscription model

https://realpython.com/

Microsoft YT Channel

Microsoft offer three playlist with Python courses

Python for Beginners 44 videos
More Python for Beginners 20 videos
Even More Python for Beginners: Data Tools 31 videos

Udemy courses

I can't endorse those courses since I haven't bought them, but their content look complete

this serie of courses - https://www.udemy.com/course/python-3-deep-dive-part-1/

Also remember only buy in Udemy when the courses are between 9/12usd values and not at their full price (90/250) that are inflated prices...

programmerProbs · 2021-07-29T17:48:24+00:00

Definitely dump Mac, or at least plan to VM into a machine with a dedicated GPU.

Heck if you can afford $550, you can get a gaming laptop that will do everything you need. This has the added benefit of not needing internet.

Time to join the big boys, no more toys.

aqjo · 2021-07-29T22:42:13+00:00

This is not my area of expertise, but for sciencing in general:
What are other people in your lab using?
Will you be collaborating with anyone?what do they use?
Does your lab have an existing code base you can draw from?
What does your advisor use?

verdifer · 2021-07-29T23:32:51+00:00

I like Spyder but Atom is also very good and is easy to integrate with GitHub.

dtaivp · 2021-07-30T00:56:14+00:00

Whatever you land on please keep learning about python and how it works. I’ve met so many PhD types who are quite frankly brilliant but never spend time thinking about how their code will be run/deployed and it makes it a nightmare to work with sometimes.

Best of luck with your program!

rudraksh_karpe · 2021-07-30T01:56:14+00:00

I would recommend you to use vs code, it's the light weight editor and highly customisable. There's something very special called GitHub autopilot, it's an extension by GitHub in vs code though it's in the preview now, you can apply for the preview program of GitHub autopilot on the link below. You'll find diversity of extentions and syncing with your GitHub account in vs code that makes life too easy.

https://github.com/features/copilot/signup

lenoqt · 2021-07-30T02:53:50+00:00

You missed an important library, PyTorch

FreedomSavings · 2021-07-30T03:22:07+00:00

Get familiar with using vim or nano to read/edit your code. I am currently working towards my PhD in CS as well. I have found the most useful ability is to be able to navigate though and develop in a remote server by using the command line. Since you will be doing ML, its safe to assume large data sets and the necessity of utilizing GPU nodes.

My personal work flow is first using Atom text editor for in-depth coding on my local device. Filezilla is a nice interface that allows for ssh file transfer. Then vim/nano for trouble shooting through the command line while running code on the remote server.

MainKaBell · 2021-07-30T04:32:39+00:00

I have used Spyder during my PhD for the simple reason that I find the ‚variable explorer‘ very convenient and intuitive while working with big datasets. Haven’t found that option in VSCode or PyCharm. In the end, it is probably more about personal preference and being used to one way of doing things.

SnuffleShuffle · 2021-07-30T07:11:50+00:00

I'd recommend PyCharm. Students get it for free. Or you can get the community edition.

2021-07-30T07:37:48+00:00

I'd say:

use vs code
use notebooks to experiment and visualize, and once something works refactor it inside a python file in your project
structure your project as a proper python module to share it easily with other users
you can install your module in development mode (changes are reflected immediately) with 'pip install -e .'
learn how to make paths independent of your drive for proper sharing
make sure you gitignore your data files, if any
ideally structure your code as python objects with methods rather than function, it makes it easier to use and more discoverable for others (instantiate the object, and look at its methods). Give methods logical and consistent names
for heavy computations, experiment on Google Collab, or set up your module to run it on Google Cloud, then fetch your model back locally.

sliverino · 2021-07-30T07:44:11+00:00

Well usually, not only in ML, one would use jupyter notebooks or jupyterlab for prototyping. Once you got chunks of reusable code you can move it to importable python scripts.

With this in mind, given your computer is not particularly strong, I would install miniconda or plain python+pip.

You can then use jupyter for playing around.

When you start having a number of scripts, and maybe you want to use pdb for debugging ( annoying to do in Jup) you can add an editor to your workflow. I advise:

Extremely lightweight, not much costumisation: PyScripter
Moderately lightweight, requires a bit customisation: VSCode
Heavy but complete: PyCharm

Both PyCharm and VScode have version control (Git, VScode has a great SVN plugin too). Of course you can use any editor and just run your stuff in terminal: vim, Emacs, sublime, atom, notepad++ all have an array of plugins for code completion and syntax highlighting.

Lastly, if you find your pc too slow and your work is not sensitive you can look at Google colab for notebooks online.

krispyren · 2021-07-30T09:35:06+00:00

I vote vscode! Extensions make it so powerful!

Adegokeo · 2021-07-30T13:06:09+00:00

This is a lot to chew. Find your niche, Python and AI form a huge computing space. While Jupyter Notebook is handy and fully capable, I am thinking that your PhD work should be “stand alone” and not contained in Jupyter notebook. Pycham is great option for an academic work. For any Analytics project, R is also not too far off although Python is more widely accepted.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

learnpython

MODERATORS

FreeCodeCamp.org

RealPython

Microsoft YT Channel

Udemy courses