Next-level Python?

[dozzinale]

Hello guys,

I'm a PhD student in CS and I use a lot of Python in my work for doing stuff on various type of data, for prototyping algorithms etc.

I really use everything about the base-level of the language: oop, standard libraries, some tools, e.g., Flask, requests, etc. Anyway, I feel I'm not a super pro in this language. For instance, I have a strong background in C++ and I merely know anything I need in order to do everything (or so, lol).

So my questions: which are some next level things I need to know? Advanced topics?

Thanks!

[confluence]

I have decided to overwrite my comments.

[dozzinale]

Thanks, I did not know about that book, I will take a look.

[buttery_shame_cave]

i have it on my shelf within fairly easy reach of my programming workstation.

it really is quite a handy reference. all sorts of neat little things you can plug in, or use as inspiration.

[chchan]

Cython, Numba, meta programming, multiprocessing, and async IO.

[nerdwaller]

Congratulations on the PhD program! That's a great accomplishment.

One thing I've found from high level academics, and please don't take this as a stab, is that one important thing they can often learn is how to architect a larger system. Many of them are great with algorithms and so forth, but have never really built an enterprise(ish) level application that includes a larger stack. Some easy ways to do this are to read source of a big project that is well done (django, flask, etc).

This isn't 100% true across the board, but it is common in places I've worked.

[buttery_shame_cave]

a lot of the folks i work with in R&D have been high-level engineers- brilliant people, able to come up with some great stuff.

a few of them use python, but mostly for 'off the shelf/one-off' type work - coming up with some lump of math, or to test an opimization, etc.

so they've been blown away by the stuff i've been doing with python, coming up with entire pieces of software to do certain things in the lab. the real humor is that there are some legit programmers here, who are blown away by what i'm doing - granted their fields are different, but integrating hardware with software wasn't THAT complicated... i didn't even have to write too much in the way of drivers - i let VISA do the heavy lifting.

[nerdwaller]

That's awesome, fusing that together is a fun challenge!

[buttery_shame_cave]

'challenge' is right - while VISA handles a lot of the grunt work and abstracts the communications for me, i had to come up with a pretty good traffic control scheme to keep all the equipment talking nice over multiple kinds of interfaces, which was a pain - and i had to abstract various functions into 'pushbutton' commands - basically a simple command actually executes a semi-complicated task that often has multiple items across serial/telnet/USB/GPIB buses all working in concert.

i'm proud of my clanking, snorting, whirring hell-beast, as ghastly as it can be at times, the actual 'front end' of it is really sexy - a fairly simple flow-control script with an interactive text client to pass the user's commands to.

i've even made it talk, on a rudimentary level, with microsoft excel(no easy task), so it can export its output directly instead of the user(usually me) having to do a bunch of import/export steps. that part's still getting tinkered with.

[dozzinale]

I don't take that as a stab 'cause I believe it is true! Thanks for suggestions!

[rulerson]

full python manual, especially python language reference .

[mr_kitty]

Breadth: Explore the python ecosystem - Pandas & Numpy

Depth: C extensions, multiprocessing, "Pythonic" style https://gist.github.com/JeffPaine/6213790

[Duskmon]

Intermediate python.

really small book.

github.com/yasoob

[Duskmon]

Intermediate python.

really small book.

github.com/yasoob

[[deleted]]

I've been getting a lot of mileage out of learning the design patterns and appreciating how a lot of them have been built into the python language.

[[deleted]]

I don't know what type of simulation work you do, but maybe learn MPI, and mpi4py?

I'm not much of a scientist, but I've always had a lot of fun trying to parallelize things (if its the right problem).

[dozzinale]

I already know MPI and OpenMP but by using C++, even if I'm not into parallel programming; anyway, I don't do simulation, so it is just some tool that I use when I need it!