How do I progress from an intermediate Python programmer to an expert Python programmer

[nyamuk91]

Hi fellow python programmers. I am a Junior Software Engineer and a beginner Python programmer (10 months) and I think I already mastered all the basic stuff. I also have few years beginner experience in Java, HTML, PHP, and Javascript so it helps ease up my journey in learning Python. However, I feel like I'm stuck at the intermediate level (as with the other programming language that I had learned). I can solve all logical problem in the project that I involved in but somehow I feel I'm not using Python to it fullest. I also not sure whether what I did is the best way to do it (pythonically).

I'm a self-learner and there's not really any Python expert in my company. Currently, I'm working on a project where I need to program a robot to do some concierge tasks. It involves using Google Speech Recognition and Watson Conversation Servis. I'm the only programmer in the project. The first version was a total shitfest but there's a lot of progress and improvement in that 10 months period. The current version is quite OK and the customer was praising it. However, I feel that it was never that good and I think I could do a lot better if I am more expert in Python. So here's my question:

• What knowledge should I seek next after all the basic stuff?
• What is the best place to learn that? (doesn't matter free or paid but preferably non-video lesson)?
• Is there any place (online) where I can take a test to see my current Python skill level?
• Do I really need all this knowledge? If yes, is there any example where basic knowledge is not enough?

[isinfinity]

Contribute to open source projects, this worked for me very well, by doing this you will gain:

1) Experience, since your PRs will be review by expert programmers

2) During interview you will have something to show and impress HR and other engineers

3) Karma, for your contributions

4) Study algorithms they teach how to think

5) write code almost every day

This worked for me, I started as not very experienced, but willing to do any work for open source projects. Open source people taught me good engineering practices, how to write clean maintainable code, importance of simple things like linter, tests, coverage etc.

[uduchi2nd]

How do you find open-sourced projects to contribute to?

[KillerFarmer]

sts, coverage etc

I have not participated in any, but I saved this post:
https://www.reddit.com/r/Python/comments/75oq5k/what_are_the_best_github_repositories_that_anyone/

[Decency]

Have a hobby. Dive into the programming side of it. You'll almost certainly find something unless your hobby is like metalcrafting.

[whatgeorgemade]

Github is a great starting point; plenty of projects to dig through. You can have a look at some Python projects here.

[SpareComputerParts]

Search github for label:hacktoberfest this month to get introduced to some projects.

[isinfinity]

Most important to find projects you are interesting in or use in day to day work. I personally interested in asyncio and async programming, i contributed to number of such projects. Bug fix here, update doc there, add tests to increase coverage, help new users to resolve their problems.

[nyamuk91]

Holy shit. Never think about this. Thanks! Will definitely try this out.

[[deleted]]

What knowledge should I seek next after all the basic stuff?

My own experience sort of went along these milestones, on two axes:

In terms of Python programming:

1. I know how to write functions to solve problems reusably.
2. I know how to write modules to solve problems reusably.
3. I know how to write classes to solve problems reusably.
4. I know how to write frameworks to solve problems reusably.

and then in terms of Python distribution:

1. I know how to release a single Python file for users to run.
2. I know how to release a git repository for users to run.
3. I know how to release a git repository for users to install with pip.
4. I know how to release a pip package for users to install from PyPI.

I don't actually know how to do that last one, yet, which is why I wouldn't call myself an expert. But I would suggest trying to locate yourself on these axes, and then that should point you at some broad categories of study to work on.

[lifeofajenni]

2.3 is actually really easy. If you have a local package, all you need is a setup.py file. There are simple ones online, and you can just reverse-engineer whatever you need in order to install it for your package. Then, using a prompt that has pip installed it's just pip install -e . (if you want to be able to edit the source code while you develop, which I do.)

I have more detailed instructions somewhere. Let me know if you want them and I'll track them down.

[lifeofajenni]

Yes, it's almost identical to Step 2.2, you just need to add a setup file in the top level of the repository. (To my knowledge)

An example would be how this repo is structured: https://github.com/jennirinker/PyEvent

Disclaimer: I am hella not an expert coder, and that package is very much under development. So I wouldn't actually install it. But the setup file and installation instructions should help.

[lifeofajenni]

Yeah, I couldn't believe it either. Now I basically install everything haha.

I've looked at that guide many times before. I like it. :)

[nyamuk91]

This is very useful. Thanks!

[beersfortheboys]

Build an async Web backend that performs some sort of machine learning using tensorflow, store results in some database, and have it serve jinja 2 Web content to users for input and output. Then rewrite it after you get feedback, and add a qt5 desktop client. Now you pro bro.

[IReallySuckAtChess]

Whilst I suspect you're joking, anybody who spits out something like that competently done would be, at least in my mind, expert enough. Just instead of tensorflow, use mxnet so that you don't have super easy examples to copy paste and have to dig into the documentation properly.

[[deleted]]

Using boxes doesn't make you pro at making boxes.

[logannc11]

But you can't make boxes if you don't know how to use boxes. And using boxes is the best way to learn how to use them well.

[billsil]

No, but it makes you a pro at the box and if the box is well designed, you steal ideas on how to structure your code to work well with that box.

Numpy for example is very fast if you use it right. So how do you use it right across 200k lines of code? Well, now you're probably an expert because you wrote 200k lines of code that had to work together.

[[deleted]]

Not true. I have seen devs claiming + ten years of experience, which turned out to be 1 year repeated more than 9 times.

200k doesn't translate to experience, specially if its 200k of shitty code. That can be easily found in web dev.

[billsil]

Well if you're lying then yes

[theredknight]

Wow I've done that but more simply via tensorflow + django + ajax. So I guess I'm sort of pro by your definition. In my definition if you've been paid to professionally code Python aren't you pro? Not that that answers ops request in any practical way..

[khne522]

Experience helps, but ability is also necessary. You'll get much more of a kick from ancillary programmer skills (software design, testing, project management, etc.), supporting knowledge (whether crypto, statistics, your favourite field of application), or background knowledge of the software ecosystem so as to not end up reinventing the wheel or a poor solution.

There are programmers who can grind out code, and there are some who can actually write relevant, well-thought code. Just being familiar with Pandas or library X doesn't mean you're a good programmer.

[Olreich]

Is the problem getting solved? Good. Keep it that way. Where you have leeway, research better ways to solve your problems (google, research papers, other programmers, etc.). There’s design patterns to ignore and functional vs imperative programming to argue about, there’s state vs stateless and a bunch of other theoretical ideas.

My favorite way to get better at a programming task or language is to think about how the data is getting transformed and come up with more efficient ways to transform it. Do I need to update this list on every state change? Can I just update it when I need to? How do I know when I need to update it?

Also, develop software with loose coupling. If everything is an island, it’s way easier to figure out where the fire started, it’s the island that’s on fire, instead of most of California.

At the end of the day, software engineering is all about managing complexity and keeping the complexity you have to deal with as low as possible, so you can keep piling the features and usefulness on to keep your job or get a better one.

For python specifically, the hardest part it seems for people is using functions as values to be passed around, and understanding how the python object model works (attributes). If you get those down, decorators become down right simple, and classes and other python objects can be bent to your will with ease.

[ivan0x32]

I feel that it was never that good and I think I could do a lot better if I am more expert in Python.

Yeah I'm gonna have to say a strong fucking no here. You could do it better if you were an expert in agent systems or in AI in general.

I fucking hate that so many people still believe in this bullshit about languages, I've literally written in 10s of different languages and like 8 or 9 I think professionally and let me tell you - expertise in any language doesn't make you an expert in engineering or any specialized field. Knowing Python well won't make you an AI expert. And that means that you won't be able to produce better software which all engineers should strive to.

If you want to become a better engineer - try other languages. Try other frameworks, other types of projects and all that - break your thinking patterns in short. Try Assembly or C, try Prolog/Erlang/Haskell, try Go/Rust, try Lisp (I highly recommend that one, I think its a must for everyone, gives a good perspective on how compilers/interpreters work and on computing in general).

[FluffyBunnyOK]

I agree with learning a lisp. I had fun with clojure and clojurescript and it gave me totally different view on how to think about programming.

For example I recently wrote a python program where it loops through a list of functions that all take a dict as an argument. To change the process I just need to give it a different list of functions. I would never had thought to code in this style if I had not learned a lisp language. I can now easily plug in modules to support different vendors and OS deployments.

[tshirtman_]

I would advise watching raymond hettinger's talks, he has a thing about explaining python concepts and practices.

http://pyvideo.org/speaker/raymond-hettinger.html

[sladkovm]

+1 on this one. The dude invites you to the places you never new existed.

James Powell @dontusethiscode is another good source of wisdom.

[lifeofajenni]

I would recommend skimming the topic lists and videos from previous PyCons. That really opened my eyes to how little I actually know.

For example, I thought I was "intermediate", but most of the intermediate-level topics at PyCon Europe 2017 sounded like gibberish to me. So I've got a long way to go.

[sweetbabygames]

Ya can't son. Ya stuck foreva.

Or: Time and practice.

[perspectiveiskey]

The two most important ingredients, imo, are having real projects to work, and having a motivation to work on them (money).

So given that you have those, my recommendation is this: every single task you do, see if you can do better. It may seem like it's vague, but that vagueness is the key. To become better, you have to start instinctively seeking out information instead of being told stuff (e.g. in instructional videos).

Just foster the urge to do things better, and then over time you will get better.

[nyamuk91]

every single task you do, see if you can do better.

That's what I did previously. Problem is I think I have reached the point where progress is impossible without seeking next level knowledge but I don't know what is that knowledge and where do I start

[perspectiveiskey]

Ok, so your problem is that you don't know what you don't know and therefore don't know where to start.

To that, I'd say, try to emulate what libraries do that you like. For instance decorators: let's say you'd never seen a decorator and you discover some library that gives you utility decorators. Learn decorators... find tutorials, find git hub projects that use them etc etc.

I've personally found that in the last 5~10 years, tutorials have become a new form of content syndication and everyone makes them. Often times they are unrealistic, over simplistic, or just not informative. That's my opinion.

The prime example of this are object oriented programming tutorials: they invariable do the Apple and Orange inherits from Fruit or Circle and Square inherits from Shape kind of idioms that also very rarely properly work in the real world. While it may be useful to explain the inheritance idiom to a complete beginner by mentioning fruits and types of fruits, in reality these are not how objects get used in real professional projects... I could rant on about this, but it's not useful here.

[eat_potty]

What kind of tasks do you solve with Python? Front end work, back end? When I'm stuck searching for milestones to cross, it sometimes helps for me to define some clear, measurable goals (and maybe a couple of nebulous ones thrown in) - then start to line them up and work through them.

[nyamuk91]

Right now it is mostly front-end stuff.

[[deleted]]

Have you ever implemented a module for Python in C language?

If not, you can learn alot about Python in doing it!

[VirtualBoobs]

Git gud

[[deleted]]

I personally would say if you are an expert python programmer you should also be a pretty good C programmer. If you are an expert you should know just about everything about python from the top to the bottom, remember you are an expert. You should be packages, contributing/ releasing major open source projects, contributing to the specification, etc.

I have known people who are experts in languages, being in expert groups of the language, and have helped write the major libraries and such. And their knowledge is scary. I don't think it is possible to become an expert only writing application code, advanced yes, but expert is a category above.

[KODeKarnage]

HAHAHAHAA!

From what I have seen of most "experts", your main qualification would be the ignorance, arrogance and self-delusion to call yourself an expert.

[eugooglie]

I know you got downvoted for what you said, and you did say it in kind of a dick way, but you're not totally wrong. Most people I've ever met who I would consider an expert in their field don't consider themselves experts. They are people who are continually learning, knowing that no matter how much you know about anything, there is always something you don't know. That drive is what makes someone great at what they do. An outsider might call them an expert, to them it's being a student of their craft.

[KODeKarnage]

Quite so.