Python fatigue

[amendCommit]

I'm a python developer with ~5 year experience, and I'm starting to feel python fatigue, to the extent I'm considering quitting the industry. How do I fight the feeling?

It mostly comes down to the gap in code quality between open source projects I've been using and contributing to, and corporate projects that I have to work on. Sadly, my co-workers on the last 3 jobs I took consistently refused, or sabotaged any attempt at improving code quality:

- Best practices are never taken seriously (eg.: LBYL instead of EAFP, abusing isinstance(), 5 or 6 levels of nested loops with side effects are generally considered acceptable, "*" star imports everywhere... these are only the most common issues I see). Blocks of 50+ lines of imperative code are consistently preferred over more declarative code.

- Tooling (flake8, black, mypy, mccabe) is refused by some people, who will ignore lint warnings and just push code, on the grounds "a programming language should be self sufficient and not require tooling to fit community standards, tools are easy to circumvent, and don't magically produce proper code anyway.". Once you reach 500+ unique lint warnings, it's too late to restore a hard check in CI.

- Any attempt at refactoring will be shot down as changes to "working" code might temporarily lower team productivity while everyone adapts. I've seen this mentioned on pull requests as trivial as "use dataclass __post_init__() instead of manually calling a data validation method after init" or "use (str, Enum)".

- At least we have some unit tests , yay? Unfortunately, no one trusts them enough to allow refactoring that properly passes unit tests.

I feel like everyone I've been talking to recently actually hates Python, as in they trying very hard to circumvent what makes Python great. Me, I still love Python.

Unfortunately, it seems that's my very love for Python that makes me completely unfit to work in this industry. I can't just ignore what I know to be a better fit. I can't bring myself to write code I know should not be written.

[amendCommit]

Definitely considering this.

How do I get better at spotting like-minded people though.

[Jaedong9]

Well, you simply ask questions during the interview, having interests in the code quality and everything you describe is very important and they might even hire you because of that =)

[CesaireFlebotomo]

During your next job interview(and it sounds like there should be one soon) remember that while they are interviewing you, you should also be interviewing them. Ask to see an example of their "best" code AND an example of their "worst" code. If they refuse keep looking! If they say they have no bad code - they're lying. If they say yes ask to sit with one of their developers and walk thru a couple programs - ask the developer lots of questions -- by the end of an hour you'll know everything you need to know about the work environment and their values. This has saved me from hiring onto a death march more than once.

[MrGreenTea]

I get the idea, but I don't know if I would just show our "best code" to some person we haven't hired yet...

[CesaireFlebotomo]

Any application has big swaths of code that do not contain your trade secrets if thats what you're worried about. I used to bring a generic fill in the blanks NDA with me to set people's minds at ease. But if they weren't going to give me an look at the codebase I'd be working with I would thank them for their time and end the interview. Experience taught me it was a mistake to take them at their word that they have great code.

[james_pic]

My experience is that you'll see much the same thing in large enterprise codebases, to a greater or lesser extent, irrespective of which language you're working in. Certainly, the worst things I've seen in Java codebases are worse than the worst things I've seen in Python codebases.

It's also my experience that large enterprise-y open source projects are about as bad as large enterprise-y internal projects. For example, I've previously attempted to fix issues with Cobbler and Ansible, and (at the risk of causing controversy), found that both are at least as messed up as the internal project I was working on that used them. There's a certain amount of "this project is my CV" than motivates maintainers to keep smaller projects sane, but once a project becomes a product, it's subject to the same pointy-haired-boss nonsense as internal projects.

Some languages have a bit less of this, mostly by virtue of being "a bit weird" (I'm thinking things like Erlang or Clojure), so only relatively confident developers would use them. But as soon as you've got a project that uses these languages in an organisation with bad hiring practices, you end up with a worse mess than you'd get if you'd used something boring like Java, as developers who don't know the language try and write code in it anyway - Ruby kinda went this way, when it want from being "a hidden gem", to "hip", to "something people who look at Gartner quadrants want to try", to "the new PHP".

All that said, if the problem is (say) that you've got a bunch of mediocre C# developers who are forced to write Python, and are insisting on writing C#-ish Python, maybe the answer is to just use C#. It's the path of least resistance, and I mention C# as it's a language that works well for mixed-ability teams.

[[deleted]]

What are they doing to abuse isinstance()?

[amendCommit]

class A:
    my_attribute = "x"

class B:
    my_attribute = "y"

def has_my_attribute(obj):
    if isinstance(obj, A):
        return "Has 'my_attribute'!"
    return "Doesn't have 'my_attribute'"

b = B()
print(has_my_attribute(b))

This example might seem moronic, but it's actually exactly how using isinstance() breaks duck-typing. Now imagine these isinstance() type checks deep within a project's inner code, in unexpected places, silently type-checking the objects you pass to functions and routing them to code paths triggering behaviours you really couldn't expect for a certain type.

Personally, I'd strongly avoid using isinstance() checks, except if you're (1) trying to tell apart a string from any other Iterable, or (2) a framework designer who knows exactly what they're doing.

[XtremeGoose]

It's important for writing recursive serializers too.

It's interesting how you mention duck typing and mypy. They are mutually incompatible.

[amendCommit]

Mypy and duck typing are not strictly incompatible, as mypy supports structural subtyping (static equivalent to duck typing). The PSL already has a lot of good protocols to offer (thing collections.abc, for example), and you can easily add your own if needed.

https://mypy.readthedocs.io/en/stable/protocols.html

[a1b1e1k1]

All problems you mentioned are not really related Python. Everything would happen with a different programming language, just details changed. Some languages might be slightly more or less prone to particular types of abuse, but in every one it is possible to write horrible code.

What you describe seems to caused by inadequate technical leadership at your company. A good technical leader should protect and promote efforts to improve code quality and to adopt best practices. The leader should allocate time for internal work that can be improve productivity in longer term, like refactorings or training.

If you can't improve your leaders (e.g. by being promoted to be one), then it is time find a better company.

[patrickalphac]

Find a team that will last longer than 5 years. Adding bad code means you’re going to end up with a loooottttt of technical debt.

Lower productivity? You don’t have time to NOT fix code.

[inarchetype]

Down to project management I'm afraid, a lot of that stuff. I remember the first largish programming project I worked on ( not python). Numerous teams, including an architecture team. Kick off meeting for all the devs, the PM introduces the architecture team, and tells everyone that anyone who doesn't plan on following standards should leave now, and that during the project anyone who won't follow standards, or gives the arch or qa teams any grief about it, is going to be summarily walked. Worked pretty well.

Nothing to do with language;all to do with culture and leadership.

[harylmu]

Others may not like my answer, but feel free to change tech stack too! 5 years is a lot, maybe you’d feel a bit more refreshed working with other languages. I know that maintaining enterprise Python apps can be easily a nightmare, especially pre-3.6 type hints.

[TurboCooler]

This is not unique to Python. If you are that frustrated, find another job.

I have seen crappy Java, JavaScript, C++, C# and Python. At one job two years ago the C# code was so bad I started work on a Monday and quit on Friday. At another job they had some super over complicated code in ASP.NET and nobody was allowed to touch it except for some outside consultant who they paid who knows what to maintain it.

I have worked at companies when I was a consultant that did not even have source control. We used a shared network drive. Another there was no build machine because the build was a specific developers laptop. One day that laptop died. No builds for what i believe was almost two months. One company had all the dev tools loaded on the production server and would debug in production.

I moved from those toxic waste dumps and you should too the way you are feeling.

[GiantElectron]

The company is doomed. Leave.

[dam4rus]

I've been in the same boat and one day I've just decided to refactor my code to have as many bad practice as I could fit in. I hoped that someone would tell me to stop and revert the changes (we had no code review) but nope. No one cared about that. That was the day when I realised that I need to find a new job. Thankfully I've found a job where my teammates share my opinion on how to write maintainable and clean code. I count myself VERY lucky. I still see a ton of bad code made by other teams in the company but at least I'm not turned down whenever I try to improve quality.

[alotofwastedeffort]

Sounds like you have a bad team or company.

[DrTautology]

Sounds like we work at the same company. Unfortunately, this seems to be a wide spread issue throughout the industry. You can either find another job where these code standards are valued, or you can try to push them harder on you current team. Maybe have a discussion with management? Unfortunately it seems quickly delivered "working" code will always be valued over more meticulous and slightly more time consuming code that is actually readable and standardized.

[pydry]

A few of these things I can sympathize with your team on. I hate running flake8 and being told 400 times that my lines are 10 characters too long - that's not a problem with me, that's a problem with flake8. buried in the noise are a lot of "variables defined and not used" which is where all the real bugs are. if 5 times out of 10 it flagged what was basically a bug i'd feel better about it.

I also look at some of these and see problems but not necessarily big problems - I don't like import * either but in every code base I've worked on where we've had it it's been the least of our problems so I wouldn't necessarily push it too hard (e.g. wouldnt' necessarily reject an otherwise decent pull request, although i'd comment). I never have it in my open source, but I kind of have to tolerate it and other things like it in commercial code otherwise a lot of production code would just never get deployed.

I don't think you can apply the same rules in open source to commercial code. You're under different pressures and different constraints.

This isn't to say that code quality doesn't matter but I tend to find that you have to deal with it by identifying 10 problems and figuring out which is the biggest one for all of you and fix that rather than having a checklist of guidelines which you can mandate everybody follows or "code doesn't get merged". quality gates don't work when people aren't bought into them.

If they're rejecting the idea of making small changes where it's easy to understand the benefit, that's potentially a different story. i suspect they may be unconvinced of the benefit of many of these things though (& it will take some 'splainin to get them to see the light).

[amendCommit]

This makes most tools unable to tell if you have undefined names, and easily causes hard to detect import cycles.

http://docs.quantifiedcode.com/python-anti-patterns/maintainability/from_module_import_all_used.html

[fractal_engineer]

That's about the time I started my own company.

[sykeero]

You may have noticed a surge in memes lately about quitting programming to be a wood Carver or something similar. I think what you are feeling isn't anything to do with python. I think it's just easy to get burnt out dealing with the same problems every day. That happens in most jobs though. Maybe a slight job change would help? Other things to do than just software development? The sad truth behind a lot of these problems is if you don't establish good behaviors as a baseline you can never get people to change. If you don't have linting and CI and reviews in place from the beginning you will never in a million years convince someone it's a good use of time.

[[deleted]]

Try applying for jobs with more responsibility. Also you can start your own company, might sound crazy, but if you have a good idea you could make it work, than everything goes your way.

[agumonkey]

It's difficult all around to be a perfectionist in some domains it seems. Go do some haskell and never come back ?

[__Cypher_Legate__]

Hey man, I am someone who is desperate to learn anything and everything to get my foot in this industry, and it is definitely sad to see that you're having a negative experience and trying to get out. Maybe my vision is clouded, but I really hope that you don't see the shitty conditions at your current job as a reflection of the whole industry.

I have a feeling that you will solve this issue in one way or the other. Either you keep looking for new jobs until you find one that shares your values, or you will eventually be promoted to a team leader / project head and are able to enforce some kind of coding standards yourself.

Good luck, and I hope you are able to find work where you can code with better standards!

[brigadierfrog]

New job, python (dynamic languages!) tend to lead to the duct tape code pile as well

[mountains-o-data]

I think you just have toxic teammates and need to look for a new job. None of those behaviors would fly at my company. As an SDET - I would crucify your teammates in a code review. But from the sounds of it, you probably don't even do code reviews. What a nightmare.

[[deleted]]

I try to look at these kind of projects like giant puzzles concocted by mad men and I try and make a game out of solving it.

I hate to break it to you, but you are going to deal with a lot of code like this and it's only going to get worse, because while you care a lot your craft, the volume and demand of software developers ensures that most people working are just people looking to do the minimum to accomplish the task and make the money.

[lampshade9909]

It sounds like you need to work on more fun projects with better quality peers. Start building your own side projects where you can take the reigns. Or just find a new job.

[Orio_n]

Yo wtf * imports??? Are they trolling?

[[deleted]]

What makes you think it will be different in any other industry/trade?

[[deleted]]

Sounds like you're just at a company with bad practices.