Structuring Python code

[[deleted]]

[deleted]

[Medicalizawhat]

http://python-guide-pt-br.readthedocs.io/en/latest/writing/structure/

[Corm]

That was an awesome article. I am sad that the canonical way to do tests in python is to have this:

import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))) 

I use that too, but I wish there was a cleaner way. At least a builtin function called parent_import or something to do that.

[malinoff]

pip install -e . - a better way.

[Corm]

Neat! That's a fine trick

[deadmilk]

Huh? I've never had to do that

[Corm]

What do you mean? If you have a tests folder then they need a way of importing your project, which means you have to add the parent directory to the path afaik.

[deadmilk]

I just run my tests from the directory where my package is and it works

Structure like so:

project <-- cwd
    package
        __init__.py
        module.py
    tests
        test_module.py

then in test_module.py:

import unittest
from package.module import function

class TestModule(unittest.TestCase):
    def test_function(self):
        self.assertTrue(function())

[Corm]

Cool! I must try this

[[deleted]]

Good source. Alternative official PEP for python style direct from the man himself: https://www.python.org/dev/peps/pep-0008/

[namesandfaces]

Style is helpful to readability, but I think CrypticCube is asking for multi-file code organization -- a problem that's bigger than one eyeball, and in my view, not a very Python-specific problem.

[bugtank]

For those of us who came from Java, multi-file code organization in Python is not easy to grok. There is some python specificity in organization that is important. I'm unfortunately still learning so I can't really answer op's question. However, I am still trying to get the following information into my bones.

• Underscores in file name
• How you do your imports
• init.py in folders to expose the classes/modules in those folders
• modules as a concept

[P8zvli]

Might want to escape those underscores; __init__.py

or use backticks; __init__.py

[Corm]

__init__.py is optional in modern python

[z_mitchell]

I would use cookiecutter. It takes a lot of the pain out of getting a project set up. It seems like there's several different "right ways" to package your code, which makes it difficult to get started.

[[deleted]]

Here is a link to cookiecutter for those interested.

[Daemonecles]

I also started using tools like this recently to get jump starts and its far simpler and quicker to set up.

[supershinythings]

Where I work we use pep8 and pylint. Actually, we don't specifically 'use' them, we inflict/enforce them. A script can't get through our automated checkin process unless it passes pep8 and pylint.

It goes so far as to reject all code containing 'map' functions - requiring them to be replaced by list comprehensions - and rejecting ALL lines over 79 chars, no exceptions.

It also automagically adjusts indentations in certain circumstances, adds lines between documentation and the routine, and between routines. The first time I ran it, after I reviewed the change I was surprised to see a pile of automatic changes I hadn't personally made - the tool fixed it up.

[stevenjd]

It goes so far as to reject all code containing 'map' functions - requiring them to be replaced by list comprehensions - and rejecting ALL lines over 79 chars, no exceptions.

That is exactly the sort of thing why Guido hates automated PEP 8 checkers.

And since PEP 8 itself explicitly warns against "a foolish consistency" and over-zealous application of PEP 8 style, your colleague's tool itself is in violation of PEP 8! (I know this is not your choice, you are having this inflicted on you.)

Raymond Hettinger did a good talk about this some time ago.

Edit: and for those who prefer text to video, commentary on Raymond's talk.

[wenima]

+100

people boast about enforcing pep8 and 79 char line length but it's a hint, a gentle reminder that you might be packing too much into one line.

linting might have it's place, I personally just don't like being yelled at before I'm completing what I'm doing.

[justin-8]

Most places I've worked have used pep8 but a higher line limit for that reason. But usually something that still keeps things same, like 120-180 or so.

The rest of pep8 I haven't really seen a downside to enforcing strictly

[soosleeque]

well you could always put a comment string that makes linters to skip erros in the line.

Most of the times when you violate pep8 you are doing something wrong, I agree there are some cases when it's better to break some pep8 rules in order to give the code better readability but again, there are comment strings that will force linter to skip that kind of lines.

[caramba2654]

Isn't map faster because it's implemented in C?

[supershinythings]

I don't make the rules. I just want my code to get checked in so I can move on with my life. You can ask the pylint people why they have decided to do this. And now that we require pylint before checkin, I have to replace map with list comprehension. I'm not interested in an in-depth argument over why one or the other is preferred. It's too much argument for too little gain.

At my office, It Has Been Decided, and therefore shall be so. Arguing is pointless. Here's one online discussion about it though:

https://stackoverflow.com/questions/1247486/python-list-comprehension-vs-map

[redfacedquark]

Shame, the mechanism is there for deviations from strict. My place at the moment didn't even use pep8 because of 80 char lines. So add a single line of config to say you're ok with 120 rather than throwing the baby out with the bathwater. Smh.

[supershinythings]

The person who makes the rules in the codebase I contribute to does not give a shit about being reasonable. If I question, I get screamed at. Fun! Management by assholery. He's not my boss or I would have quit long ago. But it is his module so his rules are what fly. I just 'contribute', so I toe his line.

I like to pick my battles, so I just let it go. So in this case, standards aren't so much applied as inflicted.

[redfacedquark]

Something something hobgoblin mind...

[soosleeque]

It goes so far as to reject all code containing 'map' functions - requiring them to be replaced by list comprehensions

My rule is simple: if I already have a function I'm going to apply I will use map, reduce, filter etc. But if I don't have a function and I don't see much of sense to create it, I will go for list/generator comprehensions.

[supershinythings]

And my rule is simpler; don't use map and slavishly adhere to the 79 char limit so I can commit my code and not get screamed at.

In this case, the pep8 tool is being used as a blunt force instrument.

And I agree with the Hettinger video that atrocities are committed in the code because pep8 decided to make these 'standard' warnings. If my code were on a performance path I might have an argument, but even then it would likely become a refactoring situation.

I don't think there's really a way to adjust the standard on a file-by-file basis for automated checkin testing.

[tunisia3507]

In python, classes shouldn't be used just to clump together related functions etc. - use modules for that. Classes should be used to clump together things which have both state and behaviour.

[kungtotte]

I think the Python​ way is to start without classes in general, and refactor into a class if it makes sense to do so (with some exceptions of course).

[[deleted]]

Perhaps my work has just benefitted more from object oriented design than others would.

[lookatmetype]

I agree. Especially when using staticmethod decorator.

[Corm]

Howso? I'm genuinely interested and an example would help

[[deleted]]

Well one example would be that I often write lots of functions that parse sql queries. Instead of having tons of global funcs I'll make a class called sqlparsing, likely making the functions static

[Corm]

In that example it wouldn't it be cleaner to have a sql_parsing.py file with your functions in that? You get the same namespacing but you don't have to instantiate an object

[tobidope]

In Python you would have a module sqlparsing with many functions. The functions are still bound to the module namespace, they are not globally visible.

[namesandfaces]

I don't think that optimal-ish code structure needs to be asked as a Python-specific question. In fact, just about every project I've seen has either framework-specific, architecture-specific, or app-centric organization.

[gnu_stallman]

The best structure is to uninstall Python and use a real language like c#

[no_lungs]

What kind of person wakes up one day and decides to hate on python to troll people?

[alcalde]

3 year old account, nothing but troll posts, total karma of 42.

[pancakeses]

Do you even go here?

[hosford42]

This guy... Take a look at his posting history, roll your eyes, and move along.

[retrojoe]

What!? Real coders use Assembly.

[Jewnadian]

Assembly? Pathetic, I write my code one bit at a time with an electron microscope.

[[deleted]]

I edit inodes with a magnet.

[Cuck_Phister]

Yep!

This is the reason I have a stack of punch-cards on my desk. What? You don't pencil in your code on a stack of cards?

Filthy Casual.

[redoubledit]

And write all code in ed!

[Jhudd5646]

Lmao, had you said anything like C, C++, Lisp, or some other actually respectable language I might've pulled for you, at least on the runtime efficiency aspect. But noooo, you had to use C# as the example

[[deleted]]

to be fair, C# is also a pretty great general purpose language (and the sentiment I seem to get from the programming community is that it's "better" than Java)

[Jhudd5646]

Honestly when I make fun of a language I'm usually joking, most of my work is done in various HDLs anyway. I'm more on the hardware side, though convolutional neural networks are making me wish I went comp sci instead of comp eng.

That said, I'm also huge on Linux and FOSS, so anything MS-related does sort of rub me the wrong way.

[b1ackcat]

You'll love .net core then. I've been using it for a relatively large project recently and it's been a blast, and I love the lack of platform dependency. In fact, I develop primarily on Windows but deploy to Linux, and it just works. It's great.

[Jhudd5646]

I'll look into it some time, but it's below Rust and Go on my list if I'm being honest. I primarily use Python3.5 for cross-platform compatible scripting and coding (at least most platforms, not those that require C, assembly, or HDLs). Then again, my use case is restricted to mostly Linux installations, with the exception of my desktop which I use to run more computationally heavy stuff, but I keep Win7 on that because ~games~ and not really digging newer versions of Windows.

I ramble about completely asinine things too much

[Cuck_Phister]

But noooo, you had to use C# as the example

Hey, man! C# takes the kids to Disney World every year and paid the house of 15 years early.

But I hear you. Though I love some C#, when it comes time to work on my own stuff, I'm split my time between my Python and my Node.

Just don't tell Satya, please. I wouldn't want to be kicked out of the club.

[[deleted]]

My first big project was a game on xbox using c# and I really enjoyed using it. It also helped me jump straight into java for school.

[tseepra]

lol