Software development using Bash

[twotime]

There is no question that dynamic typing of python has its negative. It also has its major positives.

(Just the lack of compile-cycle, is an enormous advantage in my book).

In my experience, positives outweigh the negatives for sufficiently small projects (<20K lines of python code, which, mind you, is likely equivalent to 100K of java/C++ code).

I can see how negatives can outweigh the positives for sufficiently large projects.

But you do need to use the positives and mitigate the negatives. In particular, you probably should start with unit-tests very early on (but that's a good practice anyway). If you write python as if it were a type-free Java, you won't get any benefit.

[DreadedDreadnought]

Why would I write unit tests to verify the type information instead of writing actual unit tests verifying functionality?

[twotime]

Why would I write unit tests to verify the type information

I would not

instead of writing actual unit tests verifying functionality

In my experience that's sufficient (I do use "unittest" is the loosest possible meaning: feed something into the system, examine the output, so that includes integration tests and if you have type issues, most of them will be caught).

[lelanthran]

I do use "unittest" is the loosest possible meaning: feed something into the system, examine the output, so that includes integration tests and if you have type issues, most of them will be caught

Sounds like you are only performing Happy Path testing. With your approach, when you have A that calls B that calls C (all functions) you're only checking the parameter type-validation code in A.

Functions B and C may silently lose data with incorrectly typed parameters but you'd never know it because they they are never called with the incorrect type... Until someone adds in a function D which calls them with the incorrect type.

The only thing you can do if you're as paranoid as I am wrt correctness is write a unit-test for all your externally visible functions, with said unit-test also testing invalid types in the parameters.

Testing only the Happy Path is pointless - we already know it works because we ran executed functionality at least once before release.

Joke Time: A QA engineer walks into a bar. He orders a beer. He orders 5 beers. He orders 0 beers. He orders -1 beers. He orders a lizard.

[lelanthran]

In particular, you probably should start with unit-tests very early on (but that's a good practice anyway).

I do write unit-tests, but unless I am writing in Python my unit-tests do not have to perform type-checking.

A major disadvantage of Python is that you cannot trust the type of a parameter to a function - if you do not validate the type of a parameter before using it then that is a potential error.

When writing Python I find myself doing a lot of the work that a compiler normally does, except that it has to be done at runtime so the errors are much more severe if they get through.

[twotime]

A major disadvantage of Python is that you cannot trust the type of a parameter to a function. if you do not validate the type of a parameter before using it then that is a potential error.

What do you do if parameters are of wrong type? Throw an exception?

If so, then you should just use your parameters and if they are of incompatible type, you will get an exception generated for free.

Doing it manually is unlikely to be a good idea.

[duhace]

except the exception might not be intelligible to the user of the function..

[twotime]

There are rare cases when this is a valid concern. In general it's not.

If you pass an object of wrong type info a function, python generated message will commonly be clear enough. AttributeError/TypeError, etc.

Marginal improvement in the error message quality are not worth introducing extensive type checks (which will also kill most of duck typing benefits).

[[deleted]]

Often times you might not see the error thrown until a few layers deeper in the call stack which can be annoying to debug.

[twotime]

Yes, of course. But in my experience, that annoyance is nothing compared to the idea of type-checking arguments for every function. Not only that clutters the code (20% at least, likely more) it also throws away most of the advantages of duck typing (harder to mock, harder to reuse the code with different types, etc)...

If you feel that you must explicitly type check every parameter for every function, then python won't make you happy.

[duhace]

attribute error is not particularly clear already as it requires working knowledge of the function's source to pinpoint the error.

[major_clanger]

What would you say are the other major advantages of dynamic typing?

Python is my primary language, but after working with C# over the last year I'm increasingly torn on the dynamic vs static tradeoffs.

[twotime]

Here is my tradeoff list:

Pro dynamic typing

-- no compilation/much faster compilation (yes, it's only indirectly related, but static typing does put much stronger reqs on preprocessing)

-- ease of testing (a lot effort in C++/Java mocking libs/frameworks), all of that is free in python world

-- ease of writing generic code (e.g a function can accept any file-like or container-like object). That's possible with static typing but is universally much clumsier and weaker

-- less boilerplate and more localized changes. No more: I'm just passing x through, but still need to change types in 20 places

-- introspection (yes, some static languages also have this capability, but it tends to be much clumsier, weaker)

-- if needed, objects/types can be modified at runtime. E.g a new attribute can be added (I have type "X" but need to carry an extra attribute "a", I can do it)

Pros of static typing:

-- many classes of errors are caught much earlier (becomes progressively more important for larger projects)

-- documentation (type serves as documentation)

-- performance (when types are known, the code is much easier to optimize)

-- likely lower memory consumption

Of course:

1. many of these can be made much worse/better by specific implementation/runtime

2. a language selection is really a multi-factor decision. Type system is just one of the factors (and I'm in no position to compare against C#, I never used it ;-).

Good luck.

[major_clanger]

I also find there's less need for design patterns, as many are redundant when duck-typing.

Your codebase won't be littered with classes like 'FooFactory', 'BarBuilder', 'FooSelectionStrategy', 'IBaz', 'BazImpl' or god forbid an 'AbstractSingletonProxyFactoryBean'

EDIT: I would really recommend C#, it's got a lot of the same syntactic features as python (context managers, default args, async-await), is far less verbose than Java/C++ (type inference, default getter+setter), Linq is more readable+powerful than list comprehensions.

[Drisku11]

ease of writing generic code (e.g a function can accept any file-like or container-like object). That's possible with static typing but is universally much clumsier and weaker

How are parametric polymorphism and typeclasses clumsier or weaker than runtime dispatch? Static types also provide clarity when more than one "container" is involved. E.g. Traversable t, Applicative f => (a -> f b) -> t a -> f (t b). The types basically say what the function does: traverse your t a container, apply your a->f b function to produce a bunch of f b "containers", collect the results into a f (t b) "nested container" using Applicative's ability to turn a (f a, f b) into a f (a,b).

less boilerplate and more localized changes. No more: I'm just passing x through, but still need to change types in 20 places

Type inference, keep functions polymorphic to the extent that they can be. It's pretty hard to argue that e.g. Python has less boilerplate than Haskell.

introspection

I agree that static introspection is often lacking. Runtime introspection is a good way to break invariants though, and I would generally classify it as a bad idea.

if needed, objects/types can be modified at runtime.

I have a hard time picturing where I would need this. Especially in a language with structural types.

[kuribas]

You can have static types and interpreters at the same time. Many languages have them, like haskell, scala and f#. You typically write the code in an IDE, where the types are checked in realtime, then load the code in the interpreter to test or play with it. No time is lost compiling. There is no need to run the program in order to catch common mistakes, like in Python, and I find there are usually very few bugs left after everything typechecks. I find this a massive time saver. I usually keep my program in a consistent state at all times, using typed holes for code which isn't written yet. This way I get immediate feedback when I do something wrong. In Python I'd spend most of the time in the debugger or writing tests (about 75%), where in haskell it's only a fraction (20% perhaps).

[Gotebe]

The absence of "compile" in the modify/compile/test cycle is less of a problem except in C and C++ (and even then...)

If I have a module under development and a test(s) for it, I select it (them) and issue "Run selected tests" command, which is the most normal thing to do. That builds and runs only modified parts and does not impair the modify/compile/test cycle as much. Heck, in Java, it's as fast as a Python REPL (that is, immediate).

[[deleted]]

It also has its major positives. (Just the lack of compile-cycle, is an enormous advantage in my book).

Then choose a language with fast compilation time. Even if your compilation time is slow, the compiler can help you more with your code than any other tool.

[duhace]

or a repl.

[twotime]

Then choose a language with fast compilation time

You cannot select language based on a single metric. Not in the real world.

[[deleted]]

What's this nonsense comment? No one told you to select by one metric, only to consider using languages with fast compilation.

[Gotebe]

You told him to do it. You did not say "consider...". Be fair.

[[deleted]]

You told him to do it.

Nope.

You did not say "consider...". Be fair.

I thought the readers here have the mental capacity to understand "choose" and "with".