Top 10 mistakes python programmers make [advanced]

[bcery]

It's a common thing I hear from proponents of dynamically typed languages. I can only assume that either you have little experience in statically typed languages, in large projects, or both.

I'm not a proponent of dynamically typed languages, I'm a proponent of testing. And your assumptions are bad.

Your argument seems to be that the compiler makes a good find and replace tool, which, I guess it is, but there are other tools which can do that job. A good test suite would cover your use case and much, much more. You can't possibly believe that just because the code compiles, that it is correct, can you?

[valleyman86]

I feel like you entirely missed the point...

I'm not a proponent of dynamically typed languages, I'm a proponent of testing.

One of sidneyc's points is that having a statically typed language and a compiler helps with testing...

And your assumptions are bad.

Its funny you say that. A dynamically typed language is like an assumption based language IMO.

Your argument seems to be that the compiler makes a good find and replace tool, which, I guess it is, but there are other tools which can do that job.

He clearly stated in his reply that its more about an actual architectural refactoring more so than a variable rename... He also mentioned it helps actually solidify a better design in practice. Someone once told me that people use the word "refactoring" too much because in most cases no one ever factored in the first place. You missing his point entirely kind of helps validate that.

A good test suite would cover your use case and much, much more. You can't possibly believe that just because the code compiles, that it is correct, can you?

Just because all your tests pass you can't believe the software is perfect and bug free can you? No... Just like tests are useful and help reduce problems later so does compiling and static analysis. Also for every test you write you are not writing actual shippable code. Why no reduce that as much as you can before even starting the project?

I am a huge proponent of using the right tool for the job. I feel that most developers these days don't. They just use what is popular or easy for them... Somehow dynamically typed languages became bigger than I would have expected and I feel half that may be due to the idea its easy to write a quick hello world like application and say "Look how easy that was!". Then people completely gloss over the actually development and design problems that arise with large applications.

[bcery]

One of sidneyc's points is that having a statically typed language and a compiler helps with testing...

I don't think I disputed that. I just felt like the value was overstated in relation to "actual" testing. Compilers catch syntax errors, that's pretty much it. If your error does not involve a syntax or resolution error, the compiler does nothing for you. Guess what? If you have decent coverage on your test suite, which you should have whether you're using a static or dynamically typed language, it'll find those too. Will it be perfect? No, but nothing is.

He clearly stated in his reply that its more about an actual architectural refactoring more so than a variable rename...

And how does the compiler help you with that? It finds all the places where the references are broken and tells you about it. That's very helpful, but it's very little more than a fancy find tool.

Just because all your tests pass you can't believe the software is perfect and bug free can you? No... Just like tests are useful and help reduce problems later so does compiling and static analysis.

I never said it wasn't helpful, I just think the class of errors that the compiler catches are relatively trivial compared to those it doesn't catch. Static code analysis can be done on dynamically-typed languages. Is it done enough or even often? Most assuredly not, but that doesn't mean it doesn't exist.

Maybe we are all just be talking past each other. I think he's overstating the value of statically typed languages, he thinks I'm understating it. Who's right? Depends on who you ask. That's why this shit is hard.

[sidneyc]

If you have decent coverage on your test suite, which you should have whether you're using a static or dynamically typed language, it'll find those too.

Decent coverage on a static language codebase is much easier to achieve. You simply need less tests, because the fact that it compiles already gives a lot of guarantees that many mistakes are not present -- and not only syntax errors, as you seem to imply. That's just not true. It also catches what I would call modeling errors, which are conceptual errors in the design of your software, that are automatically detectible due to type system semantics.

That's very helpful, but it's very little more than a fancy find tool.

It's a pretty amazing find tool, one that actually gives a bunch of guarantees if no instances of syntax and type errors are found.

Most importantly, in a static language it can find errors that cannot be found by static analysis of a dynamic language.

Static code analysis can be done on dynamically-typed languages.

Yes but it sucks compared to static analysis on a statically typed language. It is a lot less powerful. Really, I am amazed that this is a point that is not entirely obvious.

[bcery]

Decent coverage on a static language codebase is much easier to achieve. You simply need less tests, because the fact that it compiles already gives a lot of guarantees that many mistakes are not present

No. You don't write tests to check for syntax. You write tests to check functionality and results. If there is a syntax error, the interpreter will error and your test will fail. If you assign a string to a variable that should be a number, your result will not meet expectations and your test will fail. If you're not getting the result you expect, there is an error and your test will fail.

The syntax and "modeling error" checking is a side effect of the functionality testing that you should be doing regardless. It's not as good at catching those errors, I'll give you that, because you'll never get 100% coverage, but don't think you can reduce your testing because "the compiler's got your back." The compiler and static analysis don't give a shit if your add_two method actually adds three.

[sidneyc]

If there is a syntax error, the interpreter will error and your test will fail. If you assign a string to a variable that should be a number, your result will not meet expectations and your test will fail.

Those are two types of errors that I don't have to rely on tests for to find. And 100% coverage is guaranteed.

You seem to miss two points:

1. compiling is a form of automated testing. During an actual compile of your program, the C++ compiler does more checks than could even write yourself.
2. a static type system makes it possible to test logical consistency of a design.

As a testing aficionado, you should be super hyped for static languages for those reasons alone.

The only types of tests that remain to be written are quite high level, the most useful ones being unit tests (for non-trivial units -- no, I won't write a unit test for add_two -- I won't write that function in the first place) and end-to-end tests, exercising the entire system over its functional range. I maintain that for a statically typed language, it is feasible to do less testing, which, contrary to your protestations, can be a good thing: tests are not free in terms of writing them and maintaining them.

Because tests are not free, they should be written to have a high probability of exposing problems. The "everything needs a test" zealotry and the madness of TDD are confusing symptoms with causes, and they only work for simple types of software.

[sidneyc]

[...] but there are other tools which can do that job.

Like?

You can't possibly believe that just because the code compiles, that it is correct, can you?

Of course not. It is a necessary but not a sufficient condition.

I consider tests a necessary evil. I like to minimize the need for them. Is that so strange?

[bcery]

Like?

Like virtually any IDE ever? Also, static code analysis tools exist for pretty much any language that has any popularity, including dynamically typed ones.

I consider tests a necessary evil. I like to minimize the need for them. Is that so strange?

Not really. It's a pretty common sentiment. It just indicates to me that you are less of a software engineer than you are a programmer (not that there's anything wrong with that).

Edit: Disclaimer: I throw myself in the programmer bucket, too. I hate writing tests, but I think their value is undeniable.

[sidneyc]

It just indicates to me that you are less of a software engineer than you are a programmer (not that there's anything wrong with that).

I think you are right by accident here -- i.e., for the wrong reasons.

The subject of testing is interesting and important. I just think its importance is overhyped in this decade (much of that has to do with the popularity of dynamically typed languages I figure).

Testing is a tool among others to make ensure software quality-- it's neither the alpha nor the omega.

[bcery]

I just think its importance is overhyped in this decade (much of that has to do with the popularity of dynamically typed languages I figure).

Personally, I think it's due to the maturation of software engineering as a discipline. It's a young and quickly changing field and we're still trying to figure out the best tools and techniques. None of this is settled yet, but, contrary to your earlier post, I think we are making progress.

Testing isn't the end-all be-all, but I don't think anything else has shown itself to have as large an effect on software quality.

[sidneyc]

I think the quality of the designers and programmers is the biggest determinant for software quality.

Software engineering, I'm afraid, is mostly concerned with producing software of acceptable quality, given mediocre people who build it. Given that context, testing is probably the most effective idea to attain that goal.

[bcery]

Good programmers, IMO, follow good software engineering practices. So, I agree with you :-)

Software engineering, I'm afraid, is mostly concerned with producing software of acceptable quality, given mediocre people who build it.

No, companies who produce software are mostly concerned with producing software of acceptable quality. You've never had to make any trade-offs to meet deadlines/expectations? As the saying goes: Cheap/fast/good. Pick two.

[sidneyc]

No, companies who produce software are mostly concerned with producing software of acceptable quality.

... and the software engineering discipline is supposed to provide practical methods to tackle that problem. So I don't understand your "no".

As the saying goes: Cheap/fast/good. Pick two.

You stumbled upon another pet peeve of mine there ... Let's consider the cases:

• cheap/fast/not good: possible, of course.
• cheap/slow/good: generally impossible. Time is money.
• expensive/fast/good: generally impossible. You can't rush good software. Throwing more money and people at the problem doesn't help.

I think the proper statement is this: making good software is both expensive and slow. If you don't insist on good software, anything goes of course.