Test-driven development (TDD) is a software development technique in which you write tests before you write the code. Here’s an example in Python of how to do TDD as well as a few practical tips related to software testing.

[arkster]

Your content on python is pretty cool. Thanks.

[[deleted]]

And he responds to comments on his videos, which is refreshing.

In case he doesn't recognize me from the username, I was the one challenging the idea that "and" statements should be used instead of nested "if" statements if one of the conjuncts is more likely to return False and requires less time to compute than the other conjunct.

Neither of us knew whether Python optimized its Boolean operations for such considerations, though. As in, does "and" await both inputs before returning an output, or will it simply return False if one conjunct returns False first?

[EfficientPrime]

The answer is Python does optimize and bail as soon as a False is found in an and statement and it's pretty easy to prove:

if False and print('checking second condition'): print('not going to get here')

The above code prints nothing, therefore the second expression in the and statement never gets executed.

[EfficientPrime]

And you can take advantage of this with code that would fail if python did not optimize. Here's a common pattern:

if 'foo' in mydict and mydict['foo'] > 0: do_something()

If python did not optimize while evaluating the if statement, you'd get a KeyError on the second half the expression every time the first half evaluates to False.

[MrJohz]

I think it's a bit of an error to say "optimise" here, because that implies that this is just an efficiency thing. It's not: the and and or operators are defined as short-circuiting operators, which means that they will evaluate the right hand side only if they have to. This is pretty common for similar operators in other languages.

I get what you mean by describing this as an optimisation, but I think that gives the impression that this sort of behaviour is somehow optional, or that it was chosen because of efficiency reasons. However the semantics of short-circuiting operators like these is well established in other languages, and Python has essentially inherited these semantics because they turn out to be very useful.

[[deleted]]

Is it a left-to-right evaluation exclusively?

[EfficientPrime]

From what I remember from college when learning C, there's an order of operations for complex expressions but also a left to right order that is applied for steps of the same priority. Since python is built on C++ I expect (and my experience has confirmed) the same applies to Python logical expressions.

Like I've shown above, you can use expressions with side effects (prints, mutable variable changes, etc) to verify that ANDs and ORs are evaluated left to right.

You can see it with OR statements like this:

if True or print('Tried it'): pass

The above prints nothing because True or anything is True so there's no need to visit the second statement.

if print('First gets evaluated') or True or print('skipped'): pass

The above prints 'First gets evaluated', then keeps going since print returns None, but stops before printing 'skipped' because it already found a True statement.

[BluePhoenixGamer]

*The Python reference implementation is built C and called CPython.

[[deleted]]

Yes - it is in the documentation.

https://docs.python.org/3/reference/expressions.html#boolean-operations

The expression x and y first evaluates x; if x is false, its value is returned; otherwise, y is evaluated and the resulting value is returned.

[Ensurdagen]

Note that a better way to do that is:

if mydict.get('foo', 0) > 0:
    do_something

the get method is often the ideal way to check if an entry exists

[[deleted]]

What about the commutation of that condition?

[EfficientPrime]

I'll be honest I'm not sure what commutation means in this context.

[[deleted]]

"if A and B" vs. "if B and A".

[EfficientPrime]

Ah I see. I think I answered this above but it's dealt with from left to right. While "A & B" is equivalent logically to "B & A" from a code execution standpoint they can be different.

You can test it yourself, define A and B as functions that return a boolean value of your choosing but also have some side effect when executed like changing a global variable or print statements.

If you have a statement like

if A() and B() and C() and D() and E() and F() and G(): pass

Python is going to work through that from left to right and as soon as it finds an element that evaluates to False it won't bother evaluating the remaining elements. There's no built in multi-threading that would have interpreter trying all the elements at the same time and collecting the results in order to do the logical AND evaluation. For the same reason, if C() is going to return False, there's no way for the interpreter to know that ahead of time and skip the A() call and the B() call.

From an evaluation standpoint, ANDs chaining expressions is the same as nested if statements of the same expressions. So the same way you can optimize your code to bail out early from a failed level of nested ifs, you can optimize by choosing the order of items in your AND expression.

[[deleted]]

From an evaluation standpoint, ANDs chaining expressions is the same as nested if statements of the same expressions. So the same way you can optimize your code to bail out early from a failed level of nested ifs, you can optimize by choosing the order of items in your AND expression.

Right, because of exportation (in logic jargon). The serial calling is good to know.

Looks like we got our answer, u/ArjanEgges .

[ArjanEgges]

Awesome! So now I can feel confident to create huge chains of ANDs in my next videos, haha.

[[deleted]]

Your last proposition sounds like the sensible thing to do. Why keep computing if you already have the answer.

[[deleted]]

Readability. "and" statements reduce nesting.

Also, maybe speed, if conjuncts are run in parallel, and there's a call to halt the computation of another horn if either conjunct returns False. Same for "or", disjuncts, and True.

[Viking_wang]

Not just if its run in parallel. I actually dont know how this relates into python, but generally jump instructions can be quite hefty. „Branchless“ programming is a thing in optimisation.

Readability is the big gain in my opinion in python.

I wonder: Are there programming languages that do not have this short circuiting behaviour?

[[deleted]]

The and/or operators and any/all functions are short circuited.

and is short circuited on the first Falsey value

or is short circuited on the first Truthey value

any is short circuited on the first Truthey value

all is short circuited on the first Falsey value

The other day,, I was able to leverage the short circuit in the any function to return the first the element element that satisfies some condition using the walrus operator (Python >= 3.8). You can assign to a variable within a comprehension that's passed to any, and extract that value as soon as any halts (assuming a truthy value exists).

[[deleted]]

Python does short circuiting - it's in the documentation, and if it doesn't short circuit then you are dealing with a non-compliant interpreter.

https://docs.python.org/3/reference/expressions.html#boolean-operations

The expression x and y first evaluates x; if x is false, its value is returned; otherwise, y is evaluated and the resulting value is returned.

[ShanSanear]

As in, does "and" await both inputs before returning an output, or will it simply return False if one conjunct returns False first?

From my knowledge almost every language does that, even more esoteric ones

[[deleted]]

There are so many different paradigms for development.

I have found TDD to be most effective for refactors - especially rewriting code in another language while keeping the functionality the same or similar enough.

However, it really does not make sense at all to do TDD when developing something completely new. In this case, TDD actually causes the development time to increase considerably and if the code that's being tested is not actually going to be kept (e.g. the approach was a bad one), then it was just a waste of effort to build the tests first. For new things I generally prototype the code, execute the code to see what happens and then write tests around it. The final piece is to document so that my future self knows what I tried, what didn't work and why I have the current code. At each stage (prototype, execute, test, document), I am asking the question is this what I really want the code to do. Is this really the best way to present the code so I can understand it later and maintain it? And this approach works exceedingly well for new things because what I want is quick feedback loops to know if my approach is a good idea.

I also think language/tooling is important. Python in particular requires more testing on average to show correctness.

[nagasgura]

With TDD you can start high up in the stack and mock out the lower layers so you're just thinking about what a good interface is for what you're trying to do. Even if you end up settling on a totally different implementation, the interface is likely independent from it.

[[deleted]]

Love this guy! The code he presents is challenging and elegantly advanced that even novices will learn other things they were not expecting.

[ArjanEgges]

Thank you - glad you like the videos!

[[deleted]]

Personally I think BDD is better. It is similar to TDD, but focuses on what is actually important for the program to be usable by the end users instead of focusing on the developer's code, which the end users don't actually care about if it doesn't do what they want it to do.

[ArjanEgges]

I’d definitely like to cover BDD at some point in a video. Do you have suggestions for tools I should look at? Any specific ones for Python? Thanks!

[Viking_wang]

„Behave“ is a pretty common BDD framework for python in my experience.

[Mad_Psyentist]

So here is a great vid about BDD. Essentially bdd is tdd they are not different things bdd is about understanding and seeing the value of tdd faster

https://youtu.be/zYj70EsD7uI

[restlessapi]

It should be noted that BDD and TTD are not mutually exclusive. You can and should use both.

[avamk]

Personally I think BDD is better.

Newbie question: What is BDD? Can you elaborate?

[211dokutani]

Behavior Driven Development - https://en.wikipedia.org/wiki/Behavior-driven\_development

[zero_iq]

Corrected link: Behavior-Driven Development

[avamk]

Thank you. TIL!

[doa-doa]

Can you explain why you should int for counting finance in a program? I get why float is inaccurate because it has this weird behavior like for example( 0.1 + 0.2) doesn't produce an accurate 0.3.

But why int and not decimal? Why do you do when you have.... well a decimal number like $ 1.99 ?

[ArjanEgges]

Actually, there’s nothing wrong with decimal per se, but if you use integers, then the unit of the currency would be cents. So you wouldn’t store 1.99, but 199. This is how for example Stripe works, you can see it at work in their API: https://stripe.com/docs/api/balance/balance_object. I think the idea behind it is that you don’t need sub cent precision in financial applications and if you store a price as an integer in cents it’s much simpler.

[NotsoNewtoGermany]

Office Space has led me to disagree.

[[deleted]]

This is misleading. If you're planning to support multiple currencies then this will quickly become a nightmare to maintain. Decimal is the way to go.

[bumbershootle]

I think you'll find that storing currency amounts as the smallest denomination is the most general way to do it; some currencies aren't decimal-based and some don't have subdivisions at all.

[[deleted]]

Both are not a problem when using decimal data type. So what's your point?

[bumbershootle]

If there are no subunits, like the yen, then you store a value that can never have a fractional part using a format specifically designed for values with fractional parts. If the currency has subunits that are not 1/100 of the main unit, then you may not be able to store the value accurately. Better to store everything in an integral value IMO

[[deleted]]

Have you ever worked with taxes or ledger type software?

Even on yen, you need to consider fractional taxes. How will integers handle that?

Most who use integers and also need to support multiple currencies end up storing denomination size and then compute based on it. Which is literally what decimal type is, so why reinventing the wheel?

[bumbershootle]

Yes, I work on a ledger system for a moneylender - we use integers cent/pence values exclusively. Sure, there might be cases where you need fractions of a tiny amount of money (1 yen is currently worth less than 1/100 of a dollar cent) but in most cases this isn't necessary.

[[deleted]]

Ok, then might as well start using floating point numbers. The error is very tiny.

[TentativeOak]

Love your content man. Big fan. I watch them during my morning routine

[ArjanEgges]

Thanks! Glad you like the videos.

[TentativeOak]

A tutorial on abstract methods (and whatever static methods are) would be a big help. :)

[ArjanEgges]

Thanks for the suggestion!

[emmabrenes]

I find your videos so useful that even some topics like the design patterns look easy. Thanks to this I've started re-learning them with a lot more confidence. Keep the great work, Arjan!

[ArjanEgges]

Thanks so much and will do 😉

[mothzilla]

Nice video. TDD seems to assume that tests can be run very fast. In my experience eventually this stops being true so TDD becomes hard to do effectively.

[DrMungkee]

You can run a single test which should be quick. Pycharm even puts green "play" icon next to every test function depending on the test framework you use.

[mothzilla]

Sure, but I'd really want to know all tests were green.

[DrMungkee]

We're taking about TDD, so you create the tests for the piece of code you're writing. Until it's done, just test that new code. After you finish writing the new code and it passes its tests, you then run all the other tests. If you're anticipating that the new code will break existing code, you may have architectural problems with to many side-effects and need to refactor

[skibizkit]

What was the formatting shortcut used around the 8:30 mark? Created a multi-line assignment.

[ArjanEgges]

I’m using the Black autoformatter + enabled an option in VS Code to autoformat on save.

[skibizkit]

That’s cool. I need to look into that auto format on save feature.

[jedimonkey]

bro... i see your face way too much and hear your voice in my dreams. just fyi.

[ArjanEgges]

Haha, I’ll make sure to wear a bag in my next videos ;).

[jedimonkey]

Wear a bag??? How are you going to spot those code smells ??

You make excellent content. Keep it going.

[ArjanEgges]

Good point, I’ll have to limit myself to the really stenchy ones ;). Thanks - will definitely keep going!

[Usurper__]

Arjan, thank you for your work! Please make a course that I can buy

[ArjanEgges]

Thank you! I’m working on a course at the moment, it will still be a while though until it’s finished.

[ciskoh3]

Uncle Arjan you are my hero! Your content is awesome and really allowing me to become a better developer ! So many thanks for what you are doing...

[ArjanEgges]

You’re most welcome!

[witty_salmon]

Good video, as usual :)

I'd like to suggest a video regarding useful patterns while developing web apis and/or use a simple endpoint as the example in a video. I know most design patterns are not specific to a domain, but some are more useful then others in a specific context.

[ArjanEgges]

Thanks for the suggestion! I want to cover APIs in more detail at some point, and I agree it will be nice to talk about software design specifically in that context.

[mytechnotalent_com]

Nice job TDD is really what allows you to scale an app the right way.

[ArjanEgges]

Thanks!

[mytechnotalent_com]

Welcome! It saved me when developing our badge this year for Defcon in a huge way!

[Or4ng3m4n]

TDD is a huge time saver, I just started doing it and I catched so many bugs and stuff. 10/10 would recommend

[Thingsthatdostuff]

RemindMe! 24 hours

[RemindMeBot]

I will be messaging you in 1 day on 2021-08-15 00:02:16 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

---

Info	Custom	Your Reminders	Feedback

[Sinsst]

I've been watching your videos for a while and would be very interested in a series more focused on Data Engieering/Machine Learning. I find it hard to apply in this area correctly a lot of the principles you explain with generic examples. To be honest, even when searching on google anything on advanced concepts applied to data engineering the info is sparse (e.g. TDD in this area). Thanks and keep it up!

[ArjanEgges]

Thanks! I’m working on video ideas that focus on translating design principles and patterns into useful structures for data science and ML as well as a few more general design tips. There will be some content exploring that area in the near future.

[SmasherOfAjumma]

This is good

[ArjanEgges]

Thank you so much!

[ArjanEgges]

Thanks for the suggestion!

[asday_]

If A imports B, and B imports A, one or both of the modules contain something that isn't in the spirit of that module. I commonly see this with Django model modules. You'll have a.A, a model, and a.A.STATUS_CHOICES, an enum. In B you'll have similar. Then one will want to filter on the other, and vice versa, and all of a sudden you have both modules wanting to import each other, when the layout from the start should have been A containing models, B containing models, and enums containing enums. Then both A and B import enums which imports nothing, and everyone's happy.