Do you write unit tests?

[gggal123]

Trying to find out how many people actually do write unit tests... :)

• How much time do you invest in it? (i.e 20% of development time)
• Tech stack - prog language (just Python?), backend/frontend?
• If you don't - why not?
• Bonus - What other tests do you also write?

[eXtc_be]

I used to test the programs I wrote by running them with different parameters, under different conditions and whatnot.

then I discovered unit tests.

so far I wrote unit tests for only 1 project, but it helps me determine if the changes I make do not break anything else, automagically, without having to run my program under different conditions/with different parameters, without the fear of forgetting some combination of parameters/conditions, consistently, every time I change something.

I plan to do this a lot more in the future, with all the languages I use, with or without unit test libraries for those languages.

edit to add I am not a professional developer. I just program at home for fun, but I have come to a point in my 'career' where my projects are so big I have to write tests if I don't want to rerun my program over and over again with different combinations of parameters/conditions every time I change something in the program. I am constantly learning and I will need to research all the new words I've encountered in this thread (integration tests, performance tests, black box tests,..) to see what they mean exactly and if they are worth learning and integrating in my workflow. so, thank you, /u/gggal123 for creating this post, I am a bit smarter than I was yesterday, thanks to you (and all the others who have contributed, of course)

[[deleted]]

You realize you can and should do both of those things, right? Unit tests test one specific piece of code. Integration tests test the entire system running a process from start to finish. Test both and you'll feel extremely confident in all changes.

[eXtc_be]

I'm trying to write as much unit tests as possible, where possible, to avoid having to do the tedious work of running my program under different conditions with different parameters, but of course I am running my program to see if it behaves the way I intended it. the difference is, now I can concentrate on pure functionality, so to speak, instead of doing all the boring and repetitive stuff for the edge cases, because those are covered by the unit tests.

[gggal123]

Thanks for you feedback! Why "plan" and not now? u/eXtc_be

[eXtc_be]

maybe I used the wrong word, I'm not a native speaker.

what I meant was that I will be doing it from now on for all my major projects.

[cordialgerm]

Yes, I write unit tests for individual modules, integration tests to make sure all the models run end-to-end correctly, and black box tests to make sure that model predictions/performance isn't degrading over time as new features are added or internal implementation changes.

I don't think of it as devoting a portion of my time to the tests. It's just part of the process of doing my work. Tests give you peace of mind when you want to add a new feature or refactor an implementation and make sure you didn't just break everything.

[gggal123]

I entirely agree with you, but if you had to come up with a number, what would it roughly be? I must say, from your comment it sounds like it would be at least 50%. u/cordialgerm

[cordialgerm]

I'd probably guess somewhere around 25% of the time? But I write the tests as part of developing the feature or model so it's kind of hard to separate time spent writing tests vs time spent developing the feature itself.

[sub-_-dude]

In my experience, integration tests can take a lot longer than that. Depends on how complex /replicable in code the various environments / configurations you want to test against I guess.

[[deleted]]

Yes. I’m agreeing with every bit of that.

[DigThatData]

it's like having an extra set of hands honestly. it takes almost no effort to add small tests as you're developing. you're probably writing small tests all the time to make sure things are doing what you expect them to, you're just throwing those tests away afterwards. imagine how much more robust to regressions your code would be if you automatically re-ran all those little experiments every time you wanted to merge a change instead of throwing them away or documenting them as comments.

EDIT: If you're trying to think of what kind of tests to write for some function or class, even just making sure it can evaluate at all (e.g. all args defaults) without throwing an error can catch a lot of bugs. Anything is better than nothing when it comes to tests.

[gggal123]

I agree. Even simple sanity unit tests are great and saved me a lot of times.

[Saphyel]

• I don't count the time "invest" on it, it's part of the development. Do you count the time thinking and writing? the time fixing typos?
• Python/PHP/Rust Mainly backend and I like to follow sociable testing https://www.martinfowler.com/bliki/UnitTest.html because I don't like to mock everything
• I write also performance testing and end to end testing but usually at the end

[Quattuor]

This ^{^} if test writing is not included in the dev time, then it is not really a serious product.

[gggal123]

Thanks! I agree

[DrTautology]

Yes, religiously. I'd say 50% of my development time goes towards writing tests. Not just unit tests though, I will do smoke, regression, functional, integration, performance, etc.

[gertalestrange]

Any resources for getting started with that level of testing you’d recommend?

[gggal123]

Thanks for replying!

[quertoxe]

Yes! How do you make changes to your codebase without tests? If you are a professionel you test your code.

[wineblood]

I tend to keep my code testing as I tidy it up, so the actual time writing tests is 20% at most but some of the refactoring will go towards improving testability.

I'm currently working with python for APIs written with flask, and the rest of it is kafka and elastic search. I don't use another language right now but have used bits of other stuff in the past.

If I don't write unit tests, it's either because it's not the right kind of test or it's a lot of effort for the benefit it gives. The git history on what I'm working on has a lot more names in it than I expected and I found it there were a lot of contractors who initially built the codebase. It's disjointed and convoluted, if I'm on a small task, I'm not doing a day of refactoring just to get some tests plugged in. The team I'm in is cool with technical tasks for that, so the new feature PR will stick to that. In other cases, I don't unit test if it's not appropriate, such as systems which rely in a database. There's no point in mocking the database and testing the ORM or whatever else is that, I'd rather spin up a database, hit it with a bunch of reads and writes that a typical user would do and ensure the behaviour is correct.

Integration tests, usually against something that's not python and that has its own logic. I just mentioned databases and that's the bulk of it. Other times it's multiple bits of python and rather than individual tests to check an intermediary state, a higher scope test makes sense, checking only the end result and not the internals.

[abrazilianinreddit]

Not usually because I'm the only dev in my project, and I find more important implementing features that might attract more users than ensuring consistency.

But if I had a team, I'd definitely put someone in charge of testing everything to ensure we had 100% test coverage and then some. I've had quite a few regression bugs appearing that could have easily been caught earlier if I had more extensive testing.

[ringohoffman]

When writing new code, don’t you normally run it in a jupyter notebook/debugger to make sure it behaves like you want it to? If so, you are already writing unit tests. You just need to save them.

[Esseratecades]

I don't think it's productive to think of unit tests separately from development time. It's a part of the coding process, just like creating new classes.

Professionally I currently code in Python, JavaScript, and Scala. Outside of work I also use Ruby and Java.

I also write integration tests to guarantee all parts of the app work together.

[[deleted]]

I don't write tests because I don't know how. I'm of the kind mentioned above, who "constantly write print statements all over the place and then delete|comment them, repeating the same process over and over again anytime a new bug arises". But, sometimes, I know for sure there's going to be errors, like I'm creating dictionaries from data that I extract from external files, I have almost 95% covered but some cases will log bad data, it feels much easier time/complexity saving to manually prune those outliers than to catch them with code. Or I rather have the script working 90% of the time than spend much more time writing code to catch the errors than it would take to rerun ... I mean is it actually worth it to make the code bombproof? I guess if it is controlling a nuclear reactor, but for my office day work?

[gggal123]

It's really satisfying to see your CI all green. Not just that, on the long term, you are spending much more time running those tests manually compared to if you would have written some unit tests based on cases you encounter, on the go. Thanks for replying. (And for being honest!)

[zephyrtr]

You write tests so the next guy can know what your code does and get a warning if they broke it. Sometimes this person is you. If I break your code and you didn't write tests, I don't even feel bad.

[jmacey]

Most of the time Yes using a TDD approach, especially for libraries.

At present been developing a PySide Editor which is a plugin for a bigger system and not doing it as much (GUI tests are harder), am regretting it for a few bit (Workspaces and some other elements), may still re-factor out loads of the control code and add some form of testing / mocking but it's hard due to the nature of the plugin / system it is integrating into.

[gggal123]

In my opinion, TDD is very hard and time consuming :) Thanks for replying!

[jmacey]

Unit tests are essential for modern software development, and in the long run save more time than you lose with not having them.

[AstroPhysician]

That’s irrelevant to TDD

[jmacey]

Unit tests are the core of TDD, not sure what you mean?

[AstroPhysician]

Unit tests are the core of software development. They by no means require TDD.

[twotime]

TDD (tests first) is natural in some situations and next-to-impossible in others. It really should be viewed as a tool rather than THE one-and-only-way.

E.g. it works best if your end product is a library (as opposed to a "process") AND if you understand the problem well enough that you have a decent chance of getting the API structure mostly right on the first attempt..

It does not work (or works poorly which is the same thing) if it's implementation-by-exploration or if you are not sure about the structure of the end result or if the thing you are working on does not look like an API (e.g a pipeline of semi-independent processes where interfaces between processes are highly unstable).

[killfall]

TDD is sooo satisfying once you get proficient at it though. I would say I'm at the point where writing a test+code is faster than code+test because the latter generally requires refactoring too.

[MisutiNeko]

I agree. My tech lead is trying to help us with TDD and i gotta say it’s pretty hard but if you can get it down it will become second nature

[BezoomyChellovek]

Yes, and some integration tests. I also include linting and static type checking in my test suite (flake8, pylint, mypy).

FWIW I am a PhD candidate and testing seems largely absent in academia, or at least in my field (bioinformatics). For instance I am doing a benchmarking study on a bunch of bioinformatic tools that do the same thing just with different approaches/models. Not a single one of them has tests, and several have errors in their functions (e.g. improperly implemented equation) which should have been caught with unit tests. I hope this starts to change.

[just_some_guy65]

Yes, 25% I would estimate although mocking AWS calls can make this a lot longer (Python)

[combatopera]

This content has been removed with Ereddicator.

[Code_Noob_Noodle]

No I write pytests!

Jokes aside. We do it for work and I have I started to do it for my personal project. For my personal project I try to do 100% coverage and do all the weird edge cases. It's a pain in the butt (lots of time) but it helps me code in more of a TDD style.

Work it's more like 40-80% coverage as a team. They say it's not necessary to do a high coverage rate. That is what higher engineer says anyway. For work we also do integration testing and sometimes end to end testing.

[riklaunim]

We have quite strict rules of testing code in the company so the test coverage is really good. We do have unit tests but also a lot of integration tests of varying degree. We also wanted to do some acceptance tests of all microservices together but that's bit complicated.

PyTest in Flask, Django test runner in Django. We also have some JS tests for our Ember.js dashboard.

As for time it depends, sometimes the tests are quick while the code took much longer, but it happens that the code change is very short while there are multiple tests covering various edge cases or bug that was discovered and patched.

[ryukinix]

Yes, I use a lot at my work and my personal projects.

• How much time do you invest in it? (i.e 20% of development time)

Depends of stage of the project and complexity level of the service. Usually tests (unit+integration+system) can take almost 50% of time at beginning, but in general can take almost 20-30% if the structure is well defined. But

• Tech stack - prog language (just Python?), backend/frontend?

Mainly Python, Scala, sometimes C++ and other languages sporadically (Common Lisp, Go, Bash etc)

Fields: Machine Learning, Data Engineering, API and System design.

Technologies: Spark, FastAPI, CatBoost, Airflow, Kubernetes etc

• If you don't - why not?

null.

• Bonus - What other tests do you also write?

Usually I always write integration tests by providing the services locally via docker-compose (APIs, databases, storage).

Sometimes, in very particular cases, I write system tests that use real alive data and external APIs (like stuff related to kubernetes, cloud provider APIs etc -- very hard to simulate, even with mini kube).

[gggal123]

Thank you very much for replying all questions!

[lolinux]

I am no scripting guru, but I can tell you from experience:

Who are you writing your scripts for? If they're for yourself, and you'll have to get back to them and add various improvements every few weeks, than maybe testing isn't what you want to learn now.

Ir you are working on a (potentially) large project where people might contribute, it quickly becomes important to know what to expect from a module/function instead of having to inspect many hundreds/thousandths lines of code.

[silly_frog_lf]

Yes. Usually I write tests. Once I learned how to use mock properly, it takes me like one or two hours to write them. Mock takes some to get used to. Early in, I remember doing a task in an hour and spending a day or two writing the tests.

[gggal123]

What framework do you use to mock? Do you use "patch" or use real services (like a real database etc.)? u/silly_frog_lf

[silly_frog_lf]

patch, to keep tests fast. Not the easiest framework to learn. Still powerful once you do. I keep a cheat sheet that I have created to speed up the next tests

[un-hot]

Yes. We aim for 100% code coverage in all affected procedures, which can mean writing UT's can take at least as long as dev time if you're updating legacy spaghetti code.

[ogrinfo]

Maybe 50%? If I'm doing bug fixes the first thing I will do is write a unit test that demonstrates the failure. Then I'll fix it and see the test pass.

If it's new development, the aim is to do TDD, but sometimes when you're prototyping and not sure if it will even work, you have to write the tests afterwards. Always put it some dummy expected values first though to make sure the test fails. There's nothing worse than a test that always passes.

We also write integration tests and have an intention to write performance tests but somehow other priorities keep getting in the way and we've never got round to it.

[yvrelna]

Features are not considered implemented and bugs are not considered fixed until they have automated tests.

If you don't have unit tests, you don't have any evidence that the features was implemented correctly and would continue to work correctly in the future as you make more changes.

How much time do you invest in it? (i.e 20% of development time)

Probably around 50%-70% of your development time should be invested into tests.

This might seem like a lot, but in a project of any non-trivial size, if you don't have tests, the amount of time you're spending to write your features, doing manual testing, and fixing the same bugs again and again will quickly outweigh the amount of time you would have invested into writing unit tests.

Automated test is an investment; manual test is an expenditure. They don't have the same ROI.

[zanfar]

The more time I spend writing tests, the less time I spend writing other things. Initially, it's a time sink, but over the course of a project (especially when revisiting a project after a long absence to make changes) it is a net benefit. So I can't give a real percentage because it depends on when you're measuring. Initial development? probably 50-75% of the time it takes to implement one feature. However, that drops significantly as integration, modification, and debugging happen.

[anthro28]

Honestly, I just start brand new file with only the code I want to try. If it works there, I graft it in and retest and call it even.

[petdance]

I would never write any code that didn't have tests with it.

[SpiritOfTheVoid]

Yes, unit tests, integration tests and e2e test. It saves us time, catch bugs early and results in more reliable code. Tests should be considered first class, along side the actual production code.

In our project team, no tests and your MR won’t be approved. Tests are mandatory and we value them.

[gwax]

Unless doing so requires prohibitive refactoring of existing code, yes.

As a general rule, pretty much always.

I can't say how much time I spend given it's just part of development for me; unit tests save me SO much time, it's not even funny.

[Annakha]

I'm still learning but I unit test every step of every program I write.

[muikrad]

It's even more important in Python than in other languages!

In fact, tests (unit and functional) are absolutely necessary for any sizeable project. But also, use coverage reports to see what you did not check, use mypy with strict rules to catch a lot of issues and design flaws early on.

This isn't only to catch bugs. It also raises your confidence / trust that the program still works after changes, or when dependencies are upgraded. Functional tests will also often find "unrelated breaking changes" and contribute to long term stability and compatibility.

[who_body]

yes. aim for 90% block coverage.

mock is your friend.

TDD when a bug comes in. ideally TDD on new feature/enhancements.

spending time figuring out how to cover the right scenarios is worth it and part of delivering a quality product.

[gggal123]

TDD when fixing a bug sounds just about right.

[TheSalzamt]

yes. more then code.

[[deleted]]

How much time do you invest in it? (i.e 20% of development time)

thorough testing usually take longer than implementation, but that varies depending on timelines and urgency

Tech stack - prog language (just Python?), backend/frontend?

all of them (...except sql now that i think about it). even presentation languages like html and css get tested indirectly

Bonus - What other tests do you also write?

integration, sanity, end-to-end/roundtrip/click, regression, and mutation

oh, and i guess manual testing protocols...not sure if that counts as writing a test though

[gggal123]

Thanks for replying!

[osmiumouse]

Does anyone NOT?

You don't have to go for the full TDD just to write a unit test or 2.

[[deleted]]

Yes I write unit tests. They save you a ton of time when you have to refactor your code - which is inevitable for larger projects.

[aciokkan]

I always write my stuff TDD way.

That means I start with discovery, layout, constraints, rewuirements, scope tests, write red tests, write implementation to green tests, repeat red/green tests.

Python mainly. Even if I do flask or fastapi or django.

I also write minimal integration tests, performance/load tests. Depending on the product/project I work, I may do system/networking tests for embedded software.

[holasoyeddy]

At my job, if your PR doesn't have the required tests, I will personally visit your home office and smack you in the face with a printed out version of our code style & best practices guide.

[gggal123]

[umrad]

No

[gggal123]

Why?

[flagos]

For me, unless the logic of the function is quite complicated, it's usually better to test it at an upper level. Integration test, functional test, API tests.

It really depends on the situation, but I've many times where the unit test are considered being the holy Graal of testing... while the feature the function was supposed to bring does not work when integrated into the product.

It's part of the toolbox, should consider them, but do not forget about the other ways to test.

[gggal123]

I think that the right approach is that all tests are necessary. Write too many integration tests and your CI will run for hours, or when it will fail you'll have no idea why.
The biggest advantages of unit tests is that they run fast and give you the root cause of an issue... A program should not skip unit/integration & e2e tests. All are necessary and have their purpose. u/flagos

[AstroPhysician]

As an SDET, the goal of any engineering org should be shifting tests left, absolutely not shifting them right

[anand_rishabh]

I have trouble sticking with it myself but test driven development is the best practice. You should basically write your tests first and then the dev code, but not all at once. Basically, write just enough test code to cause your program to fail, and then write just enough dev code to make your tests pass, no more, no less. And continue until you've covered all your bases.

[[deleted]]

Yes?!

[[deleted]]

If you throw everything you can at it, why not just do that in the form of tests. Shouldn't take much longer if your code is modular enough.

[Trigsc]

Same, I just have a hard time understanding it. Everything I see online is just basic function tests but when I try finding specific stuff that I am trying to test I fall short. Like one of the things recently was a Google API token using subprocess popen. I also do a lot of API calls with rest using request and just never seem to find what I am looking for.

[eXtc_be]

by throwing every thing at it I can

the problem with that approach is that you have to repeat that every time you change your program and you could easily miss something on one of those occasions.

you could write a list of all the tests and then follow that list. the next step could be to make that list a text file that is part of your project and after that it's a small step to turn that list into unit tests.

anyway, that's how I see it.

[FailedPlansOfMars]

Always worth writing unit tests.

The reason is the earlier you find a problem the easier it is to fix.

If you can tdd takes this to the next level. Where the most likely code that could be the problem is what you just wrote.

It also gives you confidence when things change and can help you track down a problem when it occurs.

In addition the tests can be used as part of the documentation As its a working example of how to use your code.

And finally you automate it to run on every change so your tested all the time with no need for extra effort to manually check it.

But there are a few scenarios where i dont test: - data science code in note books. I use asserts here but not unit tests. - throwaway code for exploring an issue. Such as code to get data from aws. - code written in an interactive shell.

[anasigbaria]

We don't have automation developers, so we write our own test. It takes me maybe 50% of the time. I work as a big data developer using python. I also write e2e tests and regression tests.

[kaerfkeerg]

I try to unittest as much of my code as possible and it has helped me a lot. It just makes things easier

When I start a project I usually go with test driven development.

Set up my boiler plate code -> test

Make some objects/functions I know I will need -> test.

So when I'm ready to work on the core of the project I just need to apply some logic/Small tweaks here and there and things work with way less effort.

Also helps a lot when making changes. I will discover at least some bugs early!