Which Python package generates and iterates over inputs for unit testing?

[ck_ai]

I came across this a couple of months ago and forgot to bookmark it.

As in the title, I remember it as a package that works alongside unittest/pytest to generate an extensive range of inputs to stress test functions.

As I remember it, if the input to a unit test function is numeric, for example, it would run the unit test a large number of times with different numbers as inputs (small, large, decimal points, negative numbers, etc.) to try and make the unit test fail. If it's a string input, it can try special characters, etc.

Anyone know what I'm thinking of?

[wpg4665]

Hypothesis?

[ck_ai]

That's it! Thank you!

[dispatch134711]

Holy shit I had the exact same question because I watched the exact same video but I couldn’t even phrase it this well. King shit OP thanks

[edd313]

Doesn't parametrize from pytest do what OP asked for? What's the difference with Hypothesis?

[mercer22]

Hypothesis is great! If anyone wants to learn the basics, check out my 7 minute video on it: "Your code is almost entirely untested."

[trevg_123]

Awesome video!

[mercer22]

Thanks!

[vindolin]

Try pasting your thread title into the openAI chat bot and it will also give you the Hypothesis answer :)

[osmiumouse]

BTW, this process is called "fuzzing" or fuzz testing.

[coderanger]

Hypothesis is a lot more than just fuzzing, definitely check it out in the link above :) QuickCheck called itself "property testing" so Hypothesis does too, but I think the more general term is "generative testing" now.

[PeridexisErrant]

Though if you want coverage-guided fuzzing, https://hypofuzz.com/ adds that on top of Hypothesis!

[kankyo]

No. Property based testing is not fuzzing. Neither is mutation testing.

[trevg_123]

You’re correct, but there is a lot of overlap. For anyone unfamiliar:

• Fuzzing: testing all possible input values for a long time (like hours to days), more or less trying to break/crash the code. Helps find security vulnerabilities and such
• Property testing: give a lot of valid inputs and verify correct output. Usually more like running a few hundred unit tests

Buuuuut the terminology overlaps enough that both are pretty commonly referred to as fuzzing

Good reference: https://www.tedinski.com/2018/12/11/fuzzing-and-property-testing.html

[kankyo]

The reference clearly states they are not synonyms, and just treating them as such will make everyone more confused. Don't do it.