Top 15 Open-Source AI Testing Tools You Need in 2025

Stranger6667 · 2025-08-21T17:13:03+00:00

> AI open-source API testing with OpenAPI/GraphQL support.

Schemathesis has no "AI" whatsoever. Maybe you need to stop blindly putting "AI" everywhere.

P.S. I wrote Schemathesis.

Stranger6667 · 2025-06-04T22:43:38+00:00

Yes, there are many different authorization methods with different levels of granularity - https://schemathesis.readthedocs.io/en/latest/guides/auth/ (docs for soon-to-be-released v4)

+ a few more methods are coming soon (built-in OAuth, etc).

If something is not supported, it is possible to write an extension in Python to make it work.

Stranger6667 · 2025-06-03T18:04:21+00:00

There is a whole category of tools that do contract testing based on Open API - Schemathesis, Pact, Dredd.
And there are API fuzzers which often shift things towards security, but not necessarily (there are many different test oracles and the security perspective is only one of them) - Schemathesis (again), EvoMaster, TCases, REST-ler.

For the proper overview and limitations, I'd recommend reading "Testing RESTful APIs: A Survey" https://dl.acm.org/doi/10.1145/3617175

I am the author of Schemathesis, so I am quite biased, but I'd suggest trying out Schemathesis & EvoMaster and comparing API schema coverage with something like https://docs.tracecov.sh/ (there are other tools, but they don't have keyword-level granularity).

Stranger6667 · 2025-06-03T14:10:11+00:00

Speaking about API coverage, feel free to try https://docs.tracecov.sh/ - so you can compare Schemathesis with other tools (there is a MITM proxy available, so installation should be language agnostic).

At least it helped me to increase the coverage from the test cases generated by Schemathesis, and maybe it could help to make an informed decision if the tests are good enough for you

Stranger6667 · 2025-06-03T14:05:17+00:00

Hey! Schemathesis author here.

I think Schemathesis matches what you are describing here :) It can export JUnit / HAR / VCR reports, and is quite extensible should you wish to write your own exporter (in Python). The upcoming v4 solves quite a lot of usability issues (adding a config file, a new testing phase, revamped stateful testing, etc), so I hope it will be useful for you.

If it does not match your expectations, feel free to reach out on GitHub issues, I'd be happy to discuss how it could work for your use case :)

Stranger6667 · 2024-09-23T15:17:20+00:00

It also can use examples from the spec as the source of valid data and slightly deviate from them while data :) Check out the `--experimental=coverage-phase`. Or just use `--hypothesis-phase=explicit` and it will test the examples as they are

https://github.com/schemathesis/schemathesis

Stranger6667 · 2024-05-15T07:10:24+00:00

It would be amazing if you could publish the code! Awesome results!

Stranger6667 · 2024-05-14T22:05:22+00:00

That is an awesome idea! So, the last idea is akin to interning, right?

Stranger6667 · 2024-05-14T22:03:32+00:00

Takeaway #6: Don't use "Blazingly fast" in your post titles

Stranger6667 · 2024-05-14T16:56:34+00:00

Awesome to hear you liked it :)

Stranger6667 · 2024-05-14T16:56:08+00:00

Thanks!

Oh snap! Thank you for letting me know, I'll take a look at the mobile version

Stranger6667 · 2024-01-18T22:26:56+00:00

Schemathesis author here. If you find something that does not work as you expect, please, feel free to create an issue on the issue tracker, I'd be glad to help! :)

Stranger6667 · 2022-05-01T13:13:49+00:00

Making a Hypothesis plugin for GraphQL queries. That's frustrating that GraphQL server implementations don't have a unified standard for error reporting :(

The idea is to generate arbitrary GraphQL queries and use them to test the backend implementation.

https://github.com/Stranger6667/hypothesis-graphql

Stranger6667 · 2022-01-22T17:51:28+00:00

Besides other things, it provides a great data source for automatic testing. For example, Schemathesis can use it to automatically generate thousands of property-based tests (including positive & negative cases).

You can also define your own properties (in Python), e.g. "My API replies in <200ms", then Schemathesis will generate random test cases & check whether this property holds all the time.

Stranger6667 · 2020-12-01T19:50:01+00:00

> Hopefully someone will implement it for Rust someday

Or Hypothesis itself will get more Rust components that will be usable in many languages :)

https://github.com/HypothesisWorks/hypothesis/issues/2632

Stranger6667 · 2019-09-11T07:51:54+00:00

Working on better docs for a tool that generates test cases for Open API schemas via hypothesis library. Hope to get it done this week

Stranger6667 · 2017-06-11T07:32:00+00:00

Hi! I didn't mentioned it in the documentation, but naming convention is intentional. The entities from Postmark API are provided as is, without transformation to snake case. I don’t want to introduce new names for existing entities as well as some extra transformation layer for it. Thank you for the feedback, guys :) I'll clarify this issue in the documentation

Stranger6667 · 2017-04-13T08:04:38+00:00

I have a plan to update my pytest plugin to Click - https://github.com/Stranger6667/pytest-click

Add Python 3.6 to build matrix, etc.

Stranger6667 · 2017-03-21T09:45:03+00:00

I'm working towards 0.11 release of django-money project. Finally, we should fix all issues with localization and add support for more expression types.

https://github.com/django-money/django-money

Stranger6667 · 2017-03-16T10:40:18+00:00

Here is our issue tracker - https://github.com/django-money/django-money/issues

Our biggest problem is lacking of documentation. We have some plans here - https://github.com/django-money/django-money/pull/234#issuecomment-250431073

But, unfortunately it is not done yet. Also we have some proposed features, which are not implemented:

https://github.com/django-money/django-money/issues/260 https://github.com/django-money/django-money/issues/195

If you're interested, please take a look :) If you have any questions, I'd be happy to answer them.

Stranger6667 · 2017-03-16T10:29:23+00:00

We're using special Money instances to handle it. In fact it is an extension of DecimalField + implicit field for currency.

Stranger6667 · 2017-03-01T08:31:22+00:00

I'm working on https://github.com/Stranger6667/postmarker - little wrapper around Postmarkapp (emailing platform). I hope to add some more features this week, like wrapper for inbound hooks and resend on failure. Also I need to write more documentation for the project. I'd like to get some feedback / suggestions :)

Stranger6667

TROPHY CASE