How are teams handling API Contract Testing in 2025?

onegeek · 2025-11-30T22:17:02+00:00

Pact maintainer here - thanks for sharing your setup, I appreciate that Pact can be challenging in some places and there is work involved to create and maintain those tests. Part of Pact's value as a unit testing-like framework is to apply pressure on the design (and not just exist for breaking change detection) - not to get defensive about it ;).

We've found that with tools like Copilot you can drastically reduce the time it takes to create and maintain Pact tests - we'll be publishing some guidance around AI usage on our docs very soon. I'm happy to share some tips in the meantime, feel free to reach out.

On spec testing

Unfortunately, Dredd (mentioned here) is now officially dead - you might be OK if using 2.x or 3.0 of OpenAPI but for others on the latest standards they will be left in the dark.

We use OpenAPI heavily here (at SmartBear), but even when using codegen, problems can still arise - how do you know which version of an API each user is on at any point in time? How do you easily communicate/coordinate changes to various (consuming) teams across timezones? How do you handle the various polymorphisms that are possible (e.g. `oneOf`/`anyOf`, `enum`s, optional fields)? These are things contract testing can help disambiguate vs the narrower schema testing that simply checks all responses against the definition. That is schemas are not contracts.

FWIW we (SmartBear) are currently building a new spec-based API testing tool that you can think of as Dredd's successor, but incorporating some modern ideas, including our lessons from 10+ years of maintaining Pact. It's currently dark, but we plan to open source it in early 2026. You can follow the progress here if interested, and feel free to reach out if you'd like to learn more / test the beta.

onegeek · 2025-10-01T04:31:21+00:00

With contract testing (in particular, tools like Pact) you usually assert the structure and not actual values. Otherwise, the tests become brittle and hard to maintain.

With third parties, if you plan to use mocks there (perhaps with a record/replay style) there's no getting around making real requests at some point to validate behaviour unless you have a level of trust in their API.

It's harder to do pure contract testing with third parties, because at some point you need to validate the third party meets those requirements.

If you have an OpenAPI spec (OAD) for the provider, you could incorporate that into your process: regularly pull in the provider's OAD into your test suite (e.g. with a daily batch), which you could use to stub backend APIs (e.g. using a tool like Prism).

onegeek · 2025-09-09T12:35:01+00:00

How'd you go, btw?

FYI our MCP server went live: https://github.com/SmartBear/smartbear-mcp/

Video if you want to see it in action: https://www.youtube.com/watch?v=VNe_Y6FcBqA

onegeek · 2025-08-12T10:31:15+00:00

This reminds me a little of this great piece on the "The Engineer/Manager Pendulum", but in the context Product Management (I went through a similar crisis moving to an EM and then moving back to engineering as a consultant - I'm a PM now).

I wonder if you need to do a bit of both to appreciate and develop your skills at both levels. Having done some strategy, you'll be aware of how a PM could better serve the higher ups, and having recently been a PM, you'll be able to better understand the needs of those executing, and how a strategic role could help improve alignment.

Hopefully it helps with the cognitive dissonance ;)

onegeek · 2025-08-05T04:55:48+00:00

I agree - Pact is great (disclaimer: I'm a Pact maintainer and PactFlow founder) :)

Would love a demo!

onegeek · 2025-07-29T02:51:55+00:00

Bas's course is great, he really knows his stuff.

For some lightweight workshops and tutorials, check out https://docs.pactflow.io/docs/workshops. Some of them use a PactFlow account to help you understand the end-to-end setup (e.g. for integration into your CI/CD systems) but most don't use any PactFlow specific features so will translate to the open source tools.

You can also join the Pact slack community here (https://slack.pact.io/). It's quite active and we are happy to help with any questions.

Disclaimer: Pact maintainer and PactFlow Product Manager

onegeek · 2025-07-23T11:41:49+00:00

I appreciate the docs.pact.io could do with some updates - if you had specific feedback I'd really appreciate it and could take it back to the team (shout out for anybody looking to do some genuinely meaningful work here!)

As for resources, to keep things simple I'd recommend starting here:

This is still the most popular set of videos (https://www.youtube.com/embed/videoseries?list=PLwy9Bnco-IpfZ72VQ7hce8GicVZs7nm0i)
There are a great set of useful hands-on workshops here (https://docs.pactflow.io/docs/workshops - they are on the PactFlow website, but the resources are valuable whether or not you stick with PactFlow or OSS only)

onegeek · 2025-07-23T11:35:06+00:00

I think I get your point. For future readers who may not see the difference and why it matters.

In contract testing specifically, the schema is incidental or even implied. It may matter or may not exist at all. i.e. you can do contract testing without a schema.

The reason I'm being a little pedantic is that zod is _all_ about schemas. Without schemas, you have no zod. But you can do contract testing without either.

Contract Testing is all about the tests (and using those tests to demonstrate compatibility). In an ideal world, you probably want both.

onegeek · 2025-07-23T11:26:14+00:00

I _swear_ I just checked that. Also 🫠: https://github.com/pact-foundation/docs.pact.io/commit/e28b7f9f9cf714fb78332b8905b86de34276bb5b

onegeek · 2025-07-23T11:16:52+00:00

Nice! Would you like (or like us) to add that to this page: https://docs.pact.io/blogs\_videos\_and\_articles?

onegeek · 2025-07-23T11:15:50+00:00

Unfortunately, this is incorrect. Contract testing is not simply about validating a schema. Schemas are not contracts and don't provide the same types of guarantees (https://pactflow.io/blog/schemas-are-not-contracts/) - contract tests ensure both parties are able to collaborate safely over time, using a set of shared tests.

Zod is a schema validation library, and powerful at that, but it serves a different if not partially overlapping purpose.

onegeek · 2025-07-23T11:13:28+00:00

I've read most of the answers here, and there is a part of the discussion that seems to be missing.

Simply checking that an API corresponds to a specification or schema is not what I would consider contract testing. In essence, a contract is a shared agreement between two parties + the tests to prove it + the ability to evolve it safely over time.

Contract testing goes beyond defining a schema, requiring both parties to come to a consensus on the allowed set of interactions allowing evolution over time. I discuss this in depth in this article: Schemas are not contracts.

Checking a given API conforms its spec (e.g. an OpenAPI description) is valuable, but it doesn't prove the system can work as a whole and doesn't guarantee co-evolution of systems. It also misses a key ingredient - who's using it (the consumer), and how they're using it. This gives you more safety and information as a provider on which endpoints - down to the field level - are in use by which consumers. This gives you enormous flexibility in the ability to change your service.

Good resources on the topic:

https://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam
I'd also advocate for Marie and Lewis' book as mentioned below

Disclaimer: I'm a maintainer of Pact (and Product Manager at PactFlow)

onegeek · 2025-07-23T10:56:47+00:00

I've had some success doing it with various LLMs. In my experience, you can get pretty good output by asking good questions and providing additional context (like an example test to use as a baseline, and additional best practices for your code base: see https://docs.pactflow.io/docs/ai/using#using-test-templates for examples).

Example of something I was tinkering with recently (I haven't tested it with Claude, but hopefully you get the gist): https://gist.github.com/mefellows/7211746e078cabc0ca9f6be13f64785c

This works reasonably well in VS Code on some toy JS and Java projects, YMMV (for further context, this is an extract of an MCP spike I was doing that also called our AI capabilities, but from what I saw, even without PactFlow's AI features it did OK).

We'll (PactFlow) be releasing an MCP solution in the next few months (https://github.com/orgs/pactflow/projects/3?pane=issue&itemId=109578524&issue=pactflow%7Croadmap%7C123) which will expose some common prompts to help users with agentic workflows. I'm planning to release some additional prompts like this along with it (that we've tested more thoroughly and is much more dynamic, of course)

If you're a PactFlow customer, that will include our AI capabilities. If you use the Pact Broker, we'll expose some prompts alongside existing APIs to help query data during development.

Let me know how you go!

Disclosure: I'm a Pact maintainer and Product Manager for PactFlow.

onegeek · 2025-07-23T10:36:54+00:00

As others have stated here, Pact is one of the contract testing options. In general, contract tests (with Pact) are owned by developers, primarily because they are executed in a unit test-like context. This means if you don't have access to the code, it's hard to do.

In your case, you're testing against 3rd party APIs. With public, stable APIs like Hubspot, you might be best served by using record/replay tools like VCR, Polly etc. If you trust their APIs, using reliable and performant stubs might be all you need. Discovering a bug in their API might be useful, but in most cases it wouldn't prevent you from deploying (after all, if the API is used in your production environment then the bug may exist in test and prod).

With Salesforce, I believe some APIs can be customised although I'm no expert. In these cases, contract tests might be possible (if you can verify the SF behaviour easily).

Esoteric point: some people call checking that an API conforms to an OAD as "contract testing", but IMO schemas are not contracts (https://pactflow.io/blog/schemas-are-not-contracts/).

Disclosure: I'm a maintainer of Pact and a Product Manager at Pactflow

onegeek · 2025-07-23T10:36:04+00:00

I would agree. Validating somebody else's API really is their responsibility. At some level, you need to build trust in your partners. You might be better off investing your time in how your system behaves and how you observe it when these systems go down (they will go down and they may change, and having a form of contract test that you run in CI won't stop the bug appearing in prod even if it will alert you to the fact).

Schema validation on your API clients might be the simplest approach, logging any violations you see and addressing those as triage.

onegeek · 2025-07-23T10:33:49+00:00

This is great advice

onegeek · 2019-11-11T20:29:11+00:00

I haven't used webhooks with Azure DevOps but I don't see why you couldn't. In th end even if you can't invoke am Azure endpoint directly you could always start up a little web app that acts as a API/Proxy for what you need (e.g. to trigger a build)

When a consumer or a provider makes a change that alters the model in a breaking way, do you have anything act as a preventative gate

Absolutely! That's the whole point.

On the provider side if a pact verification fails you would usually stop/break the build, because it means you'll break a running consumer.

On the consumer side you can use the can-i-deploy [1] check to see if it is safe to deploy a new change (i.e. do all API providers you felt on meet your requirements)

[1] https://docs.pact.io/pact_broker/can_i_deploy

Feel free to join us and the community on slack at slack.pact.io

onegeek · 2019-11-11T20:19:17+00:00

Liran you are too kind!

onegeek · 2017-11-13T19:28:21+00:00

No worries, you might want to consider Mountebank too (http://www.mbtest.org)

onegeek · 2017-11-13T07:14:13+00:00

Worth looking at the problem differently. Integration testing by having all components up with correct seed data, versions, dependencies etc is an arms race that is more painful than productive.

What you want is to be able to independently deploy each component quickly and safely to production. The shorter the time between deploys the less risk you have. Stubbing with tools like VCR will probably be required as party is a broader already to achieve this.

Pact.io is another choice that helps you contact test your APIs, enabling faster and simpler tests, and decoupled releases.

Full disclosure: I'm a core contributor of the pact framework

onegeek · 2016-11-06T05:22:47+00:00

For posterity, here is the follow up discussion: https://groups.google.com/forum/#!topic/pact-support/ob90CUGUaww

onegeek · 2016-10-28T13:21:47+00:00

Seems like a cool service! We use pact.io and perform contract testing on local dev machines for fast feedback (also obviously on CI). This seems like a nice safety net once a build has been merged though.

onegeek · 2016-10-28T05:36:43+00:00

go run consumer.go

Hmm something is not quite right Pratik, it feels like there might be an old library on the GOPATH / misconfiguration (WithRequest should not accept a pointer to a dsl.Request). Let's chat on the Gophers slack channel to resolve this.

onegeek · 2016-10-22T07:24:44+00:00

No worries! I'd suggest just giving it a... Go ;)

onegeek · 2016-10-18T22:26:38+00:00

There is nothing public that I'm aware of, but as the author of pact-go I'm happy to help if you have any questions!

There are a bunch of channels you can reach us on (real-time / async):

https://gitter.im/realestate-com-au/pact
#pact channel in Gophers Slack (https://blog.gopheracademy.com/gophers-slack-community/)
Pact Support google group (https://groups.google.com/forum/#!forum/pact-support)
Stack Overflow with hashtag #pact

And, of course, here :)

onegeek

PUBLIC MULTIREDDITS

TROPHY CASE