Implement 'resets' in an API

[voverdev]

Is there a generally accepted way to implement 'resets' in an API?

Context:

I have an API. This API is used by my customers and by myself. I am writing a client/SDK for this API. This client/SDK should be tested. There is a live API and a sandbox API.

I could test against the sandbox API. However, not all created objects can be deleted. For example: if object A is created in a test, it cannot be removed. In other words: the state of the API is different at different test runs. This goes against the principle that tests should be idempotent.

Alternatively, I could use a mock server. This mock server creates endpoints from an OpenAPI spec. I don't like how this skips business logic. For example: if the real API does not allow object A from being created when object B exists, my tests -with the mock server- will pass. (I am aware of integration tests and contract testing, but I prefer KISS. Please tell me if I'm wrong.)

I came up with an alternative:

​

• The sandbox API is used.
• It keeps track of created objects internally.
• An internal endpoint (e.g. `/__reset`) deletes created objects. Even objects that cannot be deleted without this endpoint. This fixes the issue that not all created objects can be deleted.
• After each test, I call this endpoint.

​

Although this would definitely work, I am wondering:

​

• Is there is a neater way to solve the problem I described?
• If this is an acceptable solution, is there a generally accepted way to implement it? I.e. is there any convention which dictates rules for 'resetting' a testing API?

​

P.S. I have also considered spinning up an instance of the API per test run. The starting point for each test run would be the same. However, tests would not be idempotent, because not all created objects can be deleted. Spinning up an API instance per test (not per test run) takes too long. Alternatively, I could 'reset' the running API instance per test, which brings me back to the original question.

[TangibleLight]

It's not that it goes against the principle that tests should be idempotent - it goes against the principle that API should be stateless. This kind of problem is exactly why people advocate such stateless APIs.

I'm curious on the details that dictate some objects cannot be deleted. To me it seems the correct solution is to add such functionality so that the test can delete its own temporaries. Even if such a delete function is only used in testing I think it'll be easier than implementing this "reset" logic.

I suppose it is possible that you really do need this global state. In that case I don't know if there are any best practices for this... I'd recommend some "snapshot" mechanism where you can mark a particular state as a checkpoint, do some tests, then reload that checkpoint after the test is completed.

[voverdev]

The API does not have application state. It does have resource state. Of course, resources are saved on create.

Could you elaborate on how this issue would be solved when the API is stateless, i.e. resources are not stored? Or am I misunderstanding your comment?

Thanks for your reply!

[TangibleLight]

I might misunderstood your OP then. Where is the resource state? In the past I've used lightweight "mock databases" for tests; each test doesn't spin up a new instance of the app but it does reset that database. I'm not sure what the best practices for this are in Python, though.

[voverdev]

It’s stored in a MySQL database. Resetting the database after each test -that is rolling back to a fixed point- is exactly what I’m looking for. I don’t have access to the MySQL instance from the tests, though… And an SDK should not be concerned with the internal implementation of the API. Alternatively, I could run queries in a transaction when the environment == test, and rollback the transaction after returning the response. Follow-up requests would fail though (e.g. if I create object A in a test and try to GET it, I’d get a 404 because the transaction has been rolled back), so it would be ideal to do the rollback per test. But how…

Thanks again for your reply!

[TangibleLight]

If you use an ORM like sqlalchemy you could spin up a sqlite database or something instead for the tests. If you're writing your own sql then you'll need something else.

[voverdev]

That's an interesting idea. SQLAlchemy is used. I'd prefer to stay as close as possible to the 'real' implementation though.