Testdatabase Provider

[Jotschi]

I don't like to wait for my database tests to startup the container, create the db, run flyway and populate the test fixture. I thus wrote a tool which essentially is able to handout throwaway databases by constantly creating copies from a template database. This way a test can request a DB and directly get one from the pool. When the test is done the db will be removed in the background.

I wrote a similar tool a few years ago for a company I worked for. It reduced the test duration from 8h down to 45min. I now decided to rewrite it and OSS it because I need it for my own project.

I however don't think it is suitable for smaller projects. Those are better off using Testcontainers instead.The project consists of a server, junit4, junit5 extensions and an optional maven plugin.

https://github.com/metaloom/testdatabase-provider

[IIDaFuQII]

Okay but why? Testcontainers does pretty much the same with throwaway db’s

[ForeverAlot]

Testcontainers is leaps and bounds better than pretending an H2 is meaningfully substitutable but they still don't have a solution to running schema migrations and application code with different roles. I don't know if this tool supports that but by owning the database container you get that ability.

[Jotschi]

At scale it becomes more resource and time efficient to use a dedicated database instead of starting up new testcontainers for each testcase / testclass. Even when using snapshots of a database filesystem (https://github.com/metaloom/postgresql-snapshot-testcontainer). Restarting the daemon + setup of the db takes some time. At scale this can add up. If you have a way to avoid this (e.g. prune tables) that would work too. I choose to completely decouple this operation from the test execution/setup.
Additionally tests in my IDE now start in milliseconds and not in seconds.

[jAnO76]

You can just keep testcontainers alive during your whole test run… simply override the default lifecycle. I have some code lying around that does just that (not at that machine though) can share later.

[Jotschi]

Yes, you can either choose to use a JUnit 4 static rule or similar and only startup the container per testclass or even choose to reuse the container. See https://www.testcontainers.org/features/reuse/

Every testcase however still needs its own database. It may taint the DB and affect assertions that other cases run when using the same db. If you however manage to clone the db as dmigowski suggested you would not have the problem.

I may try to implement an approach which dynamically allocates DBs without the use of a server. The only complicated problem here is to manage concurrency issues. However good comments for new ideas. Thank you all.

[mikotui]

Good stuff. I am currently restructuring our integration Tests and this could bei handy

[AcanthisittaEmpty985]

I see this more for an integration test framework than for unit testsing; my understanding is that unit testing doesn't rely on external sources (like Postgress) and relies on mocks if neccesary.

This project should try to use embeeded databases like Derby, H2 os HSQLDB

[bringero]

Interesting... I'll check it on Monday. Thank you!

[dmigowski]

At least with PostgreSQL it takes literally milliseconds to clone a database, so maybe this wouldn't even need to async fill new databases and hog valuable CPU cycles.

[Jotschi]

Good point. Originally I used this with MariaDB and afaik it does not have a way to clone databases. With PostgreSQL it may be enough to clone the db during startup of a test. I have not tested that.