Finally Bringing Database Testing Together for Liquibase
I work for Liquibase. For the uninitiated, Liquibase is an open-source database-independent library for tracking, managing and applying database schema changes. Recently, our team was discussing how to prove which databases we support. Turns out that the question “Which databases does Liquibase support?” is not as simple as we initially thought.
What does supporting a database really mean?
First, we had to consider the difference between databases and versions. For example, MySQL 5.7, MySQL 8.0, Postgres 11, and Postgres 12. Then, we also needed to consider that each platform has database-specific variants that create minor runtime differences, i.e., Postgres 11 runs slightly differently in Docker vs AWS RDS.
This workflow is very similar to normal CI/CD workflows. A normal app dev workflow tests the same changes against different environments at different stages, from development to testing, testing to staging, and finally to production. In our case, we are testing the same changes against different environments as the end goal and not a step to releasing a new software version.
Testing against different database versions.
Now that we established Liquibase should function consistently for specific database versions based on the target platform, we could start planning out how to test it. Once we defined a few of the variables that will scale as we test against more databases, it was clear that it was going to scale very quickly and we should rely on automation to run these tests. There are several software automation platforms available that allow for different tests to be run in parallel. We picked Github Actions as our test runner because it would allow us to easily set up a build matrix running the same tests against a few different variations.
The automation is ready. Each platform infrastructure is created, the database software setup, the tests are run, and finally, the infrastructure is torn down. With everything running in different environments, we now need a way to collect the test results, inspect them and generate a pass/fail report. Generally, this is always a problem during CI/CD automation. The more you isolate and run jobs in parallel, the harder it is to view the results in one place. Luckily, a few weeks ago Liquibase released an additional tool to help with reporting: Liquibase Hub.
Liquibase Hub gives Liquibase users real-time access to view and organize database changes.
By using Hub, we were able to create a very separated CI/CD workflow that tested our changes against multiple target databases in different infrastructure while tracking the results in one place. After all the tests ran, we were able to create one API query to the Liquibase Hub API and filter through the results to create an easily readable report.
Wrapping it up
Without Liquibase Hub providing a real-time view of all the changes, we would not have been able to run multiple database tests in multiple environments. The idea of testing Liquibase against different target environments was not a new idea, but until Hub, it was not feasible to collect and generate reports. If you already use Liquibase, you can set up a free Liquibase Hub account and connect it in minutes. If you’re new to Liquibase, download the latest open source version and once you’re using it to make database changes, you can set up your Hub account and view changes in real time. Try Liquibase Hub today!