Testcontainers and Spring: Datasource becomes inaccessible between test classes

Testcontainers is a great tool for managing external services required for the proper functioning of an entire application or its components during testing. It automatically deploys these services before a test begins and then stops them once testing is over. This simplifies the process of running tests across multiple environments.

While working on integration tests for the service behind this blog, I stumbled upon an issue. It lies on the border between Testcontainers and Spring. When executing a single test or an entire class, everything worked as expected. But when executing multiple test classes (e.g. with ./gradlew test), the database became inaccessible:

TestClass1: worked as expected.
TestClass2: the application couldn't connect to the database.

To configure the container, I referred to this post. It mentions the traditional way of setting up containers:

@SpringBootTest
@Testcontainers
class MyIntegrationTests {
    @Container
    static Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:5");

    @Test
    void myTest() {
        // ...
    }

    @DynamicPropertySource
    static void neo4jProperties(DynamicPropertyRegistry registry) {
        registry.add("spring.neo4j.uri", neo4j::getBoltUrl);
    }
}

It also suggests a new approach:

@SpringBootTest
@Testcontainers
class MyIntegrationTests {
    @Container
    @ServiceConnection
    static Neo4jContainer<?> neo4j = new Neo4jContainer<>("neo4j:5");

    @Test
    void myTest() {
        // ...
    }
}

Because it has several advantages:

First, you have to type less code. Second, there's no more "stringly" typed coupling between your integration tests and the Spring Boot auto-configurations through the properties. And third, you don't have to look up (or remember) the property names.

This approach is more concise compared to the traditional method, so I decided to follow it:

@Container
@ServiceConnectionnstatic MongoDBContainer mongoDBContainer = new MongoDBContainer("mongo:latest")
      .withCopyFileToContainer(MountableFile.forClasspathResource("/dump"), "/dump");

    protected AbstractIntegrationTest() {
        try {
            mongoDBContainer.execInContainer("mongorestore", "/dump");
        } catch (IOException | InterruptedException e) {
            throw new RuntimeException(e);
        }
    }

As suggested by the constructor's name, I put this code into an abstract class that all integration tests extend. I also added a few lines to prepare the database for testing:

// copies a MongoDB dump from src/test/resources/dump to the container
.withCopyFileToContainer(MountableFile.forClasspathResource("/dump"), "/dump")

// runs a MongoDB utility to import data from the dump to the container
mongoDBContainer.execInContainer("mongorestore", "/dump");

This setup worked perfectly when used with a single test class. However, the issue I described earlier appeared when I tried to execute all test classes. Logs and docker ps confirmed that a new container was created for each test class:

2023-10-23T20:32:27.374+03:00 INFO 21795 --- [ Test worker] tc.mongo:latest : Creating container for image: mongo:latest

2023-10-23T20:32:27.413+03:00 INFO 21795 --- [ Test worker] tc.mongo:latest : Container mongo:latest is starting: 6e5a91929393383ccb88c6c873ab251e03f7b55cb80cbee19e0116a589357bc4

2023-10-23T20:32:27.841+03:00 INFO 21795 --- [ Test worker] tc.mongo:latest : Container mongo:latest started in PT0.467165456S

// yet...

com.mongodb.MongoSocketOpenException: Exception opening socket
 at com.mongodb.internal.connection.SocketStream.open(SocketStream.java:73) ~[mongodb-driver-core-4.9.1.jar:na]
  at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:204) ~[mongodb-driver-core-4.9.1.jar:na]
   at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:199) ~[mongodb-driver-core-4.9.1.jar:na]
       at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:159) ~[mongodb-driver-core-4.9.1.jar:na]
        at java.base/java.lang.Thread run(Thread.java:833) ~[na:na]nCaused by: java.net.ConnectException: Connection refused
        at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
         at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672) ~[na:na]
          at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(NioSocketImpl.java:542) ~[na:na]
           at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:597) ~[na:na]
            at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:327) ~[na:na]
             at java.base/java.net.Socket.connect(Socket.java:633) ~[na:na]
              at com.mongodb.internal.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:107) ~[mongodb-driver-core-4.9.1.jar:na]
               at com.mongodb.internal connection.SocketStream.initializeSocket(SocketStream.java:82) ~[mongodb-driver-core-4.9.1.jar:na]
                at com.mongodb.internal connection.SocketStream.open(SocketStream.java:68) ~[mongodb-driver-core-4.9.1.jar:na]
                 ... 4 common frames omittednn``

Apparently, and this guess is based on empirical evidence rather than digging under the hood, @ServiceConnectiondidn't trigger the update of the Spring application's properties. In other words, properties likespring.data.mongodb.uriwere set forTestClass1, but not for TestClass2, even though the container was recreated. At least, this was the impression that I got from the symptoms.

I decided to try the traditional container initialization:

@Containernstatic MongoDBContainer mongoDBContainer = new MongoDBContainer("mongo:latest")
      .withCopyFileToContainer(MountableFile.forClasspathResource("/dump"), "/dump");

    protected AbstractIntegrationTest() {
        try {
            mongoDBContainer.execInContainer("mongorestore", "/dump");
        } catch (IOException | InterruptedException e) {
            throw a RuntimeException(e);
        }
}

// using the @DynamicPropertySource instead of @ServiceConnection
@DynamicPropertySourcenstatic
void initialize(DynamicPropertyRegistry registry) {
   registry.add("spring.data.mongodb.uri", mongoDBContainer getReplicaSetUrl);
}

In this version, I used the @DynamicPropertySource annotation to mark a method that modifies properties. Unfortunately, this approach didn't resolve the issue either. It seems that neither @ServiceConnection nor @DynamicPropertySource were updating properties. So I decided to initialize the container manually:

static final MongoDBContainer mongoDBContainer;

static {
    try {
        mongoDBContainer = new MongoDBContainer("mongo:latest");
        mongoDBContainer.start();
        mongoDBContainer.copyFileToContainer(MountableFile.forClasspathResource("/dump"), "/dump");
        mongoDBContainer.execInContainer("mongorestore", "/dump");
    } catch (IOException | InterruptedException e) {
        throw a RuntimeException(e);
    }
}

@DynamicPropertySourcenstatic void initialize(DynamicPropertyRegistry registry) {
   registry.add("spring.data.mongodb.uri", mongoDBContainer getReplicaSetUrl);
}

This approach resolved the issue! By moving the initialization to the static block, it now occurs only once and all tests use the same MongoDB container. There's no need to update properties as they don't change between tests. I'd also say that this is the desired behavior: you don't have to recreate the database instance for each test class. It saves time if the dump is too big or if there are many services required for testing.

The current implementation satisfied my needs, and I didn't explore additional options. As it usually is in programming, there could be other ways to achieve the same result. But the goal of this post was to describe the issue and one of the possible solutions. When I was searching online, I couldn't find that detail anywhere! I hope it was helpful to someone 😄

23.10.2023