Basic API and UI test automation framework

A few weeks ago, a potential employer asked me to complete a test assignment. The task was to implement a bunch of API and UI tests. This requirement is quite standard in the field, but I had not previously compiled any work ready to be shared alongside my CV.

Therefore, I made the decision not only to complete the assignment but also to document and explain it here in this blog post.

• Explaining things helps me to understand them better.
• This post will help me to refresh my memory in the future.
• When sharing my code, I can supply it with this post. It's particularly useful as it answers questions like "Why did you choose to implement X in this way?"
• Other automation engineers can find it useful! There are a lot of articles aimed at software developers, but not as many for test engineers. By sharing my experience, I hope to help them.

I want to emphasize that I don't claim to possess exhaustive knowledge, and my experience is limited. There might be better ways to approach certain tasks, and my code can certainly be improved. I always seek information from multiple sources and I encourage you to do the same.

Constructive feedback is always welcome! If you'd like to connect or provide feedback, you can find me on LinkedIn or reach out via email.

Please note that code snippets in this post are simplified. The full project is available here.

Requirements

First of all, let's look at the requirements that I got from the company:

• Implement API tests with REST Assured.
• Implement UI tests for Android with Appium.
• Implement 3 tests:
  • A positive test.
  • A negative test.
  • A test that always fails.
• Write code that is easy to maintain and expand.
• Generate a test report after the execution.
• Provide human-readable descriptions for automated tests.

Project structure

One might argue, that separate projects and repositories should be used for API and UI tests. They work independently from each other and don't share much code. However, they do share some code. For instance, both API and UI manipulate external data in the form of strings. Utility classes that hold common data conversion, parsing and validation methods can be used in both projects. Additionally, frequently used regex patterns should be available for easy reuse too.

Another reason to combine projects comes from the fact that UI tests tend to be slow and unstable. By replacing some of repetitive UI scenarios with API calls, we can save a lot of time and get ourself a more stable suite.

Consider a system under test, where the user can create an account. To avoid issues with the account configuration, we run our tests using fresh accounts. It makes sense to create and verify these accounts primarily through API calls rather than relying on the UI. UI interactions should be reserved for tests that specifically validate the account creation process.

With that in mind, I decided that a single multi-module project was the best approach here.

.
├── api-test
├── shared
├── ui-test
└── pom.xml

In addition to the API and UI test modules, I created a shared module.

Shared module

The shared module serves two purposes:

• It contains all code that isn't exclusively dedicated to either API or UI tests: utility classes and functionality that both API and UI tests require.
• It can be safely used as a dependency for any other module.

The latter is important. Maven doesn't allow for two modules to depend on each other. That'd be a circular dependency:

In software engineering, a circular dependency is a relation between two or more modules which either directly or indirectly depend on each other to function properly. <...> Circular dependencies between larger software modules are considered an anti-pattern because of their negative effects.

Consider the following scenario:

• Module A depends on Module B.
• Module B depends on Module A.

When Maven attempts to compile Module A, it identifies Module B as a dependency and tries to compile it first. However, since Module B relies on Module A, this creates an endless loop in the compilation process. By creating a shared module, we eliminate this problem.

As for the content of the shared module, it includes two utility classes. Since both API and UI test modules interact with property files, I created a dedicated Configuration class to handle properties initialization and data retrieval. This approach minimizes code duplication. For more complex projects with multiple property files (e.g. for different test environments), the Maven property plugin or environment variables can be used to specify the property file dynamically.

It's also worth noting that the API framework (./api-test/src/main/java/org/monese/apitest/) could be stored in the shared module if I was planning to use it in the UI test module.

By replacing some of repetitive UI scenarios with API calls, we can save a lot of time and get ourself a more stable suite.

But that's not the case in this assignment. API and UI modules test different applications.

API tests

The basic usage of REST Assured is straightforward:

given()
    .baseUri("https://some-service.com") // the address of the API server
    .header("Authorization", "Bearer " + "your token") // the API token
    .log().everything() // logs request details to the console
    .get("/user/1"); // the acutal GET request to https://some-service.com/user/1

This code makes a request to retrieve a user with the ID of 1. The get method returns a Response object that offers various tools to validate the response. For example, you can verify that the request was successful (status code = 200) and that the id and email fields contain the expected data.

Response response = given()
    .baseUri("https://some-service.com")
    .header("Authorization", "Bearer " + "your token")
    .log().everything()
    .get("/user/1");
nresponse.then()
    .statusCode(200)
    .body("id", equalTo(1)) // verify that the id=1
    .body("email", equalTo("[email protected]")); // verify that the [email protected]

However, this approach doesn't scale well and could cause problems in a big automation project with hundreds of tests.

• Services may have dozens and even hundreds of endpoints, making it impossible to memorize them all.
• Each endpoint accepts and returns different data.
• Services change over time. Even in a small QA team, not everyone may be aware of these changes.
• If a field's name changes, you'd need to manually update all references to it in the code.
• The removal of obsolete fields can be quite challenging even with static analysis tools.
• As an object-oriented language, Java benefits from classes that describe requests and responses, reducing the issues mentioned above.

To address these points, I introduced an API executor layer.

• It acts as a bridge between tests and the API server.
• It accepts Java objects as requests.
• It returns Java objects as responses.

Instead of using the previous approach where we had to manually specify the expected structure, you can do it only once:

public class GetUsersByIdResponse {
    Long id;

    String email;

    String name;

    UserGender gender;

    UserStatus status;

    // getters and setters
}

And deserialize the JSON response to a Java object:

GetUsersByIdResponse response = given()
    .baseUri("https://some-service.com")
    .header("Authorization", "Bearer " + "your token")
    .log().everything()
    .get("/user/1")
    .as(GetUsersByIdResponse.class); // converts JSON to GetUsersByIdResponse

    assertThat(response.getId()).isEqualTo(1);
    assertThat(response.getEmail()).isEqualTo("[email protected]");

This approach simplifies test development. Even if the automation engineer is unfamiliar with the potential response, they can always refer to the GetUsersByIdResponse class or use code hints.

While retrieving data is straightforward, creating or modifying it on the server can be difficult. You may need to send complex objects with multiple fields and nested objects. By navigating through the codebase you can quickly find how other people deal with similar problems.

In the code above, you still need to know all response classes to feed them to the as(ClassName.class) method. This is where a dedicated API execution layer becomes especially useful.

public class ApiExecutor {
    public static GetUsersByIdResponse getUsersById(Integer userId) {
        return given()
                .baseUri("https://some-service.com")
                .header("Authorization", "Bearer " + "your token")
                .log().everything()
                .get("/user/" + userId)
                .as(GetUsersByIdResponse.class);
    }
}

With this, you can call the getUsersById method in any test and receive a deserialized response object. Great success!

@Testnpublic void getUsersById() {
    GetUsersByIdResponse response = ApiExecutor.getUsersById(1);

    assertThat(response.getId()).isEqualTo(1);
    assertThat(response.getEmail()).isEqualTo("[email protected]");
}

Unfortunately, this solution works only with positive test cases. It doesn't allow verifying the status code or reading the error message in negative tests. The test above would simply fail during the deserialization, because the error response contains different fields.

To address this issue and provide flexibility for handling various response scenarios, I created the ExtendedResponse class. Generics jn Java make it possible:

// T is a placeholder for the actual response classnpublic class ExtendedResponse<T> {
    private final Response response;
    private final Class<T> responseClass;

    public ExtendedResponse(Response response, Class<T> responseClass) {
        this.response = response;
        this.responseClass = responseClass;

        response.prettyPeek(); // prints out the response body to the console
    }

    // verify the status code
    public ExtendedResponse<T> statusCode(int statusCode) {
        response.then().statusCode(statusCode);

        return this;
    }

    // get the deserialized response and work with it
    public ExtendedResponse<T> responseConsumer(Consumer<T> consumer) {
        consumer.accept(response.as(responseClass));

        return this;
    }

    // get the validatable response (REST Assured) and work with it
    public ExtendedResponse<T> validatableConsumer(Consumer<ValidatableResponse> consumer) {
        consumer.accept(response.then());

        return this;
    }
}

This extended response class holds both the REST Assured Response object and the deserialized class. I decided not to expose the Response class, because it would be a dead end - it can't return the ExtendedResponse back.

With these changes, a positive test looks like this:

ApiExecutor.getUsersById(1)
    .statusCode(200)
    .responseConsumer(response -> {
        assertThat(response.getId()).isEqualTo(1);
        assertThat(response.getEmail()).isEqualTo("[email protected]");
    });

A negative test:

ApiExecutor.getUsersById(-1)
    .statusCode(200)
    .validatableConsumer(response -> {
        response.body("message", equalTo("user doesn't exist"));
    });

Moreover, we can handle standard errors responses too:

{
    "message": "user doesn't exist"
}

Just add a method to verify the error message:

public ExtendedResponse<T> errorMessage(String message) {
    response.then()
            .body("message", equalTo(message));

    return this;
}

And use it in your test:

ApiExecutor.getUsersById(-1).errorMessage("user doesn't exist");

Finally, here's the updated API executor for your reference:

public class ApiExecutor {
    private final String baseUrl;
    private final String token;

    public ApiExecutor(String baseUrl, String token) {
        this.baseUrl = baseUrl;
        this.token = token;
    }

    private RequestSpecification restAssured() {
        return given()
                .baseUri(baseUrl)
                .header("Authorization", "Bearer " + token)
                .log().everything();
    }

    public ExtendedResponse<GetUsersByIdResponse> getUsersById(Long id) {
        return new ExtendedResponse<>(restAssured().get("/user/" + userId), GetUsersByIdResponse.class);
    }
}

Now there's a layer that holds all available API requests and responses. You can find more examples in the final project, where you'll find how to send data to the server. My API tests are available here.

UI tests

The requirements tasked me to create an Android application with Appium. I've never worked with it! Appium is based Selenium and reuses same classes and tools, which is good news for many test engineers who are familiar with Selenium. However, I've never worked with it either! Therefore, I'm not going to focus on these two libraries. Both suggest using the page object model that I'm familiar with.

From what I've heard, the bare Selenium lacks many things, making tests harder to develop. Some teams prefer a more advanced Selenide because of that, some simply extend Selenium - and so did I. I created an Element class to serve as a wrapper for the WebElement class.

public class Element {
    private final WebElement webElement;

    public Element(WebElement webElement) {
        this.webElement = webElement;
    }

    // extending the WebElement's functionality
    public Element requireText(String value) {
        await(() -> webElement.getText().equals(value));

        return this;
    }

    // delegate
    public void click() {
        webElement.click();
    }

    // delegate
    public void submit() {
        webElement.submit();
    }

    // delegate
    public void sendKeys(CharSequence... keysToSend) {
        webElement.sendKeys(keysToSend);
    }

    // and so on...
}

This classes delegates default WebElement behavior to maintain its behavior. However, I added a requireText method to handle awaits in one place, since the expected text may not be immediately visible. This also provides a convenient Java interface for working with elements, making the test code more readable.

public Element requireText(String value) {
    await(() -> webElement.getText().equals(value));

    return this;
}

Now I can use it in tests with any element on the page without thiking about awaits and assertions:

pageObject.getSomeField().requireText("some text")

Moreover, you can add more methods and chain them:

pageObject.getSomeField()
    .requireText("some text")
    .requireDisabled()
    .awaitEnabled()
    .click()

This approach makes test development easier and faster. Also, it reduces the amount of bugs in tests and the framework itself! All dirty work is being done behind the scenes.

Another essential addition is the ElementList class, designed to work with lists of elements:

public class ElementList {
    private final List<Element> elements;

    public ElementList(List<Element> elements) {
        this.elements = elements;
    }

    public Element singleElementBy(Predicate<Element> predicate) {
        List<Element> items = elements.stream().filter(predicate).collect(Collectors.toList());

        assertThat(items)
                .overridingErrorMessage("Expected 1 element, but found %d.", items.size())
                .hasSize(1);

        return items.get(0);
    }
}

This class simplifies working with lists of elements, commonly found in UI testing: rows in a <table>, items of a list (<ol>/<ul>) or of a dropdown menu. Each row or item is already represented by a Java object, allowing you to use familiar Java tools to interact with them.

For instance, you can easily select a single element from a list based on specific criteria:

mainScreen.clickExpandMenuButton()
    .clickSettingsButton()
    .getServiceProviders()
    .singleElementBy(e -> e.getText().equals("DeepL")) // finds a single translation by name
    .click();

With this method, you don't need to handle filtering and asserting that only one element matches the criteria. I've used a similar framework on projects with over 1500 end-to-end UI tests. It greatly simplifies test development and maintenance even on that scale.

My UI test are available here.

Summary

I hope that with the codebase and this post, it's now more or less clear how this project works and why I made some decisions. I firmly believe that common Java development practices allow us to write tests more efficiently, mitigate most potential issues and scale this framework to a much larger scope.

If you have any questions, ideas or suggestions, please feel free to reach out. Your input is always welcome.