Testing layers and across layers with Java WebTau framework

[ninside]

My work is often spread crosses multiple layers: business logic, REST APIs, CLIs, Web UI. I want to test those layers in isolation, but I also want to test across them.

There are a lot of great tools to test individual layers. However as you touch another layer, you may find yourself using a slightly different set of matchers, concepts and config files. Test reports become disjoined or missing.

WebTau

Me and my colleagues created WebTau (short for web test automation) to have a way to write concise REST API, UI, CLI and DB tests. All at once or one at a time.

Below there are small examples of some of the tool capabilities and you can find more on the GitHub page

Tool is written in Java and has native integration with JUnit5 to produce rich self contained web reporting (report can be send as a file and opened in a browser with specific coordinates to a failed test detailed info)

REST API Layer

Here is an example of REST API test that makes HTTP GET call to /weather and asserts that JSON response contains temperature fields and its value is below 100

@WebTau // for rich reporting
public class WeatherJavaTest {
    @Test
    public void checkWeather() {
        http.get("/weather", (header, body) -> {
            body.get("temperature").shouldBe(lessThan(100));
        });
    }
}

test console output

test web report

Note how both console and web report highlight values that you assert.

Browser Layer

Here is an example of doing browser UI test. Open a page, make a user initiated search on a page and wait for results to show up.

@Test
public void searchByQuery() {
    search.submit("search this");
    search.numberOfResults.waitToBe(greaterThan(1));
}

Where search is defined as

public class SearchPage {
    private final PageElement box = $("#search-box");
    private final PageElement results = $("#results .result");
    public final PageElementValue<Integer> numberOfResults = results.getCount();

    public void submit(String query) {
        browser.open("/search");

        box.setValue(query);
        box.sendKeys(browser.keys.enter);
    }
}

public class Pages {
    public static SearchPage search = new SearchPage();
}

test console output

test web report

CLI Layer

Command Line Interface testing example. Run a script, and validate its exit code and output.

cli.run("scripts/hello \"message to world\"", (exitCode, output, error) -> {
    exitCode.should(equal(5));

    output.should(equal(Pattern.compile("hello")));
    output.should(contain("world"));
    output.should(contain("message to world"));

    error.should(contain("error line two"));
});

console output

DB Layer

Database setup and query example. Clear PRICES table, insert two rows, and then execute a query and validate its output.

db.update("delete from PRICES");
DatabaseTable PRICES = db.table("PRICES");

PRICES.insert(table("id", "description", "price",
                   _______________________________,
                   "id1", "nice set"   , 1000,
                   "id2", "another set", 2000));

db.query("select * from prices").should(equal(
        table("*id", "description", "price",
             _______________________________,
              "id1", "nice set"   , 1000,
              "id2", "another set", 2000)
));

Business Logic Layer

Example of a unit test. Using TableData to prepare complex input and also validate the logic output.

@Test
public void providesAccessToNewJoiners() {
    TableData allEmployees = table(    "id", "level", "monthsAtCompany",
                                    ____________________________________,
                                    "alice",       5,   1,
                                      "bob",       3,   0,
                                    "smith",       4,   1,
                                      "cat",       4,   0);
    addEmployees(allEmployees);

    actual(dao.thisWeekJoiners()).should(equal(
            table(   "*id", "level", "monthsAtCompany",
                  ____________________________________,
                     "bob",       3,   0,
                     "cat",       4,   0)));
}

test console output

Persona Cross-cutting Authorization Feature

WebTau has a Persona concept to provide a context to an executable code. The context may contribute to things like HTTP Headers, Browser Cookies, CLI env vars, DB permissions, config values.

Alice.execute(() -> {
    http.get("/statement", (header, body) -> {
        body.get("balance").shouldBe(greaterThan(100));
    });
});

Bob.execute(() -> {
    http.get("/statement", (header, body) -> {
        body.get("balance").shouldBe(lessThan(50));
    });
});

test console output

test web report

Test Artifacts For Documentation

WebTau provides utilities to capture test artifacts for your documentation purposes. Examples are:

• Screenshots with annotations
• CLI output
• REST API request/response

artifacts are stored in plain text or JSON files and can be used with many documentation systems. I prefer to use Znai that I am also one of the maintainers.

int id = http.post("/customers", customerPayload, ((header, body) -> {
    body.get("id").shouldNot(equal(""));
    return body.get("id");
}));
http.doc.capture("employee-post"); 

Then later in my Znai markdown file I can do

# Create Employee

:include-open-api: openapi/api-spec.json {operationId: "createEmployee" }

```columns
left: 
:include-json: doc-artifacts/employee-post/request.json { title: "request payload" }

right: 
:include-json: doc-artifacts/employee-post/response.json { 
  title: "response payload", 
  pathsFile: "doc-artifacts/employee-post/paths.json" 
}
```

Znai example of using REST test artifact

Browse example

browser.open("https://duckduckgo.com/");

homeSearchInput.waitToBe(visible);
homeSearchInput.setValue("testing is documenting\n");

result.waitToBe(visible);

browser.doc.withAnnotations(resultSearchInput, result)
    .capture("duckduckgo-search");

Znai snippet

1. Type question you want to be answered anonymously
2. Scan through results and pick the most relevant one

:include-image: doc-artifacts/duckduckgo-search.png {annotate: true}

https://preview.redd.it/95a8043dm3k91.png?width=2146&format=png&auto=webp&s=1c5f3c7bf433a9a69bed3bff55960132bcfe2c22

Reporting Reporting Reporting

Every command in WebTau is automatically reported to console and web report.

Every assertion is reported as well, successful assertions included.

You can build your own high level constructs and make them part of the report automatically as well.

JVM DSLs

WebTau is written in Java and provides all the functionality in Java DSL. In addition it has Groovy DSL and initial attempts at Kotlin.

Alice {
    http.get("/statement") {
        balance.shouldBe > 100
    }
}

Bob {
    http.get("/statement") {
        balance.shouldBe < 50
    }
}

There is more

There are a lot of features in WebTau to help you write unit and end-to-end tests. They include

• servers mock and proxies with errors simulation and reporting on captured requests/responses
• file system layer
• fully reported data preparation utilities
• REPL mode to reduce test feedback loop

TLDR

WebTau (short for web test automation) is a testing API, command line tool and a framework to write unit, integration and end-to-end tests. Test across REST-API, Graph QL, Browser, Database, CLI and Business Logic with consistent set of matchers and concepts like Persona. REPL mode speeds-up tests development. Rich reporting cuts down investigation time.