Cypress API Testing — Schema Validation with AJV

Reaching a higher level of automated API testing

Schema Validation

As we query the various endpoints as part of an API we are typically given a response. This response follows a specified schema set by the developers when they built the API and is traditionally returned as JSON. There are different schemas for different endpoints and even schemas based on the status of the API call. As QA, we could/should go the extra distance and verify that the schema returned is valid.

Another JSON Schema Validator (AJV)

Cypress has a marvelous post that you can read through on schema validation using JSON Schemas here

Personally, I will be using AJV which is one of many schema validation tools available, it’s well maintained and it’s excellent documentation can be found here.

Setup

I have setup a very basic project to demonstrate schema validation, here is the link. This demo project queries the free-to-use and awesome cat-facts API.

Walkthrough

In the demo project, you will find a few custom commands. For instance, the following snippet is an API helper custom command. It accepts two query parameters, one for the animal type and an optional param for the number of facts to get. This custom command fires off a GET request which populates the queryParams (url variables) using Cypress’ qs option.

Within the animal-facts.spec, I have used the above custom command to fire off a request to the cat-facts API and once the request is resolved, a few assertions run.

When the project runs, look at the API response by opening up the dev console and clicking on the request in the Cypress dashboard on the left. You should see the following:

Effectively, the above is the response the API returned, and what we want to do is verify the contents of this response using AJV.

Implementation

For the sake of this demonstration, we will only be setting up a schema check on the body of the of the response. However, you can go further and also verify the response headers should you wish to do so.

To get started, you want to setup a js file that follows the exact same schema as the body of the API response, such as:

A few important bits to understand on the above snippet:

• the first declaration of type is the datatype for the overall schema that you want to test and one must be declared
• properties are the keys in the response, such as deleted or status
• properties can be nested and each property should have a type
• assume that the types we specify is us telling AJV what datatype to expect with the corresponding property, for example, deleted should always be a boolean (true/false)
• enums can be used to inform AJV that only the specified values should return with a particular property
• enums should only be used when you know precisely which values should return
• patterns are regex patterns and a great way to describe the pattern of the value, for example you may have an id that should only return x number of characters
• required is an array that goes at the end of the schema and is a way of telling AJV that the listed members should always exist/return in the response schema

Schema Re-usability

When you have multiple schemas up and running and a number of them share the same properties, it might be wise to setup a custom schema definitions file which can be used to persist and store schema definitions that can be coupled and referenced from other schemas. In the snippet above, you may have noticed the $ref option. This can be used to reference the mentioned schema definitions.

The following example is a definition which can be referenced and reused with dateTime properties:

Ensure you specify the correct path when you reference a custom definition, this took me a while to figure out!

$ref: "customDefinitions#/definitions/dateTime"

id -> definitions -> name of definition to use

AJV Custom Command

Using the AJV library, I’ve put together a custom command to easily reuse the schema validation across various tests.

• validateSchema takes in two parameters; the schema you defined locally and the schema from the API
• validate combines the custom definitions schema along with the given local schema, enabling our references to the custom definitions called from the local schema to function correctly
• valid is where the magic happens and compares the local schema to the response schema
• if any errors occur, the whole error object is passed down to the getSchemaError function which returns a clear representation of what the issue is
• an error must be thrown to fail the test if the schema has errors, otherwise Cypress will pass the test

Running the Tests

Import the local schema and supplement the validateSchema custom command with the correct parameters in your test, making sure to pass the response schema at the correct level using dot or bracket notation; remember, the custom command we setup is a like-for-like comparison, so your response schema should start from the same place as your defined local schema, in this case, we set it up to validate the contents of the response.body.

When changing the expected datatype for the deleted property from a boolean to a string, AJV should throw an error and tell you it expected a string from the response API (as we told AJV in the local schema that we are expecting a string now)

When a value isn’t included in the defined enum for a given property, AJV will tell us that the value it received from API is not part of the defined values set in the local schema.

When a property in the required array doesn’t exist in the response schema, AJV will inform us that we declared a specific property as being required, yet it could not find this property in the response schema.

Conclusion

Other than gaining additional confidence in your API tests, Schema Validation is a great substitute for in-line response key assertions using Cypress & Chai; when properly configured, it saves you allot of bloat and reduces repetition.

Thanks for reading!