Configuring Cypress To Run On Different Environments

With larger, more complex applications and infrastructure, more intricate CI pipelines with various steps and stages and so on, the need to have your Cypress tests run on different environments often arises.

Typically, our integration tests run in a dedicated build step once we’ve deployed to Development and that’s that for tests that run on the pipeline. As the build goes through the subsequent environments (Staging -> Pre-Prod -> Prod etc.) we carry out other forms of release testing on that particular environment. Great. However, when a bug appears to have made it to the Staging environment for example, or even Pre-Prod, the Pull Request that contains the patch or fix might be merged directly to the Staging environment. Why? well:

• We might not want to go through all of the previous builds and steps in the release pipeline/process again for a hotfix
• The Development environment could already contain new features/work that may conflict with the hotfix and consequently prove harder to test

However, this could mean that the integration tests which solely run on the Development environment would be made redundant, as these tests are not configured to run on the Stage environment where this hotfix has been applied to.

What if this hotfix introduces other bugs that our existing integration tests may catch? Ultimately, you have a bit of functionality, as small as it may be, merged in and only covered by unit tests.

Alternatively, what if you need to assess the dependencies that you rely on with these releases across each environment as the dependencies themselves change on an environment basis. What if you have some kick-ass E2E tests which interact with the data-layer as well as the cloud services the product uses which you would like to test in each environment? It is questions such as these which ultimately led us to investigate the possibility of running our integration tests on each environment, and luckily, Cypress has an awesome Configuration API which we leveraged to accomplish our goal of running tests across multiple environments dynamically, both locally and on the pipeline.

Cypress already has a great bit of documentation on the Configuration API and much of what is demonstrated below follows the examples set in the documentation on this API. All this guide really is, is an extension of this documentation with examples on how to incorporate configuration files into your test projects and run them.

I have put together a basic repository where the demo code is here.

Setting up the Configuration Files

You might be using the cypress.json or a cypress.env.json file where your baseURL and environment variables are. What you want to do instead is setup a new directory within the Cypress directory which contains a JSON file for each environment and holds the environment variables and baseURL for that specific environment.

Potential project hierarchy:

- ProjectFolder
-- cypress
--- configFiles
---- development.json
---- stage.json
---- production.json
-- fixtures
-- integration
--- specs
-- plugins
-- support
- package.json
- node.modules
- cypress.json

Example Environment Config Files:

development.json

stage.json

production.json

Pay close attention to the “env” node. This is where Cypress will look for environment variables and effectively where you would place environment variables that are required for that particular environment. For example, your development user might not function accordingly or even authenticate on the staging environment, you would thus put the credentials required for the staging user in the stage.json file and do the same for development.json and production.json. It doesn’t have to be credentials, its just key and values you need for that particular environment.

The baseURL will now also be read from these configuration files instead of the cypress.json file. You must specify the baseURL’s for each environment in the correct file and effectively remove the baseURL from cypress.json. In the demo files above, you can see that the domain changes in each environment, .org, .net and .com. In reality, this would be mywebsite.development.com, mywebsite.stage.com, and so on.

Telling Cypress About the Config Files

Once you’ve setup the environment config’s, you need to tell Cypress about these files. The place to do this is in the index.js file within the plugins folder that is generated when you create a Cypress project. The index.js file is one of the first things that executes when you run or open Cypress.

A simple breakdown of what is happening in the snippet above:

• getConfigurationbyFile, a function to retrieve files from the path that is specified
• Using the Cypress Configuration API, the fileConfig environment variable is looked for and set to the file constant at line 20
• If this environment variable fileConfig does not exist or is not specified, it will use the value of development instead using the OR (||) operator
• The getConfigurationbyFile function is called and we specify the file as a parameter for this function

Note, that the we pass the function either development, stage or production as the argument. The function will append .json to it as can be seen at line 13. For example, if we were to run getConfigurationbyFile(“stage”), the function will effectively look in the directory specified at line 12 for a file called stage.json.

Once the fileConfig variable is set, Cypress will resort to using this file to fetch environment variables from as well as the baseURL.

Setting up the Node Scripts

Within the package.json file, under the “scripts” node, we can create a number of scripts which contain different configFile values. Each script has a different value for the configFile environment variable, corresponding to the environment names.

Cypress is setup to accept arguments from the command line and we can specify an environment variable we’d like Cypress to use like so:

--env [environmentVariableName]= value

In our case, we are stating what the value for the fileConfig environment variable is when we execute this script; this will then use this argument and pass it back to the getConfigurationbyFile function, which then looks for a configuration json file with the value set for fileConfig.

To run Cypress on your machine against the Stage environment you would effectively run:

npm run cy:stage

and against Production, it would be:

npm run cy:production

Here is a a bit of documentation on Cypress command line arguments.

Dynamically Switching Configuration Files, CI/CD

This is namely for when you need to run tests on the pipeline across different environments. Unfortunately, this is not something that I can publicly demonstrate, especially as it may not particularly apply to the CI/CD service that you use. However, the objective is to:

1. Have environment variable values for each environment persisted in the pipeline/CI files which have the exact same value as the name of the config file stored in the Cypress project.
2. Fetch and pass these values down to a Cypress environment variable called fileConfig. To do this, you need add a prefix of CYPRESS_ to the environment variable. Cypress will automatically look for environment variables that have a prefix of CYPRESS_, more on Cypress environment variables here.
3. Ultimately, what you want, is to set the fileConfig environment variable to the required value. So if your test suite is about to run on Development, you need to set CYPRESS_fileConfig = “development”. Once again, the getConfigurationbyFile function will read the value of this variable and use it to determine which environment configuration file to use. Once your build is deployed to Stage, you want to do the same but have the value of fileConfig be stage, which will run the Cypress configuration specified in the stage.json file.
4. Alternatively, you can setup another Node script which is solely used in the pipeline and execute that script with an argument when you run Cypress in the pipeline

Results

Screenshots from my local machine running the demo project.

Running Development:

npm run cy:development

Note the contents of the Settings tab under Configuration using the environment variables from the specified environment config file:

Test:

Running Production:

npm run cy:production

Cypress.json

Your cypress.json file should now only contain:

• Environment variables that are shared/used across all environments
• Command timeouts
• Viewports width/height
• Other Cypress options you want to use in every environment

Conclusion

Whether you need to configure your tests to run on each environment depends on your confidence in these tests and how much value they add by running on all environments. It might slow down your time to market, feedback loop and so on. It might also be completely overkill. This is up to yourself and the team you work with to decide, but the ability to very easily have Cypress run against Stage or Production on your machine locally without needing to adjust the set configs or baseURL’s is definitely great.

I hope this has been useful, thanks for reading!