Generated tests with XState and Cypress
This article is inspired by the talk "Write Fewer Tests! From Automation to Autogeneration" given by David Khourshid. The talk showcases XState's new library @xstate/test, which helps you to write or, better yet, not write tests.
During the talk, David writes tests for a React component using Jest and Puppeteer. Because I'm a Cypress and XState fan, I wanted to rewrite the same test, but this time using Cypress for fun and practice. As a starting point, I'm re-using the source code of the demo. The demo application is a multi-step form.
A reusable state machine across tests link
@xstate/test adds a
test property to the
meta property for each state. The
test property accepts a callback and, in this callback, we can write an assertion against the current state of the application. When a test has run, it will invoke the
test callback to verify that the application is in the correct state.
The state machine in the demo application adds the test code directly within the state machine. This couples the state machine to a specific testing library.
One of the advantages of state machines is that these are library/framework agnostic, which I highlighted in a previous article "My love letter to XState and statecharts ♥". To make the state machine reusable across different testing libraries, the first step I took was to extract the test code from the state machine. By doing this, we also exclude the test code from the production build.
Configuring the states for Cypress link
The second step was to re-add the
test properties to the appropriate states.
For this, I created a helper function
addTests to make the test readable and simple.
This helper function, will iterate over all states and add the test meta property to each state.
In the tests, we create a new state machine by using the same states and by adding a test case for each state. The argument that you can already see in each test case will be defined during the next steps.
As you can see, these test cases seem a lot like the existing tests. This is because both tests are written using Testing Library.
Generating test plans link
Because the entire application is represented with a state machine, it is possible to calculate the next possible states.
By using algorithms, for example Dijkstra's algorithm,
@xstate/test generates a path to end up in a specific state. It does this for each possible state.
This means that by using this method, we can test every state of the application. In practice, it will probably generate tests that will end up into states you haven't thought of.
In the snippet below we use the
createModel method to create the test model, and the
testModel.getSimplePathPlans method to generate the tests for the feedback machine. Next, we iterate over each generated test plan, create a test for the plan, and assert that test with the
path.test method. Here, we pass the
cy variable as the argument to the test method. Because we imported
findBy query commands will be added to the global
cy variable, which makes them available to use in the tests.
Interacting with the application link
To interact with the application we re-use the events of the real state machine. The generated test plans will execute these events and wait untill they are executed. Afterward, the test plan verifies if the application is in the correct state.
To configure the events during the tests, we must use the
withEvents method on the test model.
It's here where we interact with the application, for every event.
To verify that the generated test plans cover each possible state, the test model has a
This will throw an error, and the error message will say which state node is missing from the coverage.
This way of writing tests will take some time to getting used to, but I can already see how this can be useful. It gives me confidence that the whole application is tested and that there won't be any uncovered states that I haven't thought of. Writing these tests is simple, and it doesn't take a long time to write them. Especially, in comparison to writing manual tests.
Herein lies the true power of both libraries, and this emphasizes what I believe in. Which is, that we shouldn't care which framework and libraries are used to build an application. This is also why I like Cypress because it hides the implementation details.
For a more detailed explanation with more possibilities and advantages, I'm referring you to the docs and the article Model-Based Testing in React with State Machines, written by the same David.
The entire Cypress test looks as follows, and the entire code for this article can be found on GitHub.
I appreciate it if you would support me if have you enjoyed this post and found it useful, thank you in advance.