Tim Deschryver

Improve your test experience with Playwright Soft Assertions

profile
Tim Deschryver
timdeschryver.dev

In this blog post, we take a look at Playwright's (soft) assertions to improve the test report. Going from this report which doesn't give a lot of information:

The Playwright test report only includes the first failure message because the first name has a wrong value.

To a better experience like this:

The Playwright test report includes all failure messages.

Soft assertion is a feature that I frequently use in the .NET world, which I blogged about in Assertion Scopes, the one FluentAssertions feature you didn't know you needed.

So what does this mean and why should you care?

With soft assertions, you can write multiple assertions in a single test without having to worry that one of them fails and bail out of the test. Instead, you can verify of all assertions are successful when you're interested in the outcome.

Compare both behaviors, and you can see how soft assertions can improve the flow of writing (and fixing) test cases.

Let me give you an example of where I think soft assertions are useful.

Let's say that we're writing an application that contains a form in which you want to verify that all fields are prefilled correctly. A test for this looks like this, we get the form fields and use expect to check that each field has the correct value.

Now, imagine one of the form fields that has an incorrect value. Of course, causing the test to fail.

The Playwright test report only includes the first failure message because the first name has a wrong value.

With this behavior, you need to fix the bug and rerun the test with the hope that the other assertions are passing for the test case to succeed. But of course... there is another bug, and again, the test case fails. This repetitive loop goes on and on until the test finally turns green. I find this counterproductive and it can also demotivate me, especially after a hard day.

Luckily there's a solution to this problem.

By using soft assertions in Playwright, the cycle of fixing the assertions is fine-tuned to just one single step. The test can be refactored to the following, where we swap expect with expect.soft.

The refactored version results that the test case continuing to execute even if one or more assertions fail. In the next image, we can see that all assertions are executed and that all fields are incorrect.

The Playwright test report include all failure messages.

When using soft assertions in Playwright the test's outcome is verified when the test case is fully executed. This is not always the behavior that you want. In some cases, you want to bail early when you detect that the state of the application is not what you expect it to be. For example, you don't want to submit the form when it's not filled in correctly because you already know that it will fail. Bailing early saves you time and resources.

To prevent the test to run further you need to use test.info().errors, which contains a collection of errors for the current test. When the collection is not empty, you can throw an error to stop the test execution.

Outgoing links

Follow me

You can find me on Twitter, LinkedIn, and GitHub.

Support me

I appreciate it if you would support me if have you enjoyed this post and found it useful, thank you in advance.

Buy Me a Coffee at ko-fi.com PayPal logo
Support the blog Share on Twitter Discuss on Twitter Edit on GitHub