E2E testing: Cucumber and Playwright

December 6, 2024

Nowadays, it is quite important to make sure that the managerial level of the company has a full understanding of what kind of testing the QA teams are performing. For a person without any technical background it might get frustrating looking at the complex code. To understand what kind of testing has been done, some help from software engineers is definitely needed. To fill in this gap, we have a BDD (Behavior Driven Development) testing framework. BDD framework simplifies the complex and robust test suites into human readable text. One of the most popular tools in BDD is Cucumber.

What is Cucumber?

Cucumber is an open-source software testing tool. In simple terms, Cucumber enables you to write test cases that most people can understand even without any technical background. The secret sauce is the simple, yet powerful syntax - Gherkin. The main focus of Gherkin language is to define the "behavior" rather than the "technical tests". Let's have a look at the example below:

Feature: Web search

Scenario: Search for rental cars

    Given the Google.com is open
        When I search for rental cars
        Then Search result for rental cars is visible

Firstly, we create a file with the name of webSearch.feature. The feature file in cucumber holds the test scenarios and the sequence of the test steps (defined as Given, When, Then steps). The scenario name represents the current test suite case of what we are trying to test.

What is Gherkin?

Gherkin is a set of grammar rules that makes plain text structured enough for Cucumber to understand. The scenario above is written in Gherkin. Gherkin documents are stored in .feature text files and are typically versioned in source control alongside the software.

Gherkin serves multiple purposes:

Unambiguous executable specification
Automated testing using Cucumber
Document how the system actually behaves

As you can see, it is pretty self explanatory and it is clear what is happening in this scenario. Firstly, we have a Given step, where we provide the test some background on what is given to start with, Next, we have a When step, which is being executed after the Given. Lastly, we have Then step, where appropriate logic will be executed at this step. It is worth to mention that, there can be multiple Given-When-Then steps within a scenario, which can be repeated using And or * symbol.

Each keyword has their purpose. As a person with no technical background, you can understand it well as plain text.

Given: What we have
When: What we do
Then: What's the expected result

Feature: Web search

Scenario: Search for rental cars

    Given the Chrome browser is open
  * The Google.com is open

    When I search for rental cars
    And when I select image search

    Then Search result for rental cars is visible
  * Search result contains pictures

While this approach can be really helpful, one might say that this method of writing test cases can become redundant. To avoid it, Cucumber allows us to run parameterized tests, which can be run in parallel mode (multiple test scenarios at once). This way, we can speed up the test runs and perform different tasks with reusable steps. Next picture represents a parameterized test scenario, where we can pass our own parameters in to test logic:

Feature: Web search
  
  Scenario Outline: Search for items
  
    Given the Chrome browser is open
  * the Google.com is open

    When I search for <item>
  And when I select image search

  Then Search result of <item> is visible
    * Search result contains pictures

    Examples:
    |item              |
    |cars              |
    |fruit             |
    |office supplies   |
    |laptops           |
    |groceries         |

Firstly, we have to let Cucumber know that this scenario should run on a loop using our Examples given below. Then it replaces the strings with passed parameters.

Cucumber is widely used throughout the QA practice and is one of the leading BDD frameworks these days. It can be combined with various other frameworks to support the logic of the tests. The picture below shows Cucumber's ability to be combined with most modern frameworks.

Since we already have .feature files which consist of the behavior that should be executed, we need a tool to be able to automate this process and actually perform the activities needed to fulfill the test suites. As seen earlier, there can be many different combinations with Cucumber, but for this demonstration purpose, we will use a modern and rapidly growing Playwright framework.

What is Playwright?

While Cucumber feature files possess the behavior of the tests, we need a tool to be able to perform a logic based on what we want to do during the test. Here comes the Playwright.

Playwright is an open-source test automation library initially developed by Microsoft contributors. It supports programming languages such as Java, Python, C#, and NodeJS. Playwright comes with Apache 2.0 License and is most popular with NodeJS with which results in tests being able to be written in Javascript or Typescript.

I have chosen Playwright, not just because it might seem a comfortable tool, but with the features it comes with:

Easy installation and configuration
Multi-Browser Support
Multi-Language Support
Parallel Browser Testing
Support for Multiple Tab/Browser Window
Built-in Reporters
CI/CD Integration Support

Ever since I started to work with Playwright, few of the features made automation much easier. Firstly, the Playwright allows you to use multiple Contexts within the testing suite, which results in a better way of handling each browser's cache and session data. It allows testing the software with particular data related to the session. In addition, it allows you to work around the popups and it has integration with most modern browser engines: Chromium, WebKit and Firefox.

If you are still hesitating in using Playwright, let me give you a couple more examples, why it might be a great fit for your project. Playwright supports cross-platform, which allows you to test on Windows, Linux, and macOS, locally or on CI, even headless or headed! It can be really handy to test modern JavaScript framework components, since Playwright selectors pierce shadow DOM and allow entering frames seamlessly.

Also, Playwright has a tool called Codegen, which can be really useful to visually perform the actions needed, which will be returned to you in any selected language. This tool will allow you to generate tests quickly.

Since now we know what are the benefits of Cucumber and Playwright, let's put it in to use and see how it works in action!

Cucumber and Playwright integration

In order for Cucumber and Playwright to work together, we need to add NodeJS to our project. It will give Playwright the ability to execute JS which will be defined as our test logic. After installing NodeJS, Cucumber and Playwright in our project, it is time to try and run our previously created feature file.

1) Scenario: Search for items # features\future.feature: 18
    ✔ Before # cucumber.conf.js: 38
  ? Given the Chrome browser is open
Undefined.Implement with the following snippet:

Given('the Chrome browser is open', async function () {
              // Write code here that turns the phrase above into concrete actions
              Return 'pending';
            });
    ?  * the Google.com is open
Undefined.Implement with the following snippet:

Given('the Google.com is open', async function () {
              // Write code here that turns the phrase above into concrete actions
              Return 'pending';
});

As you can see above, Cucumber test runner indicates to us that we have feature files written, however, we do not have logic to be executed. In simple words, we need to create a steps.js file, where we will store each step logic and using Playwright, execute the code, so let's go ahead and create these steps.

Given('the Chrome browser is open', async function () {
  Browser = await chromium.launch({
    Headless: false,
  });
  context = await browser.newContext();
  page = await context.newPage();
});

The picture above shows how the steps should be implemented. Inside curly braces we include our code to be executed in the step. For demo purposes, I have taken the very first given step. Here we define our browser, context and page, which can carry different cache and session data. The other steps should contain the logic that we want to execute within the steps, which can be seen on the next image.

Given('the Chrome browser is open', async function () {
  Browser = await chromium.launch({
    Headless: false,
  });
  context = await browser.newContext();
  page = await context.newPage();
});
Given('the Google.com is open', async function () {
  await page.goto('https://www.google.com');
});

When('I search for {string}', async function (string) {
  // Fill search input [aria-label="Search"]
  await page.locator('[aria-label="Search"]').fill(string);
  await page.locator('input[type="submit"]').click();
});

When('I select image search', async function () {
  // Click images search
  await page.locator('a:text-is("Images")');
});

Then('Search result contains pictures', async function () {
  expect(await page.locator('.search-result')).toBeVisible();
});

Then('Search result contains pictures', async function () {
  expect(await page.locator('.search-result.img')).toBeVisible();
});

As you can see, every step now contains logic and the test scenario is ready to be run. Now, to explain the flow of the logic: the Cucumber test runner will execute the feature file. Feature file explain Cucumber, the order of the steps they should be executed. Then It looks for the steps and executes the logic in each step. At the very end, we get the result of how things went in our test scenario.

.....
1 scenario(1 passed)
3 steps(3 passed)
0m33.611s(executing steps: 0m32.015s)

Advanced tips and tricks

One must mention that a wide variety of customization is available using Cucumber and Playwright. Probably one of the most useful features in Cucumber test runner is profiles. Using profiles, we can set a particular data according to our test needs. This can be very useful while testing different environments of the software.

Also, it is worth mentioning that Playwright lets us create separate browser contexts, which can hold particular data for testing purposes. Session is being active until the same context is active and to create a new browser context is just a simple one-liner. In addition, Cucumber lets us run tests in parallel mode. This means that multiple tests can be run at the same time, which greatly reduces the duration of testing.

The last feature of Playwright I wanted to mention is ability to execute particular code before or after each, or all tests. This way, we can simply tell Playwright to create a new browser context every time we run a new test. This will assure our session is limited to every test scenario.

To conclude, I must say that Cucumber and Playwright combination is lightweight, fast and has many customization options to suit your project's requirements. It is easy to learn if you have some basic JavaScript background. Using this combination of Cucumber and Playwright will assure that the software is working as expected and flaws will be discovered immediately.