Zebrunner Cypress reporting agent#

The official Zebrunner Cypress reporting agent.

Info

The Agent supports all versions of Cypress starting from v9, the versions below may not work. Since v9/v10+ versions have configuration differences, please refer to the corresponding version-specific tabs for more details.

Inclusion into your project#

Adding dependency#

First, you need to add the Zebrunner Agent into your package.json.

YarnNPM

yarn add @zebrunner/javascript-agent-cypress

npm install @zebrunner/javascript-agent-cypress

Reporter setup#

The agent does not work automatically after adding it into the project, it requires extra configuration. For this, you need to perform the following 3 steps:

Enable reporting plugin in your Plugins file:

Cypress 10+Cypress 9

cypress.config.js

const { defineConfig } = require('cypress');
const zbrPlugin = require('@zebrunner/javascript-agent-cypress/lib/plugin');

module.exports = defineConfig({
    e2e: {
        setupNodeEvents(on, config) {
            zbrPlugin(on, config);
        },
    },
});

cypress/plugins/index.js

const zbrPlugin = require('@zebrunner/javascript-agent-cypress/lib/plugin');

module.exports = (on, config) => { zbrPlugin(on, config); }

Add reporting commands by including the following in your Custom commands file (e.g. cypress/support/commands.js):

require("@zebrunner/javascript-agent-cypress/lib/commands/commands");

In order to enable browser registration in Zebrunner, include the following code in your Support file (e.g. cypress/support/index.js for v9 or cypress/support/e2e.js for v10) file where cy.registerBrowser is a custom command provided by Zebrunner:

import "./commands";

before(() => {
  cy.registerBrowser(Cypress.browser);
});

Reporter configuration#

Once the agent package is included and enabled in your test project, the valid configuration must be provided.

It is currently possible to provide the configuration via:

Environment variables
cypress.json / cypress.config.js config file depends on your Cypress version (v9 / v10 accordingly)

The configuration lookup will be performed in the order listed above. It is also possible to override configuration parameters by passing them through a configuration provider with higher precedence.

via environment variables#

The following configuration parameters are recognized by the agent:

Parameter	Description
`REPORTING_SERVER_HOSTNAME`	Mandatory if reporting is enabled. It is Zebrunner server hostname. It can be obtained in Zebrunner on the 'Account & profile' page under the 'Service URL' section
`REPORTING_SERVER_ACCESS_TOKEN`	Mandatory if reporting is enabled. Access token must be used to perform API calls. It can be obtained in Zebrunner on the 'Account & profile' page under the 'API Access' tab
`REPORTING_PROJECT_KEY`	Optional value. It is the project that the test run belongs to. The default value is `DEF`. You can manage projects in Zebrunner in the appropriate section
`REPORTING_LAUNCH_DISPLAY_NAME`	Optional value. It is the display name of the test run. The default value is the name of first executed test file name
`REPORTING_LAUNCH_BUILD`	Optional value. It is the build number that is associated with the test run. It can depict either the test build number or the application build number
`REPORTING_LAUNCH_ENVIRONMENT`	Optional value. It is the environment where the tests will run
`REPORTING_SLACK_CHANNELS`	Optional value. Predefined list of Slack channels for results notifications
`REPORTING_EMAIL_RECIPIENTS`	Optional value. Predefined list of recipients for Email notifications
`REPORTING_LAUNCH_TREAT_SKIPS_AS_FAILURES`	Optional value. If the value is set to true, skipped tests will be treated as failures when the result of the entire launch is calculated. For example, launch with all passed tests but one skipped will be considered a failure.
`REPORTING_MILESTONE_ID`	Optional value. Id of the Zebrunner Milestone to link the automation launch to. The id is not displayed on Zebrunner UI, so the field is basically used for internal purposes. If the milestone does not exist, the launch will continue executing.
`REPORTING_MILESTONE_NAME`	Optional value. Name of the Zebrunner Milestone to link the automation launch to. If the milestone does not exist, the appropriate warning message will be displayed in logs, but the test suite will continue executing.

via Cypress config#

In order to configure agent via Cypress config (cypress.json / cypress.config.js), reporter configuration needs to be provided.

Please see reporter configuration snippet below:

Cypress 10+Cypress 9

cypress.config.js

const { defineConfig } = require('cypress');
const zbrPlugin = require('@zebrunner/javascript-agent-cypress/lib/plugin');

module.exports = defineConfig({
  reporter: '@zebrunner/javascript-agent-cypress',
  reporterOptions: {
    reportingServerHostname: '<YOUR_ZEBRUNNER_SERVER_URL>',
    reportingServerAccessToken: '<YOUR_ZEBRUNNER_ACCESS_TOKEN>',
    reportingProjectKey: 'DEF',
    reportingLaunchEnvironment: 'STAGE',
    reportingLaunchBuild: '1.0-alpha',
    reportingLaunchDisplayName: 'My regression suite',
    reportingLaunchLocale: 'en_US',
    reportingSlackChannels: 'dev',
    reportingEmailRecipients: 'manager@mycompany.com'
    reportingLaunchTreatSkipsAsFailures: true,
    reportingMilestoneName: "",
    reportingMilestoneId: "",
  },
  e2e: {
    setupNodeEvents(on, config) {
      zbrPlugin(on, config);
    },
  },
});

cypress.json

{
    "reporter": "@zebrunner/javascript-agent-cypress",
    "reporterOptions": {
        "reportingServerHostname": "<YOUR_ZEBRUNNER_SERVER_URL>",
        "reportingServerAccessToken": "<YOUR_ZEBRUNNER_ACCESS_TOKEN>",
        "reportingProjectKey": "DEF",
        "reportingLaunchEnvironment": "STAGE",
        "reportingLaunchBuild": "1.0-alpha",
        "reportingLaunchDisplayName": "My regression suite",
        "reportingLaunchLocale": "en_US",
        "reportingSlackChannels": "dev",
        "reportingEmailRecipients": "manager@mycompany.com",
        "reportingLaunchTreatSkipsAsFailures": true,
        "reportingMilestoneName": "",
        "reportingMilestoneId": "",
    }
}

The following configuration parameters are recognized by the agent:

Parameter	Description
`reportingServerHostname`	Mandatory if reporting is enabled. It is Zebrunner server hostname. It can be obtained in Zebrunner on the 'Account & profile' page under the 'Service URL' section
`reportingServerAccessToken`	Mandatory if reporting is enabled. Access token must be used to perform API calls. It can be obtained in Zebrunner on the 'Account & profile' page under the 'API Access' tab
`reportingProjectKey`	Optional value. It is the project that the test run belongs to. The default value is `DEF`. You can manage projects in Zebrunner in the appropriate section
`reportingLaunchDisplayName`	Optional value. It is the display name of the test run. The default value is the name of first executed test file name
`reportingLaunchBuild`	Optional value. It is the build number that is associated with the test run. It can depict either the test build number or the application build number
`reportingLaunchEnvironment`	Optional value. It is the environment where the tests will run
`reportingLaunchLocale`	Optional value. Locale, that will be displayed for the run in Zebrunner if specified
`reportingSlackChannels`	Optional value. Comma separated list of slack channels for notifications.
`reportingEmailRecipients`	Optional value. Comma separated list of recipients for email notifications.
`reportingLaunchTreatSkipsAsFailures`	Optional value. If the value is set to true, skipped tests will be treated as failures when the result of the entire launch is calculated. For example, launch with all passed tests but one skipped will be considered a failure. notifications.
`reportingMilestoneName`	Optional value. Id of the Zebrunner Milestone to link the automation launch to. The id is not displayed on Zebrunner UI, so the field is basically used for internal purposes. If the milestone does not exist, the launch will continue executing.notifications.
`reportingMilestoneId`	Optional value. Name of the Zebrunner Milestone to link the automation launch to. If the milestone does not exist, the appropriate warning message will be displayed in logs, but the test suite will continue executing. notifications.

Launcher configuration#

Zebrunner Launcher provides a great way to execute tests without having to worry about the runtime environment.

Info

To learn more about the Zebrunner Launcher and all it's capabilities, refer to the Zebrunner Launcher documentation page.

The Cypress Agent is fully integrated with the Zebrunner Launcher and requires even less configuration when used with it. The Zebrunner Launcher automatically provides REPORTING_ENABLED, REPORTING_PROJECT_KEY, REPORTING_SERVER_HOSTNAME, REPORTING_SERVER_ACCESS_TOKEN and some other environment variables, so there is no need to explicitly specify them or the corresponding cypress.json / cypress.config.js file properties.

If you have included the Zebrunner Agent into your project and pushed the changes to a Git repository, you can easily run your tests using Zebrunner Launcher. For this, you need to navigate to Zebrunner Launcher and add the Git repository (if you have not already). Next, add a new Launcher for the Git repo.

Configuration of the Launcher for Cypress tests is pretty straightforward:

Add a meaningful name for the Launcher.
Select a Git branch to launch tests from.
Make sure the Zebrunner Executor is selected as the Execution Environment.
Select from the dropdown or fill in manually necessary CyServer docker image: public.ecr.aws/zebrunner/cyserver:latest.
Enter the launch command. Since each launch with Zebrunner Launcher starts from scratch, you need to install project dependencies as part of the launch command. The example of the launch command is npm install && npx cypress run --headed.
If necessary, add environment variables that will be passed to the tests at runtime.
For parallel execution it is necessary to define thread_count environment variable as INTEGER type with number of threads you would like to run.
Select Zebrunner Selenium Grid as Testing Platform along with operating system and/or desired browser.
If the launcher is configured, hit the Add button at the bottom of the page.

Note

It is recommended to put explicitly --headed argument in the launch command to display the browser (instead of running headlessly) for capturing videos.

Now you can launch the tests using Zebrunner CyServer. To do this, click the Launch button which is located under the configuration of the selected launcher.

If something went wrong while running the tests, you can examine the logs captured from the docker container. If the launch is stuck in "In Progress" status and there is no link to the logs file, you need to manually abort the launch in the 3-dot menu.

Running tests#

To run Cypress tests, you can use the following command in your terminal:

npx cypress run --browser chrome

Tracking test maintainer#

You may want to add transparency to the process of automation maintenance by having an engineer responsible for evolution of specific tests or test classes. Zebrunner comes with a concept of a maintainer - a person that can be assigned to maintain tests.

owner attribute should be set in your test implementation. Here is an example:

describe('some spec', () => {
  it('test name', {'owner': '<username>'}, () => {...})
})

The maintainer username should be a valid Zebrunner username, otherwise it will be set to anonymous.

Attaching labels#

To attach a label to a test, you need to invoke the cy.attachZbrTestLabel command in scope of the test method. To attach label to the entire launch, you can either invoke the cy.attachZbrLaunchLabel command.

The attachZbrTestLabel and attachZbrLaunchLabel commands accepts the key as first argument and values starting from the second argument.

describe("test-spec |", () => {
  before(() => {
    // will be attached to the entire launch
    cy.attachZbrLaunchLabel("label", "value");
  });

  it("first test", () => {
    cy.attachZbrTestLabel("tag", "ff");
    // ...
  });

  it("second test", () => {
    cy.attachZbrTestLabel("customLabelName", "smoke_test", "slow");
    // ...
  });
});

In this example:#

Label with label key and value value will be added to entire launch.

Label with tag key and ff value will be added to first test.

Labels with customLabelName key, smoke_test and slow values will be added to second test.

Collecting artifacts automatically#

By default, agent pushes to Zebrunner server screenshot for every test failure. Also, agent automatically sends video of entire spec execution to Zebrunner for every failed test. You may find them attached in the appropriate test results page.

Attaching custom artifact references to test and launch#

Labels are not the only option for attaching meta information to test and launch. If the information you want to attach is a link (to a file or webpage), it is more useful to attach it as an artifact reference (or to put it simply as a link).

The cy.attachZbrTestArtifactReference and cy.attachZbrLaunchArtifactReference commands serve exactly this purpose. These methods accept two arguments. The first one is the artifact reference name which will be shown in Zebrunner. The second one is the artifact reference value. attachZbrTestArtifactReference should be invoked in scope of the test method.

describe("test-spec |", () => {
  before(() => {
    // will be attached to the entire launch
    cy.attachZbrLaunchArtifactReference("Zebrunner", "https://zebrunner.com");
  });

  it("first test", () => {
    // will be attached to 'first test' only
    cy.attachZbrTestArtifactReference("SUT", "https://myapp.com/app");
    // ...
  });
});

Attaching custom artifacts to test and launch#

In case your tests or the entire launch produce some artifacts, it may be useful to track them in Zebrunner. The agent comes with convenient methods for uploading artifacts in Zebrunner and linking them to the currently running test or the launch.

The cy.attachZbrTestArtifact and cy.attachZbrLaunchArtifact commands serve exactly this purpose. This method accept two arguments. The first one is the artifact path on disk or Buffer instance. The second one is the artifact name which will be shown in Zebrunner. attachZbrTestArtifact should be invoked in scope of the test method.

describe("test-spec |", () => {
  before(() => {
    // will be attached to the entire launch and named "some_text_file1.txt"
    cy.attachZbrLaunchArtifact("<pathOnDisk>/some_text_file1.txt");
    // ...

    // will be attached to the entire launch and named "text1.txt"
    cy.attachZbrLaunchArtifact(
      "<pathOnDisk>/some_text_file1.txt",
      "text1.txt"
    );
  });

  it("first test", () => {
    // will be attached to 'first test' only and named "some_text_file1.txt"
    cy.attachZbrTestArtifact("<pathOnDisk>/some_text_file1.txt");
  });
});

Reverting test registration#

In some cases, it might be handy not to register test executions in Zebrunner. This may be caused by very special circumstances of a test environment or execution conditions.

Zebrunner Agent comes with cy.revertZbrTestRegistration() command for reverting test registration at runtime. The following code snippet shows a case where test is not reported on Monday. cy.revertZbrTestRegistration() should be invoked in scope of test method.

describe("test-spec |", () => {
  it("not important test", () => {
    if (new Date().getDay() === 1) {
      cy.revertZbrTestRegistration();
    }
    // test code
  });
});

Syncing test executions with external TCM systems#

Zebrunner provides an ability to upload test results to external test case management systems (TCMs) on test run finish. For some TCMs it is possible to upload results in real-time during the test run execution.

This functionality is currently supported for TestRail, Xray and Zebrunner Test Case Management systems.

TestRail#

If you want to push the execution results to TestRail, you need to provide additional configuration for the Agent using reporterOptions of your cypress.json / cypress.config.js file or environment variables as described below.

Env var / Reporter config	Description
`REPORTING_TCM_TESTRAIL_PUSH_RESULTS` `tcm.testRail.pushResults`	Boolean value which specifies if the execution results should be pushed to TestRail. The default value is `false`.
`REPORTING_TCM_TESTRAIL_PUSH_IN_REAL_TIME` `tcm.testRail.pushInRealTime`	Boolean value. Specifies whether to push execution results immediately after each test is finished (value `true`) or not (value `false`). The default value is `false`. Enabling of this option forces the `includeAllTestCasesInNewRun` to be `true`.
`REPORTING_TCM_TESTRAIL_SUITE_ID` `tcm.testRail.suiteId`	Specifies the numeric id of the TestRail Suite in which the tests reside. TestRail displays the ids prefixed with 'S' letter. You need to provide the id without this letter.
`REPORTING_TCM_TESTRAIL_RUN_ID` `tcm.testRail.runId`	The id of the TestRail Test Run where the results should be pushed. TestRail displays the ids prefixed with 'R' letter. You need to provide the id without this letter.
`REPORTING_TCM_TESTRAIL_RUN_NAME` `tcm.testRail.runName`	Specifies the name of a new Test Run in TestRail. If push is enabled and run id is not provided, Zebrunner will create a new run in TestRail. If the value is not provided, Zebrunner will use the launch display name.
`REPORTING_TCM_TESTRAIL_INCLUDE_ALL_IN_NEW_RUN` `tcm.testRail.includeAllTestCasesInNewRun`	If the value is set to `true`, all cases from the Suite will be added to the newly created Test Run. The value is forced to be `true` if real-time push is enabled. Default value is `false`.
`REPORTING_TCM_TESTRAIL_MILESTONE_NAME` `tcm.testRail.milestoneName`	The newly created Test Run will be associated with the milestone specified using this property.
`REPORTING_TCM_TESTRAIL_ASSIGNEE` `tcm.testRail.assignee`	Assignee of the newly created Test Run. The value should be the email of an existing TestRail user.

cypress.config.js

const { defineConfig } = require("cypress");
const zbrPlugin = require("@zebrunner/javascript-agent-cypress/lib/plugin");

module.exports = defineConfig({
  reporter: "@zebrunner/javascript-agent-cypress",
  reporterOptions: {
    // ...
    tcm: {
      testRail: {
        pushResults: true,
        pushInRealTime: true,
        suiteId: 18,
        runId: 20,
        runName: "",
        includeAllTestCasesInNewRun: true,
        milestoneName: "",
        assignee: "",
      },
    },
  },
  e2e: {
    setupNodeEvents(on, config) {
      zbrPlugin(on, config);
    },
  },
});

The Agent comes with following custom commands:

cy.testRailTestCaseKey(...testCaseKeys) — accepts a list of test cases which should be linked to the current test;
cy.testRailTestCaseStatus(testCaseKey, resultStatus) — links one test case and provides\overrides its result status. This may be useful if the test case result status does not correlate with the test execution status or if you have conditional logic determining the actual result status for the test case.

If these methods are invoked for the same test case id many times within a test method, the last invocation will take precedence. For example, if you invoke the cy.testRailTestCaseStatus('KEY-1', 'SKIPPED') first, and then invoke the cy.testRailTestCaseKey('KEY-1'), then the result status you provided in the first invocation will be ignored.

describe("some spec", () => {
  it("test name", () => {
    cy.testRailTestCaseKey("KEY-1", "KEY-2", "KEY-3");
    if (someCondition) {
      // overriddes the status of the test case when results are pushed to the Zebrunner TCM.
      // using this method, you can manually specify the desired result status.
      cy.testRailTestCaseStatus("KEY-2", "FAILED");
    }
  });
});

Xray#

If you want to push the execution results to Xray, you need to provide additional configuration for the Agent using reporterOptions of your cypress.json / cypress.config.js file or environment variables as described below.

Env var / Reporter config	Description
`REPORTING_TCM_XRAY_PUSH_RESULTS` `tcm.xray.pushResults`	Boolean value which specifies if the execution results should be pushed to Xray. The default value is `false`.
`REPORTING_TCM_XRAY_PUSH_IN_REAL_TIME` `tcm.xray.pushInRealTime`	Boolean value. Specifies whether to push execution results immediately after each test is finished (value `true`) or not (value `false`). The default value is `false`.
`REPORTING_TCM_XRAY_EXECUTION_KEY` `tcm.xray.executionKey`	The key of the Xray Execution where the results should be pushed.

cypress.config.js

const { defineConfig } = require("cypress");
const zbrPlugin = require("@zebrunner/javascript-agent-cypress/lib/plugin");

module.exports = defineConfig({
  reporter: "@zebrunner/javascript-agent-cypress",
  reporterOptions: {
    // ...
    tcm: {
      xray: {
        pushResults: true,
        pushInRealTime: true,
        executionKey: "",
      },
    },
  },
  e2e: {
    setupNodeEvents(on, config) {
      zbrPlugin(on, config);
    },
  },
});

The Agent comes with following custom commands:

cy.xrayTestCaseKey(...testCaseKeys) — accepts a list of test cases which should be linked to the current test;
cy.xrayTestCaseStatus(testCaseKey, resultStatus) — links one test case and provides\overrides its result status. This may be useful if the test case result status does not correlate with the test execution status or if you have conditional logic determining the actual result status for the test case.

If these methods are invoked for the same test case id many times within a test method, the last invocation will take precedence. For example, if you invoke the cy.xrayTestCaseStatus('KEY-1', 'SKIPPED') first, and then invoke the cy.xrayTestCaseKey('KEY-1'), then the result status you provided in the first invocation will be ignored.

describe("some spec", () => {
  it("test name", () => {
    cy.xrayTestCaseKey("KEY-1", "KEY-2", "KEY-3");
    if (someCondition) {
      // overriddes the status of the test case when results are pushed to the Zebrunner TCM.
      // using this method, you can manually specify the desired result status.
      cy.xrayTestCaseStatus("KEY-2", "FAILED");
    }
  });
});

Zephyr#

If you want to push the execution results to Zephyr, you need to provide additional configuration for the Agent using reporterOptions of your cypress.json / cypress.config.js file or environment variables as described below.

Env var / Reporter config	Description
`REPORTING_TCM_ZEPHYR_PUSH_RESULTS` `tcm.zephyr.pushResults`	Boolean value which specifies if the execution results should be pushed to Zephyr. The default value is `false`.
`REPORTING_TCM_ZEPHYR_PUSH_IN_REAL_TIME` `tcm.zephyr.pushInRealTime`	Boolean value. Specifies whether to push execution results immediately after each test is finished (value `true`) or not (value `false`). The default value is `false`.
`REPORTING_TCM_ZEPHYR_JIRA_PROJECT_KEY` `tcm.zephyr.jiraProjectKey`	Specifies the key of the Jira project where the tests reside.
`REPORTING_TCM_ZEPHYR_TEST_CYCLE_KEY` `tcm.zephyr.testCycleKey`	The key of the Zephyr Test Cycle where the results should be pushed.

cypress.config.js

const { defineConfig } = require("cypress");
const zbrPlugin = require("@zebrunner/javascript-agent-cypress/lib/plugin");

module.exports = defineConfig({
  reporter: "@zebrunner/javascript-agent-cypress",
  reporterOptions: {
    // ...
    tcm: {
      zephyr: {
        pushResults: true,
        pushInRealTime: true,
        jiraProjectKey: "",
        testCycleKey: "",
      },
    },
  },
  e2e: {
    setupNodeEvents(on, config) {
      zbrPlugin(on, config);
    },
  },
});

The Agent comes with following custom commands:

cy.zephyrTestCaseKey(...testCaseKeys) — accepts a list of test cases which should be linked to the current test;
cy.zephyrTestCaseStatus(testCaseKey, resultStatus) — links one test case and provides\overrides its result status. This may be useful if the test case result status does not correlate with the test execution status or if you have conditional logic determining the actual result status for the test case.

If these methods are invoked for the same test case id many times within a test method, the last invocation will take precedence. For example, if you invoke the cy.zephyrTestCaseStatus('KEY-1', 'SKIPPED') first, and then invoke the cy.zephyrTestCaseKey('KEY-1'), then the result status you provided in the first invocation will be ignored.

describe("some spec", () => {
  it("test name", () => {
    cy.zephyrTestCaseKey("KEY-1", "KEY-2", "KEY-3");
    if (someCondition) {
      // overriddes the status of the test case when results are pushed to the Zebrunner TCM.
      // using this method, you can manually specify the desired result status.
      cy.zephyrTestCaseStatus("KEY-2", "FAILED");
    }
  });
});

Zebrunner Test Case Management (TCM)#

Zebrunner integrates with Zebrunner Test Case Management (TCM) system and provides the following capabilities:

Linking test cases to test executions
Previewing linked test cases in Zebrunner
Pushing test execution results to the TCM system

If you want to push the execution results to the TCM system, you need to provide additional configuration for the Agent using reporterOptions of your cypress.json / cypress.config.js file or environment variables as described below.

Env var / Reporter config	Description
`REPORTING_TCM_ZEBRUNNER_PUSH_RESULTS` `tcm.zebrunner.pushResults`	Boolean value which specifies if the execution results should be pushed to Zebrunner TCM. The default value is `false`.
`REPORTING_TCM_ZEBRUNNER_PUSH_IN_REAL_TIME` `tcm.zebrunner.pushInRealTime`	Boolean value. Specifies whether to push execution results immediately after each test is finished (value `true`) or not (value `false`). The default value is `false`.
`REPORTING_TCM_ZEBRUNNER_TEST_RUN_ID` `tcm.zebrunner.testRunId`	Numeric id of the target Test Run in Zebrunner TCM. If a value is not provided, no new runs will be created.

cypress.config.js

const { defineConfig } = require("cypress");
const zbrPlugin = require("@zebrunner/javascript-agent-cypress/lib/plugin");

module.exports = defineConfig({
  reporter: "@zebrunner/javascript-agent-cypress",
  reporterOptions: {
    // ...
    tcm: {
      zebrunner: {
        pushResults: true,
        pushInRealTime: true,
        testRunId: 18,
      },
    },
  },
  e2e: {
    setupNodeEvents(on, config) {
      zbrPlugin(on, config);
    },
  },
});

The Agent comes with following custom commands:

cy.zebrunnerTestCaseKey(...testCaseKeys) — accepts a list of test cases which should be linked to the current test;
cy.zebrunnerTestCaseStatus(testCaseKey, resultStatus) — links one test case and provides\overrides its result status. This may be useful if the test case result status does not correlate with the test execution status or if you have conditional logic determining the actual result status for the test case.

If these methods are invoked for the same test case id many times within a test method, the last invocation will take precedence. For example, if you invoke the cy.zebrunnerTestCaseStatus('KEY-1', 'SKIPPED') first, and then invoke the cy.zebrunnerTestCaseKey('KEY-1'), then the result status you provided in the first invocation will be ignored.

describe("some spec", () => {
  it("test name", () => {
    cy.zebrunnerTestCaseKey("KEY-1", "KEY-2", "KEY-3");
    if (someCondition) {
      // overrides the status of the test case when results are pushed to the Zebrunner TCM.
      // using this method, you can manually specify the desired result status. 
      cy.zebrunnerTestCaseStatus("KEY-2", "FAILED");
    }
  });
});

Custom Result Statuses#

By default, when the execution results are being pushed to a TCM system, Zebrunner maps each test execution result to an appropriate result status in the target TCM system. Most of the time this works perfectly, but in some cases Zebrunner is not able to derive the appropriate target result status.

One of the examples of such cases is when a test case result status does not correlate with the test execution status, or when you have conditional logic determining the actual result status for the test case. For such cases, the Agent comes with a special method which sets a specific Result Status to the test case.

Another example is custom Result Statuses in the target TCM system. In this case, we cannot anticipate the correct status and simply skip the test execution. In order to tackle this, Zebrunner allows you to configure default status for passed and failed test executions (for skipped tests, this is not technically possible).

Env var / Reporter config	Description
`REPORTING_TCM_TEST_CASE_STATUS_ON_PASS` `tcm.testCaseStatus.onPass`	The default status that will be assigned to passed test executions when they are pushed to a TCM system.
`REPORTING_TCM_TEST_CASE_STATUS_ON_FAIL` `tcm.testCaseStatus.onFail`	The default status that will be assigned to failed test executions when they are pushed to a TCM system.
`REPORTING_TCM_TEST_CASE_STATUS_ON_SKIP` `tcm.testCaseStatus.onSkip`	The default status that will be assigned to failed test executions when they are pushed to a TCM system.

cypress.config.js

const { defineConfig } = require("cypress");
const zbrPlugin = require("@zebrunner/javascript-agent-cypress/lib/plugin");

module.exports = defineConfig({
  reporter: "@zebrunner/javascript-agent-cypress",
  reporterOptions: {
    // ...
    tcm: {
      testCaseStatus: {
        onPass: "PASSED",
        onFail: "FAILED",
        onSkip: "SKIPPED",
      },
    },
  },
  e2e: {
    setupNodeEvents(on, config) {
      zbrPlugin(on, config);
    },
  },
});

When pushing results to a TCM system, Zebrunner derives the Result Status in the following order:

Checks the explicitly assigned value (which was assigned using the cy.zebrunnerTestCaseStatus() method).
Takes the default status provided via configuration for passed and/or failed tests.
Uses internal mapping of Zebrunner statuses to the Result Statuses of the target TCM system.

Logging#

By default, agent logging of output data is disabled. To enable logging to file, you need to add reportingLoggingEnabled parameter with true value to your cypress.json / cypress.config.js. Moreover, it is possible to choose level of logging by setting reportingLoggingLevel parameter. By default it is info, but you can select one of the next options: debug | info | warn | error.

Cypress 10+Cypress 9

cypress.config.js

const { defineConfig } = require('cypress');
const zbrPlugin = require('@zebrunner/javascript-agent-cypress/lib/plugin');

module.exports = defineConfig({
  reporter: '@zebrunner/javascript-agent-cypress',
  reporterOptions: {
    // ...
    reportingLoggingEnabled: true,
    reportingLoggingLevel: 'debug'
  },
  e2e: {
    setupNodeEvents(on, config) {
      zbrPlugin(on, config);
    },
  },
});

cypress.json

{
  "reporter": "@zebrunner/javascript-agent-cypress",
  "reporterOptions": {
    // ...
    "reportingLoggingEnabled": true,
    "reportingLoggingLevel": "debug"
  }
}

Note

If logging is enabled, then .log files will be rotated on daily basis. Also old files (dated older than 14 days ago) will be cleaned up automatically with new test executions.