Step

Definition

A step is a single unit of work that can be executed as part of a test. It is defined in a YAML document with the spec field set to step (not required by default). A step can represent an action to perform and a sequence of checks to validate.

At runtime:

  • The core locates the plugin that registered this action

  • The action schema is validated

  • Arguments are parsed and compiled (including deferred expressions)

  • The plugin executes the action implementation

Step definition example
title: Short human-readable step title
description: >
  Detailed human-readable description of the step.
  May span multiple lines and is intended for documentation
  and reporting purposes
action: empty

vars:
  expectedValue: Hello, world!
export:
  actualValue: !var expectedValue

expect:

- title: Check that value is correct
  value: !var actualValue
  match: !var expectedValue

After execution, the action produces a result object. By convention, this result is stored in the case context under result. But user can redefine output variable name by output field.

The step definition can include an optional vars section, which allows you to define local variables that will be added to the execution context before the step is executed. These variables can then be accessed within the step using the !var syntax.

The step definition can include an optional export section, which allows you to define variables that will be added to the global execution context after the step is executed. These variables can then be accessed in next steps using the !var syntax.

Export:

  • Evaluates expressions after action execution

  • Stores values in case context

  • Makes them available to next steps

Base input schema

entity schemas.Action

Base class for executable actions.

param action: str [Required]

Name of the action executed by a scenario step. An action may be specified either as a builtins name (for example, debug) or as a plugin-qualified name using dot notation (for example, http.get). Action identifiers are limited to ASCII letters, digits, and underscores.

param description: str | None = None

Detailed human-readable description of the DSL element.

param expect: list[Check] = []

List of checks that must pass after the action execution.

Each check validates some aspect of the action result or side effects. All expectations are evaluated to determine the success or failure of the step.

param export: dict[str, Deferred[Value]] = {}

Mapping of variable names to values derived from the action result.

Exported variables are merged into the execution context and become available to subsequent steps.

param output: str = 'result'

Name of the variable under which the action result context will be stored.

The stored value may later be referenced or selectively exported into the execution context.

param spec: Literal['step'] = 'step'
param title: str | None = None

Short human-readable title of the DSL element.

param vars: dict[str, Deferred[Value]] = {}

Local context variables provided to block.

These variables are added to the execution context and can be referenced by subsequent steps.

By default, the core provides a single action called empty, which does nothing. You can use core extension mechanism to define your own actions and make them available in steps. This allows you to extend the functionality of steps and reuse common actions across different steps and tests.

Expectations

Expectations are defined in the expect field of a step. They represent assertions that must hold true for the step to be considered successful. Each expectation can have an optional title for better reporting and documentation. Expectations can reference variables defined in the step or in the global context.

Expectations:

  • Are evaluated after export

  • Operate on fully resolved values

  • Produce structured assertion results

  • Are reported individually

  • Failure of an expectation fails the step and the case

Base input schema

entity schemas.Check

Base class for declarative value checks.

param description: str | None = None

Detailed human-readable description of the DSL element.

param title: str | None = None

Short human-readable title of the DSL element.

param value: Deferred[Value] [Required]

Value to be validated by the check.

The value is resolved against the execution context before the check is executed.

Expectations can be extended by plugins.

Builtins

Match

Match check validates that the value matches the expected value. The check can be used to compare values of any type, including strings, numbers, lists, and dictionaries. The check performs a deep comparison, meaning that it will compare the contents of lists and dictionaries rather than just their references.

For partial matching of list or dictionaries, you can use the partialMatch check, which allows you to specify only a subset of the expected values. The check will validate that the actual list or dictionary contains at least the specified values, while ignoring any additional keys.

Partial match example
expect:

- title: Check that actual value contains expected values
  match:
    name: Bob Plissken
    age: 35
  partialMatch: yes
  value:
    name: Bob Plissken
    age: 35
    city: New York

Input schemas

entity schemas.MatchCheck
param description: str | None = None

Detailed human-readable description of the DSL element.

param match: Deferred [Required]

Value that must match the actual result.

Aliases: match, eq and equal

param partial_match: bool | Deferred = False

If true, performs recursive partial matching instead of strict equality comparison.

Aliases: partial_match and partialMatch

param title: str | None = None

Short human-readable title of the DSL element.

param value: Deferred[Value] [Required]

Value to be validated by the check.

The value is resolved against the execution context before the check is executed.

entity schemas.NotMatchCheck
param description: str | None = None

Detailed human-readable description of the DSL element.

param not_match: Deferred [Required]

Value that must not match the actual result.

Aliases: not_match, notMatch, ne and notEqual

param partial_match: bool | Deferred = False

If true, performs recursive partial matching instead of strict equality comparison.

Aliases: partial_match and partialMatch

param title: str | None = None

Short human-readable title of the DSL element.

param value: Deferred[Value] [Required]

Value to be validated by the check.

The value is resolved against the execution context before the check is executed.

Comparison

Comparison checks allow you to compare values using standard comparison operators such as greater than, less than, etc. These checks can be used to validate that a value meets certain criteria, such as being greater than a specified threshold or less than a maximum value. Comparison checks can be applied to numeric values, strings and other comparable types.

Comparison check example
expect:

- title: Check that actual value is greater than expected value
  greaterThan: 10
  value: 15

Input schemas

entity schemas.GreaterThanCheck
param description: str | None = None

Detailed human-readable description of the DSL element.

param greater_than: Deferred [Required]

Actual value must be greater than this value.

Aliases: greater_than, gt and greaterThan

param title: str | None = None

Short human-readable title of the DSL element.

param value: Deferred[Value] [Required]

Value to be validated by the check.

The value is resolved against the execution context before the check is executed.

entity schemas.GreaterThanOrEqualCheck
param description: str | None = None

Detailed human-readable description of the DSL element.

param greater_than_or_equal: Deferred [Required]

Actual value must be greater than or equal to this value.

Aliases: greater_than_or_equal, gte and greaterThanOrEqual

param title: str | None = None

Short human-readable title of the DSL element.

param value: Deferred[Value] [Required]

Value to be validated by the check.

The value is resolved against the execution context before the check is executed.

entity schemas.LessThanCheck
param description: str | None = None

Detailed human-readable description of the DSL element.

param less_than: Deferred [Required]

Actual value must be less than this value.

Aliases: less_than, lt and lessThan

param title: str | None = None

Short human-readable title of the DSL element.

param value: Deferred[Value] [Required]

Value to be validated by the check.

The value is resolved against the execution context before the check is executed.

entity schemas.LessThanOrEqualCheck
param description: str | None = None

Detailed human-readable description of the DSL element.

param less_than_or_equal: Deferred [Required]

Actual value must be less than or equal to this value.

Aliases: less_than_or_equal, lte and lessThanOrEqual

param title: str | None = None

Short human-readable title of the DSL element.

param value: Deferred[Value] [Required]

Value to be validated by the check.

The value is resolved against the execution context before the check is executed.

Regular expression

See also

Python Regular expressions

This module provides regular expression matching operations similar to those found in Perl.

Regular expression checks allow you to validate that a string value matches a specified regular expression pattern. These checks can be used to ensure that a value conforms to a specific format, such as an email address, phone number, or any custom pattern defined by the user. Regular expression checks can be applied to string values and provide a powerful way to validate complex patterns and formats.

Regular expression check example
expect:
- title: Check that actual value matches regular expression pattern
  regex: '^[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,5}$'
  ignoreCase: yes
  value: !var emailAddress

Input schemas

entity schemas.RegexCheck
param description: str | None = None

Detailed human-readable description of the DSL element.

param ignore_case: bool | Deferred = False

If true, performs case-insensitive matching.

Aliases: ignore_case and ignoreCase

param multiline: bool | Deferred = False

If true, performs multiline matching.

param regex: Deferred [Required]

Actual value must be match to this pattern.

Aliases: regex, reMatch and regexMatch

param title: str | None = None

Short human-readable title of the DSL element.

param value: Deferred[Value] [Required]

Value to be validated by the check.

The value is resolved against the execution context before the check is executed.