Tutorial 11: Use Tags

Goal:Understand the usage of tags to organize the testsuite and optimize test runs.

Several test frameworks support a concept of tags to mark a number of tests (py.test markers, TestNG test groups, JUnit Categories, NUnit CategoryAttribute). This provides a simple, flexible and effective mechanism to:

  • select a number of tests
  • exclude a number of tests

for a test run. This mechanism is orthogonal to the static test package structure.

Hint

Predefined or often used tags:

Tag Kind Description
@wip predefined “Work in Process” (under development).
@skip predefined Skip/disable a feature, scenario, ...
@slow user-defined Mark slow, long-running tests.

Hint

Tag Logic:

Logic Operation Command Options Description
select/enable --tags=@one Only items with this tag.
not (tilde/minus) --tags=~@one Only items without this tag.
logical-or --tags=@one,@two If @one or @two is present.
logical-and --tags=@one --tags=@two If both @one and @two are present.

Notes:

  • The tag name prefix ‘@’ (AT) is optional in tag options
  • Use --tags-help for a short description of the tag logic.

See also behave tags documentation for more information on tags.

Write the Feature Test

# file:features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)

    In order to increase the ninja survival rate,
    As a ninja commander
    I want my ninjas to decide whether to take on an opponent
    based on their skill levels.

    @ninja.any
    Scenario: Weaker opponent
        Given the ninja has a third level black-belt
        When attacked by a samurai
        Then the ninja should engage the opponent

    @ninja.chuck
    Scenario: Stronger opponent
        Given the ninja has a third level black-belt
        When attacked by Chuck Norris
        Then the ninja should run for his life

Run the Feature Test

When you run the feature file by excluding the tag @wip, then any feature marked with this tag is skipped as well as all of its scenarios.

$ behave --tags=-wip ../features/tutorial11_tags.feature
0 features passed, 0 failed, 1 skipped
0 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 6 skipped, 0 undefined
Took 0m0.000s

Note

Check the test summary for the skipped count for features and scenarios.

When you enable the tag @ninja.chuck:

$ behave --tags=ninja.chuck ../features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)   # ../features/tutorial11_tags.feature:2
  In order to increase the ninja survival rate,
  As a ninja commander
  I want my ninjas to decide whether to take on an opponent
  based on their skill levels.

  @ninja.chuck
  Scenario: Stronger opponent                    # ../features/tutorial11_tags.feature:16
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by Chuck Norris                # ../features/steps/step_tutorial02.py:65
    Then the ninja should run for his life       # ../features/steps/step_tutorial02.py:69

1 feature passed, 0 failed, 0 skipped
1 scenario passed, 0 failed, 1 skipped
3 steps passed, 0 failed, 3 skipped, 0 undefined
Took 0m0.000s

Note

Now only the second scenario is executed and the first one is skipped.

When you disable the tag @ninja.chuck:

$ behave --tags=-ninja.chuck ../features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)   # ../features/tutorial11_tags.feature:2
  In order to increase the ninja survival rate,
  As a ninja commander
  I want my ninjas to decide whether to take on an opponent
  based on their skill levels.

  @ninja.any
  Scenario: Weaker opponent                      # ../features/tutorial11_tags.feature:10
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by a samurai                   # ../features/steps/step_tutorial02.py:61
    Then the ninja should engage the opponent    # ../features/steps/step_tutorial02.py:69

1 feature passed, 0 failed, 0 skipped
1 scenario passed, 0 failed, 1 skipped
3 steps passed, 0 failed, 3 skipped, 0 undefined
Took 0m0.000s

Note

Now only the first scenario is executed and the second one is now skipped.

When you select items with either tag @ninja.any or the tag @ninja.chuck (tag-or):

$ behave --tags=@ninja.any,@ninja.chuck ../features/tutorial11_tags.feature
@wip
Feature: Using Tags with Features and Scenarios (tutorial11 := tutorial2)   # ../features/tutorial11_tags.feature:2
  In order to increase the ninja survival rate,
  As a ninja commander
  I want my ninjas to decide whether to take on an opponent
  based on their skill levels.

  @ninja.any
  Scenario: Weaker opponent                      # ../features/tutorial11_tags.feature:10
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by a samurai                   # ../features/steps/step_tutorial02.py:61
    Then the ninja should engage the opponent    # ../features/steps/step_tutorial02.py:69

  @ninja.chuck
  Scenario: Stronger opponent                    # ../features/tutorial11_tags.feature:16
    Given the ninja has a third level black-belt # ../features/steps/step_tutorial02.py:57
    When attacked by Chuck Norris                # ../features/steps/step_tutorial02.py:65
    Then the ninja should run for his life       # ../features/steps/step_tutorial02.py:69

1 feature passed, 0 failed, 0 skipped
2 scenarios passed, 0 failed, 0 skipped
6 steps passed, 0 failed, 0 skipped, 0 undefined
Took 0m0.001s

Note

Now both scenarios are executed.

When you select items that have the tag @ninja.any and the tag @ninja.chuck (tag-and):

$ behave --tags=@ninja.any --tags=@ninja.chuck ../features/tutorial11_tags.feature
0 features passed, 0 failed, 1 skipped
0 scenarios passed, 0 failed, 2 skipped
0 steps passed, 0 failed, 6 skipped, 0 undefined
Took 0m0.000s

Note

Now no scenario is executed, all are skipped.