How to retry the failed test cases in PyTest?

When working with automated testing in Python, PyTest is one of the most popular and powerful testing frameworks. However, in real-world scenarios, tests can sometimes fail due to transient issues—network latency, database unavailability, or even slight timing mismatches. Manually rerunning failed test cases is time-consuming and inefficient. Fortunately, PyTest provides built-in capabilities to automatically rerun failed tests, ensuring robustness and efficiency in test execution.

In this article, we’ll explore why rerunning failed test cases is essential, how to do it in PyTest, and the different ways to implement it effectively.

Why Do We Need to Retry Failed Test Cases?

Automated test cases are designed to simulate real-world scenarios and verify the functionality of an application. However, the testing ecosystem often faces transient issues that can cause tests to fail sporadically. Here are a few reasons why retrying failed test cases is essential:

1. Transient Issues: Tests may fail due to temporary issues like network instability, server overload, or environment setup delays. Retrying can help bypass such issues without manual intervention.
2. Flaky Tests: Some tests are inherently flaky due to external dependencies or timing issues. Retrying allows these tests to re-execute, reducing false negatives in test reports.
3. Time-Saving: Automating the retry process saves significant time by eliminating the need to manually re-execute failed tests.

How can we retry failed Test Cases?

• Retry a test case as soon as it fails
• Or retry all failed tests once at the end

Let’s discuss both of the ways one by one

Retry a test case as soon as it fails

To rerun a test case as soon as it fails in pytest, you can use the pytest-rerunfailures plugin.

Install pytest-rerunfailures

pip install pytest pytest-rerunfailures

We can either rerun all tests a set number of times or choose to rerun only specific tests.

Reruns all failing tests

Consider the below test file where we are purposefully failing 2 tests.

def test_1():
    assert False

def test_2():
    assert False

def test_3():
    assert True

Run the tests using the below command, where the test will rerun 2 more times if it fails.

pytest --reruns 2

There were 4 reruns in total because 2 tests failed, and each test was retried twice.

We can also add a delay using “–reruns-delay” command line option before rerunning a failed test case.

pytest --reruns 1 --reruns-delay 2

What does the above command do? It reruns failed test cases only once, with a 2-second delay before retrying.

Rerunning specific test case

We can configure specific test cases to rerun when they fail using the @pytest.mark.flaky decorator.

import pytest

@pytest.mark.flaky(reruns=2)
def test_1():
    assert False

def test_2():
    assert False

def test_3():
    assert True

Now, if we run the pytest command, it will retry only the test_1() method if it fails.

Here, we can see that only one test case was retried twice.

Similarly, we can also add the delay while retrying a specific test case.

import pytest

@pytest.mark.flaky(reruns=2, reruns_delay=2)
def test_1():
    assert False

def test_2():
    assert False

def test_3():
    assert True

Retry all failed tests once at the end

We can also retry all the failed cases once after the execution of the tests is completed. Just run the “pytest --lf” command.

Here we can see that we needed to run a second command ( “pytest --lf” ) to rerun the failed test cases.

This is it. We hope that you have liked the article. If you have any doubts or concerns, please write to us in the comments or mail us at admin@codekru.com.