DEV: Remove flaky tests report when there are too many failures (#25031)
Why this change? Currently we only rerun failing tests to check if they are flaky tests when there are 10 or less failing tests. When there are more than 10 failing tests in the first run, we assume that the odds of those tests being flaky are low and do not rerun the tests. However, there was a bug where we do not clean up the potential flaky tests being logged when there are too many test failures. This resulted in those test failures being treated as flaky tests. What does this change do? Clean up the flaky tests report when we do not rerun the tests.
This commit is contained in:
parent
d03f6727b1
commit
cfec408bc1
|
@ -11,6 +11,10 @@ module TurboTests
|
|||
.map { |failed_example| failed_example["location_rerun_argument"] }
|
||||
end
|
||||
|
||||
def self.remove_flaky_tests
|
||||
File.delete(PATH) if File.exist?(PATH)
|
||||
end
|
||||
|
||||
# This method should only be called by a formatter registered with `TurboTests::Runner` and logs the failed examples
|
||||
# to `PATH`. See `FailedExample#to_h` for the details of each example that is logged.
|
||||
#
|
||||
|
|
|
@ -103,6 +103,7 @@ module TurboTests
|
|||
return rerun_failed_examples(@reporter.failed_examples)
|
||||
else
|
||||
STDOUT.puts "Retry and log flaky tests was enabled but ignored because there are more than #{retry_failed_examples_threshold} failures."
|
||||
Flaky::Manager.remove_flaky_tests
|
||||
end
|
||||
end
|
||||
|
||||
|
|
Loading…
Reference in New Issue