Got played by AI good recently.
I had a passing test suite, but looking at it closely, all the tests were marked as @pytest.mark.skip :D
A good reminder that 100% coverage means nothing and to, of course, check that the genie granted you the wish you wanted, not just the wish you asked for.
@bitecode Amateur hour. Doesn't the AI know failing tests should be marked xfail? 🤦
@bitecode Though also skipped tests wouldn't help with coverage metrics, and presumably would be detected by lack of coverage, no? It is a strong argument for including tests in coverage metrics: https://nedbatchelder.com/blog/201908/dont_omit_tests_from_coverage.html
xfail might have that problem though, which is why I exclude xfailing tests from coverage: https://blog.ganssle.io/articles/2021/12/xfail-coverage.html
@pganssle True, especially since those were e2e tests which I excluded from coverage but, hey.