[DTrace-devel] [PATCH 13/61] test: Report unstable tests that fail as xfail

[eugene.loh at oracle.com]

From: Eugene Loh <eugene.loh at oracle.com>

Generally, tests are deterministically expected to pass or xfail.

Some tests fail intermittently, however, and are tagged "unstable".
Either pass or fail is acceptable, but one has to check manually
to see if the test is unstable.

If a test fails and it is unstable, just report it as xfail and
unstable.

Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
---
 runtest.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/runtest.sh b/runtest.sh
index af1f67d3..4c0bb381 100755
--- a/runtest.sh
+++ b/runtest.sh
@@ -1393,6 +1393,18 @@ for dt in $dtrace; do
             pass "$xfail" "$xfailmsg" "$capturing"
 
         else
+
+            # If it fails, is not xfail, and is unstable, just call it xfail.
+            if [[ -z $xfail ]] && [[ "$(extract_options tags $_test)" == *unstable* ]]; then
+                xfail=t
+
+                if [[ -z $xfailmsg ]]; then
+                    xfailmsg="unstable"
+                else
+                    xfailmsg="unstable; $xfailmsg"
+                fi
+            fi
+
             fail "$xfail" "$xfailmsg" "$failmsg${capturing:+, $capturing}"
         fi
 
-- 
2.18.4