[DTrace-devel] [PATCH v2] test: Account for unreliable tick firing
eugene.loh at oracle.com
eugene.loh at oracle.com
Thu Jan 13 20:58:44 UTC 2022
From: Eugene Loh <eugene.loh at oracle.com>
As mentioned in commit 3e380b35cbcc "Change trigger for rand() test",
the tick-* probe is unreliable on some kernels, depending on how their
timers subsystem is configured (CONFIG*_HZ*). Some tests depend on
many probe firings and use tick-* probes. They are susceptible to
taking unnecessarily long and timing out rather than passing. Such
intermittent failures have been observed.
Fix tests accordingly. Specifically, one add bogus-ioctl as a trigger
and then use syscall::ioctl:entry as the probe to fire repeatedly.
For tst.substrminate.d, simply cut down the excessive iteration count.
For the time being, this test XFAILs anyhow.
Add some tick quality-of-service tests to expose the underlying
problem. For now, mark these tests as "unstable".
Commit
c25036d6f2aa "runtest: automatic reinvocation of specific erratic tests"
introduced the reinvocation of certain failing tests (test/unittest/aggs
and test/demo/intro), though the exact problems are not described. A
quick survey of some recent test runs indicates that reinvocations are
only used for tests that are fixed with the current patch. In particular,
reinvocations were observed only for aggs tests with strenuous dependence
on the profile provider. The demo/intro tests, while not reinvoked in
recent runs, have modest reliance on the profile provider. If those
tests fail, they can also be reworked. In short, with the current patch,
remove the reinvocation tags in the test suite. (The reinvocation
mechanism in runtest.sh can remain untouched for now.)
Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
---
test/demo/intro/test.options | 1 -
test/unittest/aggs/test.options | 1 -
test/unittest/aggs/tst.aggmod_full.sh | 2 -
test/unittest/aggs/tst.aggmod_full2.sh | 2 -
test/unittest/aggs/tst.llquantincr.d | 7 +--
test/unittest/aggs/tst.llquantize.d | 7 +--
test/unittest/aggs/tst.llquantneg.d | 9 ++--
test/unittest/aggs/tst.llquantstep.d | 7 +--
test/unittest/funcs/substr/tst.substrminate.d | 4 +-
test/unittest/speculation/tst.SingleCPU.d | 21 ++++----
test/unittest/tick-n/tst.tickquality-100ms.d | 54 +++++++++++++++++++
test/unittest/tick-n/tst.tickquality-10ms.d | 54 +++++++++++++++++++
12 files changed, 136 insertions(+), 33 deletions(-)
delete mode 100644 test/demo/intro/test.options
delete mode 100644 test/unittest/aggs/test.options
create mode 100644 test/unittest/tick-n/tst.tickquality-100ms.d
create mode 100644 test/unittest/tick-n/tst.tickquality-10ms.d
diff --git a/test/demo/intro/test.options b/test/demo/intro/test.options
deleted file mode 100644
index 2220d676..00000000
--- a/test/demo/intro/test.options
+++ /dev/null
@@ -1 +0,0 @@
-@@reinvoke-failure: 2
diff --git a/test/unittest/aggs/test.options b/test/unittest/aggs/test.options
deleted file mode 100644
index 2220d676..00000000
--- a/test/unittest/aggs/test.options
+++ /dev/null
@@ -1 +0,0 @@
-@@reinvoke-failure: 2
diff --git a/test/unittest/aggs/tst.aggmod_full.sh b/test/unittest/aggs/tst.aggmod_full.sh
index b876d736..9707340d 100755
--- a/test/unittest/aggs/tst.aggmod_full.sh
+++ b/test/unittest/aggs/tst.aggmod_full.sh
@@ -33,8 +33,6 @@
# Run time will increase with the number of errors that need to be isolated.
# So you may have to increase the timeout value to see all errors.
#
-# @@reinvoke-failure: 0
-#
# This script will read its own timeout, listed here:
#
# @@timeout: 120
diff --git a/test/unittest/aggs/tst.aggmod_full2.sh b/test/unittest/aggs/tst.aggmod_full2.sh
index 6baea432..2ae5a28c 100755
--- a/test/unittest/aggs/tst.aggmod_full2.sh
+++ b/test/unittest/aggs/tst.aggmod_full2.sh
@@ -18,8 +18,6 @@
# @@xfail: dtv2
-# @@reinvoke-failure: 0
-
dtrace=$1
CC=/usr/bin/gcc
CFLAGS=
diff --git a/test/unittest/aggs/tst.llquantincr.d b/test/unittest/aggs/tst.llquantincr.d
index 8bc3113c..27755050 100644
--- a/test/unittest/aggs/tst.llquantincr.d
+++ b/test/unittest/aggs/tst.llquantincr.d
@@ -1,6 +1,6 @@
/*
* Oracle Linux DTrace.
- * Copyright (c) 2013, 2020, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2013, 2022, Oracle and/or its affiliates. All rights reserved.
* Licensed under the Universal Permissive License v 1.0 as shown at
* http://oss.oracle.com/licenses/upl.
*/
@@ -10,15 +10,16 @@
*
* SECTION: Aggregations/Aggregations
*/
+/* @@trigger: bogus-ioctl */
#pragma D option quiet
-tick-1ms
+syscall::ioctl:entry
{
@ = llquantize(i++, 10, 0, 6, 20, 2);
}
-tick-1ms
+syscall::ioctl:entry
/i == 1500/
{
exit(0);
diff --git a/test/unittest/aggs/tst.llquantize.d b/test/unittest/aggs/tst.llquantize.d
index 0080abe3..3161ac97 100644
--- a/test/unittest/aggs/tst.llquantize.d
+++ b/test/unittest/aggs/tst.llquantize.d
@@ -1,6 +1,6 @@
/*
* Oracle Linux DTrace.
- * Copyright (c) 2013, 2020, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2013, 2022, Oracle and/or its affiliates. All rights reserved.
* Licensed under the Universal Permissive License v 1.0 as shown at
* http://oss.oracle.com/licenses/upl.
*/
@@ -10,15 +10,16 @@
*
* SECTION: Aggregations/Aggregations
*/
+/* @@trigger: bogus-ioctl */
#pragma D option quiet
-tick-1ms
+syscall::ioctl:entry
{
@ = llquantize(i++, 10, 0, 6, 20);
}
-tick-1ms
+syscall::ioctl:entry
/i == 1500/
{
exit(0);
diff --git a/test/unittest/aggs/tst.llquantneg.d b/test/unittest/aggs/tst.llquantneg.d
index 3c69839c..61d6686c 100644
--- a/test/unittest/aggs/tst.llquantneg.d
+++ b/test/unittest/aggs/tst.llquantneg.d
@@ -1,6 +1,6 @@
/*
* Oracle Linux DTrace.
- * Copyright (c) 2013, 2020, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2013, 2022, Oracle and/or its affiliates. All rights reserved.
* Licensed under the Universal Permissive License v 1.0 as shown at
* http://oss.oracle.com/licenses/upl.
*/
@@ -10,22 +10,23 @@
*
* SECTION: Aggregations/Aggregations
*/
+/* @@trigger: bogus-ioctl */
#pragma D option quiet
-tick-1ms
+syscall::ioctl:entry
/i % 2 == 0/
{
@ = llquantize(i++, 10, 0, 6, 20);
}
-tick-1ms
+syscall::ioctl:entry
/i % 2 != 0/
{
@ = llquantize(i++, 10, 0, 6, 20, -1);
}
-tick-1ms
+syscall::ioctl:entry
/i == 1500/
{
exit(0);
diff --git a/test/unittest/aggs/tst.llquantstep.d b/test/unittest/aggs/tst.llquantstep.d
index 4c055bf0..1420ebe4 100644
--- a/test/unittest/aggs/tst.llquantstep.d
+++ b/test/unittest/aggs/tst.llquantstep.d
@@ -1,6 +1,6 @@
/*
* Oracle Linux DTrace.
- * Copyright (c) 2014, 2021, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2014, 2022, Oracle and/or its affiliates. All rights reserved.
* Licensed under the Universal Permissive License v 1.0 as shown at
* http://oss.oracle.com/licenses/upl.
*/
@@ -11,15 +11,16 @@
*
* SECTION: Aggregations/Aggregations
*/
+/* @@trigger: bogus-ioctl */
#pragma D option quiet
-tick-1ms
+syscall::ioctl:entry
{
@ = llquantize(i++, 3, 3, 5, 27);
}
-tick-1ms
+syscall::ioctl:entry
/i == 1500/
{
exit(0);
diff --git a/test/unittest/funcs/substr/tst.substrminate.d b/test/unittest/funcs/substr/tst.substrminate.d
index 94ccfacb..84aa60c7 100644
--- a/test/unittest/funcs/substr/tst.substrminate.d
+++ b/test/unittest/funcs/substr/tst.substrminate.d
@@ -1,6 +1,6 @@
/*
* Oracle Linux DTrace.
- * Copyright (c) 2008, 2020, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2008, 2022, Oracle and/or its affiliates. All rights reserved.
* Licensed under the Universal Permissive License v 1.0 as shown at
* http://oss.oracle.com/licenses/upl.
*/
@@ -15,7 +15,7 @@
*/
tick-1ms
-/i++ > 1000/
+/i++ > 10/
{
exit(0);
}
diff --git a/test/unittest/speculation/tst.SingleCPU.d b/test/unittest/speculation/tst.SingleCPU.d
index a1cb4612..0a96b47d 100644
--- a/test/unittest/speculation/tst.SingleCPU.d
+++ b/test/unittest/speculation/tst.SingleCPU.d
@@ -1,6 +1,6 @@
/*
* Oracle Linux DTrace.
- * Copyright (c) 2021, Oracle and/or its affiliates. All rights reserved.
+ * Copyright (c) 2021, 2022, Oracle and/or its affiliates. All rights reserved.
* Licensed under the Universal Permissive License v 1.0 as shown at
* http://oss.oracle.com/licenses/upl.
*/
@@ -10,59 +10,56 @@
*
* SECTION: Speculative Tracing
*/
+/* @@trigger: bogus-ioctl */
#pragma D option quiet
-/* This test should take only 10s, but is taking much longer than the 10s
- one would expect (48s here). Boosting timeout temporarily. */
-/* @@timeout: 120 */
-
BEGIN
{
n = 0;
}
/*
- * Each tick, n is incremented. Which clause is used rotates modulo 4.
+ * Each firing, n is incremented. Which clause is used rotates modulo 4.
* 0: get a specid
* 1: speculate some output
* 2: speculate some other output
* 3: commit (sometimes) or discard (usually)
*/
-tick-10ms
+syscall::ioctl:entry
/ (n & 3) == 0 /
{
i = speculation();
}
-tick-10ms
+syscall::ioctl:entry
/ (n & 3) == 1 /
{
speculate(i);
printf("%4d %4d", n, i);
}
-tick-10ms
+syscall::ioctl:entry
/ (n & 3) == 2 /
{
speculate(i);
printf("%4d hello world\n", n);
}
-tick-10ms
+syscall::ioctl:entry
/ (n & 3) == 3 && (n & 63) == 3 /
{
commit(i);
}
-tick-10ms
+syscall::ioctl:entry
/ (n & 3) == 3 && (n & 63) != 3 /
{
discard(i);
}
-tick-10ms
+syscall::ioctl:entry
/ n++ >= 1000 /
{
exit(0);
diff --git a/test/unittest/tick-n/tst.tickquality-100ms.d b/test/unittest/tick-n/tst.tickquality-100ms.d
new file mode 100644
index 00000000..6407245e
--- /dev/null
+++ b/test/unittest/tick-n/tst.tickquality-100ms.d
@@ -0,0 +1,54 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+/*
+ * ASSERTION: tick-* fires with approximately the correct period.
+ *
+ * SECTION: profile Provider/tick-n probes
+ */
+/* @@tags: unstable */
+
+#pragma D option quiet
+
+BEGIN
+{
+ last = timestamp;
+
+ expected = 100 * 1000 * 1000; /* nsecs */
+ low = expected / 2;
+ high = expected * 2;
+ toohigh = expected * 20;
+
+ ntotal = 0;
+ ngood = 0;
+ nbad = 0;
+}
+tick-100msec
+{
+ t = timestamp;
+ delta = t - last;
+ last = t;
+ ntotal++;
+
+ /* delta between low and high is good */
+ ngood += delta < low ? 0 : (delta > high ? 0 : 1);
+
+ /* delta above toohigh is bad */
+ nbad += delta < toohigh ? 0 : 1;
+}
+tick-100msec
+/delta > toohigh/
+{
+ printf("too high:\n%15d (observed)\n%15d (tolerated)\n",
+ delta, toohigh);
+}
+tick-25sec
+{
+ printf("%d counts; %d good and %d bad\n", ntotal, ngood, nbad);
+ printf("%s\n", nbad ? "ERROR" : "success");
+ exit(nbad != 0);
+}
diff --git a/test/unittest/tick-n/tst.tickquality-10ms.d b/test/unittest/tick-n/tst.tickquality-10ms.d
new file mode 100644
index 00000000..60c90103
--- /dev/null
+++ b/test/unittest/tick-n/tst.tickquality-10ms.d
@@ -0,0 +1,54 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2022, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+/*
+ * ASSERTION: tick-* fires with approximately the correct period.
+ *
+ * SECTION: profile Provider/tick-n probes
+ */
+/* @@tags: unstable */
+
+#pragma D option quiet
+
+BEGIN
+{
+ last = timestamp;
+
+ expected = 10 * 1000 * 1000; /* nsecs */
+ low = expected / 2;
+ high = expected * 2;
+ toohigh = expected * 20;
+
+ ntotal = 0;
+ ngood = 0;
+ nbad = 0;
+}
+tick-10msec
+{
+ t = timestamp;
+ delta = t - last;
+ last = t;
+ ntotal++;
+
+ /* delta between low and high is good */
+ ngood += delta < low ? 0 : (delta > high ? 0 : 1);
+
+ /* delta above toohigh is bad */
+ nbad += delta < toohigh ? 0 : 1;
+}
+tick-10msec
+/delta > toohigh/
+{
+ printf("too high:\n%15d (observed)\n%15d (tolerated)\n",
+ delta, toohigh);
+}
+tick-25sec
+{
+ printf("%d counts; %d good and %d bad\n", ntotal, ngood, nbad);
+ printf("%s\n", nbad ? "ERROR" : "success");
+ exit(nbad != 0);
+}
--
2.18.4
More information about the DTrace-devel
mailing list