[DTrace-devel] [PATCH] test: Loosen tolerances on args_kernel tests

[Kris Van Hees]

Question below (just to better understand this)...

On Mon, Feb 27, 2023 at 04:16:10PM -0500, eugene.loh--- via DTrace-devel wrote:
> From: Eugene Loh <eugene.loh at oracle.com>
> 
> Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
> ---
>  test/unittest/cpc/tst.args_kernel.sh       | 7 +++----
>  test/unittest/profile-n/tst.args_kernel.sh | 7 +++----
>  2 files changed, 6 insertions(+), 8 deletions(-)
> 
> diff --git a/test/unittest/cpc/tst.args_kernel.sh b/test/unittest/cpc/tst.args_kernel.sh
> index 0cb2480a..c1911588 100755
> --- a/test/unittest/cpc/tst.args_kernel.sh
> +++ b/test/unittest/cpc/tst.args_kernel.sh
> @@ -71,9 +71,8 @@ END {
>  
>  # report
>  status=0
> -margin=$(($expect / 10))
> -$utils/check_result.sh $ntotal $expect $margin; status=$(($status + $?))
> -$utils/check_result.sh $nwarn     0    $margin; status=$(($status + $?))
> -$utils/check_result.sh $nerror    0       0   ; status=$(($status + $?))
> +$utils/check_result.sh $ntotal $expect $(($expect /  2)); status=$(($status + $?))

Does this mean that we're accepting an error margin of 50%?  That seems rather
extreme... I wonder whether there is a way to tighten up the test to not need
such a wide margin.  I know the test is marked unstable for now, so perhaps
mostly we need a comment explaining why such a wide margin is acceptable?

> +$utils/check_result.sh $nwarn     0    $(($expect / 10)); status=$(($status + $?))
> +$utils/check_result.sh $nerror    0    $(($expect / 10)); status=$(($status + $?))
>  
>  exit $status
> diff --git a/test/unittest/profile-n/tst.args_kernel.sh b/test/unittest/profile-n/tst.args_kernel.sh
> index d489e554..8ef59893 100755
> --- a/test/unittest/profile-n/tst.args_kernel.sh
> +++ b/test/unittest/profile-n/tst.args_kernel.sh
> @@ -66,9 +66,8 @@ END {
>  
>  # report
>  status=0
> -margin=$(($expect / 10))
> -$utils/check_result.sh $ntotal $expect $margin; status=$(($status + $?))
> -$utils/check_result.sh $nwarn     0    $margin; status=$(($status + $?))
> -$utils/check_result.sh $nerror    0       0   ; status=$(($status + $?))
> +$utils/check_result.sh $ntotal $expect $(($expect /  2 )); status=$(($status + $?))
> +$utils/check_result.sh $nwarn     0    $(($expect / 10 )); status=$(($status + $?))
> +$utils/check_result.sh $nerror    0    $(($expect / 10 )); status=$(($status + $?))
>  
>  exit $status
> -- 
> 2.18.4
> 
> 
> _______________________________________________
> DTrace-devel mailing list
> DTrace-devel at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/dtrace-devel