[DTrace-devel] rand() issues

[Eugene Loh]

On 11/21/21 4:47 PM, Kris Van Hees wrote:

> I gave a reviewed-by on your rand() patch and added it to my queue branch to
> push to dev.
>
> Running some iterations of tests for other features, I noticed that (on OL8
> x86_64, which is the only place I was testing some other things) the rand
> tests:
>
> test/unittest/funcs/tst.rand_inter.sh
> test/unittest/funcs/tst.rand_intra.sh
>
> are intermittently failing.  Looking closer at these tests, I am a bit unclear
> why we have them.
Minimum sanity tests on the output.
> They seem to be testing the quality of the rand() function,
> which amounts to testing the quality of the random number generator function in
> BPF itself.  That is not really we should be testing because we have no control
> over it, and DTrace does not actually impose quality expectations on the pseudo
> random generator that rand() provides.
Using BPF helper functions strikes me as insufficient reason not to test 
things.  If we find that BPF helper functions are bad, we should "do 
something" (fix bugs, not use those functions, etc.) rather than simply 
not run our tests.  And while we might not document standards for our 
RNG, I would hope we would not be satisfied with returning, say, -1 all 
the time.  So, we need at least some sanity check on the output.

But I suspect all that is beside the point.  It would be good to know 
why the tests are failing.  I suspect the reason has nothing to do with 
the BPF helper function.  It more likely has to do with other, broader 
issues we're facing.
> So, I'd prefer that we either remove those tests as not being relevant to the
> rand() implementation itself (and out of our control to ensure they pass), or
> rework them in a way that they will consistently pass (and then document why
> we expect them to pass and what should be done if they do not).
>
> I think I favour the option of removing them simply because it seems rather
> impractical and unnecessary to maintain a test for something outside of DTrace
> in terms of assertions that do not reflect expectations that DTrace places on
> rand().