[DTrace-devel] rand() issues

[Eugene Loh]

The log says that there was insufficient data... about 300 values.  The 
tests are supposed to fire every 200usec and run for 5 secs.  So there 
should be 25000 values.  The tests allow that fewer values may be 
produced -- they have very generous tolerances -- but only 300 values is 
deemed too egregious.  Indeed, would a DTrace user be satisfied that 
tick-200us fires only 300 times in 5 seconds?  Or that waiting for 25000 
tick-200us firings should take not 5 secs but about ten minutes?

I think this has nothing to do with bpf_get_prandom_u32().  I suspect 
the problem is related to our profile provider not working depending on 
how the kernel timers subsystem is configured.  Some CONFIG*_HZ* 
parameter/s.  And I suspect we have other tests with similar problems.

On 11/22/21 12:04 PM, Kris Van Hees wrote:
> Log sent as private email - please reply to the issue in this thread though so
> it makes it to the list.
>
> On Mon, Nov 22, 2021 at 11:16:30AM -0500, Eugene Loh via DTrace-devel wrote:
>> On 11/21/21 4:47 PM, Kris Van Hees wrote:
>>
>>> I gave a reviewed-by on your rand() patch and added it to my queue branch to
>>> push to dev.
>>>
>>> Running some iterations of tests for other features, I noticed that (on OL8
>>> x86_64, which is the only place I was testing some other things) the rand
>>> tests:
>>>
>>> test/unittest/funcs/tst.rand_inter.sh
>>> test/unittest/funcs/tst.rand_intra.sh
>>>
>>> are intermittently failing.  Looking closer at these tests, I am a bit unclear
>>> why we have them.
>> Minimum sanity tests on the output.
>>> They seem to be testing the quality of the rand() function,
>>> which amounts to testing the quality of the random number generator function in
>>> BPF itself.  That is not really we should be testing because we have no control
>>> over it, and DTrace does not actually impose quality expectations on the pseudo
>>> random generator that rand() provides.
>> Using BPF helper functions strikes me as insufficient reason not to test
>> things.  If we find that BPF helper functions are bad, we should "do
>> something" (fix bugs, not use those functions, etc.) rather than simply not
>> run our tests.  And while we might not document standards for our RNG, I
>> would hope we would not be satisfied with returning, say, -1 all the time.
>> So, we need at least some sanity check on the output.
>>
>> But I suspect all that is beside the point.  It would be good to know why
>> the tests are failing.  I suspect the reason has nothing to do with the BPF
>> helper function.  It more likely has to do with other, broader issues we're
>> facing.
>>> So, I'd prefer that we either remove those tests as not being relevant to the
>>> rand() implementation itself (and out of our control to ensure they pass), or
>>> rework them in a way that they will consistently pass (and then document why
>>> we expect them to pass and what should be done if they do not).
>>>
>>> I think I favour the option of removing them simply because it seems rather
>>> impractical and unnecessary to maintain a test for something outside of DTrace
>>> in terms of assertions that do not reflect expectations that DTrace places on
>>> rand().
>> _______________________________________________
>> DTrace-devel mailing list
>> DTrace-devel at oss.oracle.com
>> https://oss.oracle.com/mailman/listinfo/dtrace-devel