[DTrace-devel] rand() issues

[Kris Van Hees]

On Mon, Nov 22, 2021 at 01:22:23PM -0500, Eugene Loh via DTrace-devel wrote:
> The log says that there was insufficient data... about 300 values.  The
> tests are supposed to fire every 200usec and run for 5 secs.  So there
> should be 25000 values.  The tests allow that fewer values may be produced
> -- they have very generous tolerances -- but only 300 values is deemed too
> egregious.  Indeed, would a DTrace user be satisfied that tick-200us fires
> only 300 times in 5 seconds?  Or that waiting for 25000 tick-200us firings
> should take not 5 secs but about ten minutes?
> 
> I think this has nothing to do with bpf_get_prandom_u32().  I suspect the
> problem is related to our profile provider not working depending on how the
> kernel timers subsystem is configured.  Some CONFIG*_HZ* parameter/s.  And I
> suspect we have other tests with similar problems.

So, the test should be eother considered unstable (which is not good), or can
you rework it in a way that it is more tolerant?  While there does seem to be
some underlying issue that affects the test, we do need to be able to have
tests that are as dependable and stable as possible.

Can you have a look at that, especially in view of OL8 UEK6?  I know you have
looked into timer event issues before in view of CONFIG_* parameters so I hope
you can make a determination fialy easily on whether there is a bug here that
ought to be fixed (in order to allow for timer based tracing to work correctly)
or a recommendation on what config settings are needed to make this work right.

> On 11/22/21 12:04 PM, Kris Van Hees wrote:
> > Log sent as private email - please reply to the issue in this thread though so
> > it makes it to the list.
> > 
> > On Mon, Nov 22, 2021 at 11:16:30AM -0500, Eugene Loh via DTrace-devel wrote:
> > > On 11/21/21 4:47 PM, Kris Van Hees wrote:
> > > 
> > > > I gave a reviewed-by on your rand() patch and added it to my queue branch to
> > > > push to dev.
> > > > 
> > > > Running some iterations of tests for other features, I noticed that (on OL8
> > > > x86_64, which is the only place I was testing some other things) the rand
> > > > tests:
> > > > 
> > > > test/unittest/funcs/tst.rand_inter.sh
> > > > test/unittest/funcs/tst.rand_intra.sh
> > > > 
> > > > are intermittently failing.  Looking closer at these tests, I am a bit unclear
> > > > why we have them.
> > > Minimum sanity tests on the output.
> > > > They seem to be testing the quality of the rand() function,
> > > > which amounts to testing the quality of the random number generator function in
> > > > BPF itself.  That is not really we should be testing because we have no control
> > > > over it, and DTrace does not actually impose quality expectations on the pseudo
> > > > random generator that rand() provides.
> > > Using BPF helper functions strikes me as insufficient reason not to test
> > > things.  If we find that BPF helper functions are bad, we should "do
> > > something" (fix bugs, not use those functions, etc.) rather than simply not
> > > run our tests.  And while we might not document standards for our RNG, I
> > > would hope we would not be satisfied with returning, say, -1 all the time.
> > > So, we need at least some sanity check on the output.
> > > 
> > > But I suspect all that is beside the point.  It would be good to know why
> > > the tests are failing.  I suspect the reason has nothing to do with the BPF
> > > helper function.  It more likely has to do with other, broader issues we're
> > > facing.
> > > > So, I'd prefer that we either remove those tests as not being relevant to the
> > > > rand() implementation itself (and out of our control to ensure they pass), or
> > > > rework them in a way that they will consistently pass (and then document why
> > > > we expect them to pass and what should be done if they do not).
> > > > 
> > > > I think I favour the option of removing them simply because it seems rather
> > > > impractical and unnecessary to maintain a test for something outside of DTrace
> > > > in terms of assertions that do not reflect expectations that DTrace places on
> > > > rand().
> > > _______________________________________________
> > > DTrace-devel mailing list
> > > DTrace-devel at oss.oracle.com
> > > https://oss.oracle.com/mailman/listinfo/dtrace-devel
> 
> _______________________________________________
> DTrace-devel mailing list
> DTrace-devel at oss.oracle.com
> https://oss.oracle.com/mailman/listinfo/dtrace-devel