[DTrace-devel] [PATCH 07/12] Add tests for various options and tunables

[Eugene Loh]

On 7/19/22 21:43, Kris Van Hees wrote:
> On Mon, Jul 18, 2022 at 10:30:20AM -0700, Eugene Loh wrote:
>> On 7/17/22 21:43, Kris Van Hees wrote:
>>
>>> On Sun, Jul 17, 2022 at 08:38:45AM -0700, Eugene Loh via DTrace-devel wrote:
>>>> Reviewed-by: Eugene Loh <eugene.loh at oracle.com>
>>>> The tests
>>>>       test/unittest/options/tst.D.d
>>>>       test/unittest/options/tst.define.d
>>>> should be fine, but might as well add .r results files for them, to get that
>>>> special reassurance that :BEGIN shows up.
>>> Well, I care less whether BEGIN shows up or not.  That is not the condition we
>>> are testing here.  The way the test is written, it will not compile if the
>>> define is not honoured.
>> When we know the expected output, a .r file is a "free" extra check.  I
>> think there have been a number of times in the last months that
>> self-checking tests were passing spuriously but fixed just by adding a .r
>> file.  It strikes me as a simple "best practice."
> But what does it check extra?

There is a range of tests that can make sense.  Some of those tests can 
be more barebones, some more stringent.  More stringent tests offer the 
possibility of catching unanticipated problems.  Adding a .r file is one 
simple way of doing that.  I'm not saying the proposed test is wrong, 
but only that in my opinion adding a .r file makes it better.

> I do not like the notion of expecting specific components to report debug output
> when -xdebug is specified.  All we are checking is that there is debug output
> when -xdebug is specified - that is what the option does.  Any additional
> expectations on the output are not adding to what needs to be tested here.

That's fine.  In general, I think there's some value in being a little 
pedantic with testing when it's easy enough to do so -- to be a little 
suspicious.  How do we know that what we're seeing is expected -xdebug 
output?  The $2=="DEBUG" check struck me as weak, but I guess it's 
sufficient for such a niche option.