[DTrace-devel] [PATCH 3/7] dtrace: add default options and apply path options earlier

[Nick Alcock]

On 9 May 2023, Eugene Loh via DTrace-devel stated:

> I'm having problems getting my head around this patch (and therefore subsequent patches).  Maybe I'll try a few questions:
>
> 1)  What does procfspath have to do with this patch?  Was procfspath broken?  If so, is there now a test to show that it's fixed?

The procfspath is used by (existing) code in dt_vopen(): thus changes to
the procfspath do not take effect for that use (but do for later uses).
So in effect it was broken, yes: if you mounted /proc somewhere else you
would find you couldn't tell dtrace where it was soon enough to affect
the dt_vopen() call.

No test because there is more breakage: the BPF rewrite introduced many,
many places that just hardwire /proc as /proc rather than using the
option. They all need fixing before we can observe less breakage :)

(This was just a note, really -- there is *other* observable breakage
now, the CTF stuff.)

> 2)  Will this patch break other consumers?  The new "bt" option
> category does change how DTrace has been documenting options... but
> maybe that does not need to be exposed?

Nah. The old behaviour was inconsistent (options worked, except some of
them which failed in their earliest applications: and functions that
used things like ctfpath, which are only used once, could not be called
early at all or the associated option would never take effect). The new
behaviour simply marks such options to be applied earlier than their
first use in dtrace_open, so they have consistent behaviour now (see
below for detail). This should not be otherwise user-visible: all
btoptions remain ctoptions from the perspective of users.

> 3)  How about other approaches?  Like, can dtrace_open() just set up
> the dtp, deferring other work (e.g., that requires options to have
> been set) until dtrace_init()?

Tried that. It was a disaster. The problem is that the C and D dicts are
set up early so that the compiler can then get run to initialize a bunch
of things that are necessary for it to work at all, like the hardwired
definitions table that defines NULL, and the dt_ints table. After all
that is done, dtrace_open() ends by force-setting the libdir (and if you
set the libdir earlier you can interfere with the early compilations).

All this means that all the caller's dtrace_setopts() calls are done in
an environment in which the libdir is already set and will not be reset
except by the user asking for it.

Move the early types initialization and everything that depends on it
into dtrace_init(), so as to get them to run after dtrace_setopt(), and
suddenly the compiler's early invocations are run with libdir set -- so
you can't really reset it in dtrace_init() because you might overwrite a
user-set value, but you can't *not* set it either because the user might
not have set it at all! So you now need some sort of complex tracking
mechanism specific to not only the libdir but every other
compiler-related option, none of which are expected to be set in the
early compiler invocations. But we *do* need to set some options (the
path-related ones like ctfpath) much earlier if we are going to
initialize the C dict from the kernel types, since we need to *find* the
kernel types first.

So doing this is *really really* disruptive. Meanwhile, changing things
so that options are defaulted by dtrace_setopt calls and then the
defaults are applied in a burst late in dtrace_open does not modify
underlying call order.

Before:
  dtrace_open():
    CTF init
    early compiler invocations
    early forced libdir set
  caller:
    options setting using dtrace_setopt(dtp)
  dtrace_init() ...

After:
  caller:
    (options defaulting, no immediate effect, using
     dtrace_setopt(NULL))
  dtrace_open:
    boot option replaying from defaults (ctfpath, a couple more:
      *not* libpath) using dtrace_setopt(dtp)
    CTF init, using those paths
    early compiler invocations
    early forced libdir set
    defaulted options replaying using dtrace_setopt(dtp)
  caller:
    dtrace_init()

So the before and after invocations call all dtrace-related functions in
the same order and at the same time (as long as you ignore
dtrace_setopt(NULL), which we can because we know what it does and we
literally just wrote it here: we know it only provides option values to
the later replay, replicating what the original caller invocations of
dtrace_setopt() did).

So -- as long as we deal with the little problem that the options are
now being replayed one layer deeper in the call stack, so we need a way
to validate them and report errors which doesn't rely purely on testing
the return value of dtrace_setopt() -- it's fairly easy to see that this
isn't going to break anything. Meanwhile I'm afraid I have no idea how
to make the scheme where CTF and later things move into dtrace_init()
*not* break things: it seems to require a contradiction or loads of
extra options validation complexity (I tried it and gave up).