[DTrace-devel] [PATCH 02/10] dtprobed: add the DOF stash

[Nick Alcock]

On 29 Aug 2023, Nick Alcock via DTrace-devel said:

> On 29 Aug 2023, Kris Van Hees stated:
>> handle struct changes.  Alternatively, you could just leave the parsing to
>> DTrace itself and only store the raw DOF along with the helper info, right?
>
> Hmm. As long as DTrace jails the parser too (which is fine, it can just
> use the exact same code as the daemon does now), that dooes seem like a
> *much* better approach. I don't think the daemon needs the result of
> parsing at all any more: we literally just suck it in and stash it, so
> DTrace could just as easily do it all itself. I'm kicking myself for not
> noticing that its need for the parsed output had fallen to zero. (The
> stash still needs maintenance, but only of the raw stuff and it should
> be a bit simpler.)
>
> Not a small change though. We can converge on that in the next release,
> perhaps? I doubt it would take less than a week to do even if I had
> nothing else on my plate.

I am still musing on this, trying to figure out improvements which
retain the ability to parse DOF only once per mapping that doing the
parsing in dtprobed has. Here's my current idea (largely your idea, with
elaborations):

 - dtprobed just listens for DOF-contributing ioctls and maintains the
   DOF stash, with raw DOF and a per-pid per-mapping dh and everything
   necessary to detect outdated DOF (the exec-mapping file, etc). No
   parsing of the DOF at all. (Trivial change, actually already did it
   in my tree.)

 - when dtrace needs to get DOF for some process to find likely probes,
   it tries to open the parsed representation of the DOF out of the
   stash first. It won't find it because nothing has written it out: so
   it forks off a jailed parser child and does the parsing itself,
   writing out the parsed DOF to the same place in the stash that
   dtprobed does now. We can use the same struct-versioning scheme as
   now: it's just that rather than doing some sort of aggressive
   reparsing, dtrace just deletes the parsed representation in the DOF
   stash if it finds it's incompatible, and reparses it as if it was
   never there. Difficulty: a couple of hours, max, and a net code
   saving. All the code is already there and was written in the
   expectation that dtrace might end up using it instead of dtprobed.

That's it! It's easy enough that I can probably do it in the next day or
two. Future developments below.

 - the above gives you the efficiency benefits of parsing only once
   while entirely avoiding parsing DOF for processes that never get
   traced, so actually a big efficiency gain, with no need for a textual
   serialization format or anything. The only possible efficiency loss
   happens when we do systemwide tracing and suddenlyj dtrace needs to
   know about all probes in all running processes. In that case, we
   rejig the jailed parser to be a pool of parsers, and poll() on it so
   we can rapidly traverse all unparsed DOF in (affected processes in)
   the stash and parse the lot, parsing multiple DOFs in parallel as
   needed. Difficulty: higher, but is part of systemwide probing which
   is a bigger change in itself.

 - we could also revive the code that lets us extract DOF from processes
   even before their constructors run, but figuring out the dh there
   seems likely to be a bit harder and the benefits (probes in ld.so and
   in very early ELF constructors) are marginal enough that I think we
   should put this off until neeeded :)

-- 
NULL && (void)