[DTrace-devel] [PATCH v3 06/21] dtprobed: add the DOF stash

[Nick Alcock]

On 12 Feb 2024, Kris Van Hees spake thusly:

> Initial comments (mostly on the comments, that are pretty crucial for people
> to understand what is going on here).

... here's the new top-of-file comment block, which hopefully make a bit
more sense now:

/*
 * Oracle Linux DTrace; DOF state storage management.
 * Copyright (c) 2024, Oracle and/or its affiliates. All rights reserved.
 * Licensed under the Universal Permissive License v 1.0 as shown at
 * http://oss.oracle.com/licenses/upl.
 *
 * The DOF stash is the principal state preservation mechanism used by dtprobed.
 * Keeping DOF on disk keeps memory management complexity low, makes it
 * possible for dtprobed to track DOF across daemon restarts and upgrades,
 * and gives the rest of dtrace a place to acquire the DOF contributed by
 * processes on the system for USDT probe operations.
 *
 * The state storage structure is as follows:
 *
 * /run/dtrace/stash: Things private to dtprobed.
 *
 *    .../dof/$dev-$ino: DOF contributed by particular mappings, in raw form
 *    (as received from some probe-containing program).
 *
 *    .../dof-pid/$pid/$dev-$ino: contains everything relating to DOF
 *    contributed by a particular USDT-containing ELF object within a given
 *    process (henceforth "DOF source").  Each DTRACEHIOC_ADDDOF ioctl call
 *    creates one of these.
 *
 *    .../dof-pid/$pid/$gen: symlink to a single $dev-$ino dir. "$gen" is an
 *    incrementing generation counter starting at zero for every unique PID,
 *    used to uniquely identify DOF pieces to remove.
 *
 *    .../dof-pid/$pid/next-gen: The next generation counter to use for a new
 *    piece of DOF from this process.  Encoded as a sparse file whose size is
 *    the next counter value to use.
 *
 *    .../dof-pid/$pid/$dev-$ino/raw: hardlink to the DOF for a given DOF
 *    source.  Pruned of dead processes at startup and on occasion: entries also
 *    deleted on receipt of DTRACEHIOC_REMOVE ioctls.  A hardlink is used in
 *    order to bump the link count for the corresponding DOF in the dof/
 *    directory: when this link count falls to 1, the DOF is considered dead and
 *    the corresponding probe is removed.
 *
 *    .../dof-pid/$pid/$dev-$ino/dh: Raw form of the dof_helper_t received from
 *    a given DTRACEHIOC_ADDDOF, serialized straight to disk with no changes.
 *    Preserved for all DTRACEHIOC_ADDDOFs, even if the DOF was already
 *    contributed by some other process.  Keeping this around allows the DOF to
 *    be re-parsed as needed, even by a later re-execution of a different
 *    version of the same daemon with a different dof_parsed_t structure, and to
 *    be re-parsed with appropriate addresses for whichever mapping of this DOF
 *    is under consideration.
 *
 *    .../dof-pid/$pid/$dev-$ino/parsed: Directory containing parsed DOF for a
 *    given DOF source.  Deleted when empty.
 *
 *    .../dof-pid/$pid/$dev-$ino/parsed/$prv$pid:$mod:$fun:$prb: parsed DOF for
 *    a given PID/mapping/probe triplet.  Generated from the helper for PIDs
 *    that are the first appearance of a given piece of DOF.  Deleted at daemon
 *    startup if the dof_parsed_t structure changed.  The file consists of a
 *    uint64_t version number (DOF_PARSED_VERSION), a uint64_t count of
 *    tracepoints, and then that many struct dof_parsed_t's of type
 *    DIT_TRACEPOINT.
 *
 * /run/dtrace/probes: Per-probe info, written by dtprobed, read by DTrace.
 *
 *    .../$pid/$prv$pid/$mod/$fun/$prb: Hardlink from $prv$pid:$mod:$fun:$prb
 *    above; parsed representation of one probe in a given process. Removed by
 *    dtprobed when the process dies, or if all mappings containing the probe
 *    are unmmapped.  Used by DTrace for tracing by PID.
 *
 *    (If a probe exists in multiple different mappings in a given process, they
 *    must all be identical.)
 *
 * In future, symlinks in /run/dtrace/probes/$prv$pid may exist, used for
 * globbing provider names across processes.
 */