[DTrace-devel] [PATCH v4 0/9] Add python, PCP support via libdtrace interfaces
Alan Maguire
alan.maguire at oracle.com
Wed Aug 12 13:22:01 UTC 2026
Having python bindings for libdtrace interfaces to compile, run
and collect information from DTrace programs is valuable because
it could help integration into metric collection frameworks like
PCP. This series adds python bindings for DTrace (patch 4)
and provides tests for them (patch 7). The support is
similar to what cmd/dtrace.c can do; compile a program,
enable probes, run it and collect data. Support is also
added to grab or create processes as is done by
dtrace -p, -c options. Aggregation snapshot walk is also
supported, with aggregation values filled out as raw values,
ints, lists of function names for stack keys and dicts for
quantized aggregations.
For detailed description/usage see the README in patch 4 and the
tests.
Patch 8 then provides a consumer of python support - a Performance
Co-Pilot (PCP) Performance Metric Data Agent (PMDA) which enables
flexible metric collection via DTrace scripts. See the README.md
in patch 8 for more details.
In order to support an environment where we have multiple handles
per process, some prep work is required. Patch 1 fixes some issues
with multiple handle use, while patch 2 improves sharing for
vmlinux BTF (and CTF generated from it if needed) which multiple
handles in a process image will benefit from. The upper bound
of per-process handles is now limited by the maximum number of
BPF programs attachable to a uprobe which is 64 (BPF_TRACE_MAX_PROGS)
which is encountered with attaches to the BEGIN probe.
Patch 3 makes available some of the functions used in calculating
stddev() values which is useful for the python bindings since they
do not use the print callbacks to do this.
Patch 4 is the cpython bindings themselves; the README.md describes
their usage. Patch 5 packages them and patches 6 and 7 facilitate
testing them.
Patch 8 then builds the PCP PMDA using the python bindings while
patch 9 packages them in a pcp-pmda-dtrace package.
Changes since v3:
- Fixed up issues identified by Kris around custom BTF path (patch 2)
- Added PCP patches 8/9
Changes since v2:
- Fixed some handle lifetime issues and related potential segfaults
in error paths (patch 4)
- Fixed up packaging to derive python package version from DTrace
version (patch 5)
Changes since v1:
- Fixed multi-handle issues (patches 1/2)
- Improved aggregation representation of stacks, added stddev and
*quantize representations (patch 4)
Alan Maguire (9):
libdtrace: Support multiple DTrace handles per process
libdtrace: share vmlinux BTF/CTF globally to support faster startup
libdtrace: Refactor math functions into dt_math.h
python: Add cpython bindings for libdtrace
dtrace.spec: add python bindings packaging
runtest.sh: Export PYTHONPATH when running tests in-tree
test: add tests for python bindings
dtrace: Add a PCP PMDA to expose DTrace data as metrics
dtrace.spec: Add optional PCP PMDA packaging pcp-pmda-dtrace
GNUmakefile | 2 +
bindings/Build | 35 +
bindings/python/README.md | 167 ++
bindings/python/pyproject.toml | 3 +
bindings/python/setup.py | 67 +
bindings/python/src/pydtrace_module.c | 1997 +++++++++++++++++
configure | 4 +-
dtrace.spec | 58 +-
libdtrace/dt_aggregate.c | 1 +
libdtrace/dt_bpf.c | 6 +-
libdtrace/dt_btf.c | 79 +-
libdtrace/dt_btf.h | 1 +
libdtrace/dt_consume.c | 340 +--
libdtrace/dt_impl.h | 7 +-
libdtrace/dt_math.h | 358 +++
libdtrace/dt_open.c | 5 +-
libdtrace/dt_printf.c | 1 +
libdtrace/dt_prov_dtrace.c | 6 +-
pcp/Build | 56 +
pcp/Install | 37 +
pcp/README.md | 260 +++
pcp/Remove | 27 +
pcp/autostart.d/.gitkeep | 0
pcp/dtrace.conf | 12 +
pcp/examples/packet_drop_reasons.d | 14 +
pcp/examples/packet_drop_reasons.json | 8 +
pcp/examples/syscall_counts.d | 11 +
pcp/examples/syscall_counts.json | 8 +
pcp/pmdadtrace.python | 1232 ++++++++++
runtest.sh | 2 +
test/unittest/python/tst.aggr-actions.sh | 133 ++
test/unittest/python/tst.aggr-change.sh | 75 +
test/unittest/python/tst.aggr-stack.sh | 61 +
test/unittest/python/tst.aggr.sh | 58 +
test/unittest/python/tst.exit.sh | 64 +
test/unittest/python/tst.multi-session-end.sh | 75 +
test/unittest/python/tst.multi-session.sh | 58 +
test/unittest/python/tst.proc-create.sh | 56 +
test/unittest/python/tst.proc-exit.sh | 57 +
test/unittest/python/tst.proc-grab.sh | 81 +
40 files changed, 5165 insertions(+), 357 deletions(-)
create mode 100644 bindings/Build
create mode 100644 bindings/python/README.md
create mode 100644 bindings/python/pyproject.toml
create mode 100644 bindings/python/setup.py
create mode 100644 bindings/python/src/pydtrace_module.c
create mode 100644 libdtrace/dt_math.h
create mode 100644 pcp/Build
create mode 100755 pcp/Install
create mode 100644 pcp/README.md
create mode 100755 pcp/Remove
create mode 100644 pcp/autostart.d/.gitkeep
create mode 100644 pcp/dtrace.conf
create mode 100644 pcp/examples/packet_drop_reasons.d
create mode 100644 pcp/examples/packet_drop_reasons.json
create mode 100644 pcp/examples/syscall_counts.d
create mode 100644 pcp/examples/syscall_counts.json
create mode 100755 pcp/pmdadtrace.python
create mode 100755 test/unittest/python/tst.aggr-actions.sh
create mode 100755 test/unittest/python/tst.aggr-change.sh
create mode 100755 test/unittest/python/tst.aggr-stack.sh
create mode 100755 test/unittest/python/tst.aggr.sh
create mode 100755 test/unittest/python/tst.exit.sh
create mode 100755 test/unittest/python/tst.multi-session-end.sh
create mode 100755 test/unittest/python/tst.multi-session.sh
create mode 100755 test/unittest/python/tst.proc-create.sh
create mode 100755 test/unittest/python/tst.proc-exit.sh
create mode 100755 test/unittest/python/tst.proc-grab.sh
--
2.43.5
More information about the DTrace-devel
mailing list