[DTrace-devel] [oracle/dtrace-utils] 9f6fc1: Handle unlabeled BPF_NOP instructions

[euloh]

  Branch: refs/heads/dev
  Home:   https://github.com/oracle/dtrace-utils
  Commit: 9f6fc198f353e82a5b965423df4c1acc2bf40394
      https://github.com/oracle/dtrace-utils/commit/9f6fc198f353e82a5b965423df4c1acc2bf40394
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2021-01-14 (Thu, 14 Jan 2021)

  Changed paths:
    M libdtrace/dt_as.c
    M libdtrace/dt_dis.c

  Log Message:
  -----------
  Handle unlabeled BPF_NOP instructions

In BPF code generation, we emit branches with a label id stored in
the offset field, to be resolved during assembly.  Since "jmp 0" is
used to encode a NOP in BPF, we need to exclude jumps with offset 0
from jump-target relocation.

Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: e8a21c2b24e11ff956f22d86c5e7aafd43d5f299
      https://github.com/oracle/dtrace-utils/commit/e8a21c2b24e11ff956f22d86c5e7aafd43d5f299
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2021-01-14 (Thu, 14 Jan 2021)

  Changed paths:
    M test/unittest/aggs/tst.max.d
    M test/unittest/aggs/tst.max_neg.d
    M test/unittest/aggs/tst.max_neg.r
    M test/unittest/aggs/tst.min.d
    M test/unittest/aggs/tst.min.r
    M test/unittest/aggs/tst.min_neg.d

  Log Message:
  -----------
  Test min/max aggregation initialization

The min and max aggregation functions have a special code path in
which initial values are set to max and min values, respectively.
All other aggregation functions expect zeroes for initialization.
The min and max tests include zeroes as values, and therefore do
not detect if the special code path is not used.

Change the min and max tests to use values of a particular sign
(positive by default, negative in the _neg variants) to exercise
the special code path.

Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: ba5ab3e2ab37acf5dbcb3761ba6e9e3fb4e7053d
      https://github.com/oracle/dtrace-utils/commit/ba5ab3e2ab37acf5dbcb3761ba6e9e3fb4e7053d
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2021-01-14 (Thu, 14 Jan 2021)

  Changed paths:
    M test/unittest/aggs/err.D_CG_EXPR.from_expr.d
    M test/unittest/aggs/err.D_NORMALIZE_AGGARG.bad.d
    M test/unittest/aggs/err.D_NORMALIZE_PROTO.bad.d
    M test/unittest/aggs/err.D_NORMALIZE_SCALAR.bad.d
    R test/unittest/aggs/err.D_PROTO_LEN.quantize_too_amny_args.d
    A test/unittest/aggs/err.D_PROTO_LEN.quantize_too_many_args.d

  Log Message:
  -----------
  Minor aggregations test cleanup

Typo in test name.  Nonsensical references to "clear" section.

Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: ee132c78f99ccae53f22403d2900e0630f51bd4d
      https://github.com/oracle/dtrace-utils/commit/ee132c78f99ccae53f22403d2900e0630f51bd4d
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2021-01-14 (Thu, 14 Jan 2021)

  Changed paths:
    M libdtrace/dt_aggregate.c
    M libdtrace/dt_bpf.c
    M libdtrace/dt_cg.c
    M libdtrace/dt_map.c
    A test/unittest/aggs/tst.empty.d
    A test/unittest/aggs/tst.empty.r

  Log Message:
  -----------
  Switch from per-CPU latches to per-CPU/per-agg latches

Change the latch sequence number to be per-aggregation, not only per-CPU.
This way, one can see whether an aggregation has any data or not.  (Some
aggregations, like count(), already inherently have such information, but
we need to handle the general case.)

This change has the added advantage of reducing producer-consumer conflicts.
E.g., imagine that the producer is firing very often, updating different
aggregations, while the consuming is trying to read a single, large
aggregation (e.g., some *quantize aggregation).  A single latch sequence
number for all aggregations would keep updating, causing the consumer to
keep rereading an aggregation unnecessarily.

Also, when taking snapshots of aggregations, copy only a single copy of
the aggregation data.  Neither the latch sequence number nor the second
copy are needed.

This also solves the problem that aggregations that appear in a D script
were being printed even if the probes that use them never fire.

Introduce a test to catch the printing of unused aggregations.

https://github.com/oracle/dtrace-utils/issues/3
Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: 7639f3cc17e3677d09bfc01174d815f76c25b135
      https://github.com/oracle/dtrace-utils/commit/7639f3cc17e3677d09bfc01174d815f76c25b135
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2021-01-14 (Thu, 14 Jan 2021)

  Changed paths:
    M libdtrace/dt_aggregate.c
    M test/unittest/aggs/tst.llquantize_basic.d
    A test/unittest/aggs/tst.reset.d
    A test/unittest/aggs/tst.reset.r
    M test/unittest/multiaggs/tst.same.d

  Log Message:
  -----------
  Aggregations are not appropriately reset

In DTv1, each user-space aggregation snapshot would read data from the
kernel and aggregate, not only over CPUs but also with previous
snapshots.

In DTv2, the kernel maintains aggregations in BPF maps.  Each user-space
snapshot should still aggregate over CPUs, but not with previous data.

Reset aggregations appropriately with each snapshot.

Tweak testing to catch this problem.  The test suite will catch this
problem even more once other functionality has been implemented.

https://github.com/oracle/dtrace-utils/issues/4
Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: 8068b2c6251a8aa71e7b9c16c2af4d5c34ab4ab0
      https://github.com/oracle/dtrace-utils/commit/8068b2c6251a8aa71e7b9c16c2af4d5c34ab4ab0
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2021-01-14 (Thu, 14 Jan 2021)

  Changed paths:
    M libdtrace/dt_aggregate.c
    M libdtrace/dt_cg.c
    M libdtrace/dt_impl.h
    M libdtrace/dt_map.c

  Log Message:
  -----------
  Introduce mechanism to turn off dual-copy aggregation code

To provide the producer a lock-free write mechanism for aggregations,
we have two copies of aggregations.  The producer updates both, in
turn, and the consumer reads the one not being written, as indicated
by a latch sequence number.

Currently, however, we are using older kernels that do not allow us
to mmap BPF maps.  The consumer is forced to lookup the entire BPF
aggregation map at once, making the dual-copy mechanism useless.
Meanwhile, the dual copies are consuming valuable BPF map space.

Introduce a preprocessor macro DT_AGG_NUM_COPIES that can be set to
turn off the dual-copy mechanism for now while retaining the code for
when we want to restore the mechanism.

For example, the D script
    BEGIN {x = 100; @ = lquantize(x, 0, 2045, 1); exit(0) }
runs, but increasing to 2046 fails.  With DT_AGG_NUM_COPIES=1,
one can go up to 4093.

The latch sequence number is preserved since it is used to see
which aggregations have data and should be reported.

Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: 4413e6db035add7dce7d73b848ebf4a3edab0797
      https://github.com/oracle/dtrace-utils/commit/4413e6db035add7dce7d73b848ebf4a3edab0797
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2021-01-14 (Thu, 14 Jan 2021)

  Changed paths:
    M libdtrace/dt_printf.c
    M test/unittest/aggs/tst.clearavg2.d
    M test/unittest/aggs/tst.multiaggs3.d
    M test/unittest/printa/tst.basics.d
    M test/unittest/printa/tst.basics.r

  Log Message:
  -----------
  Formatted printa() is broken

A few tests were failing due simply to this bug.  Their XFAIL
annotations should be removed.  Add stddev and llquantize to
tst.basics.d so that all aggregation functions are tested;  each
function causes a specific callback to be used.

https://github.com/oracle/dtrace-utils/issues/5
Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>
Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


Compare: https://github.com/oracle/dtrace-utils/compare/4439e6bea4a7...4413e6db035a