[DTrace-devel] [oracle/dtrace-utils] c7a39f: Fix double declaration of yylineno

Thu Oct 7 10:26:47 PDT 2021

  Branch: refs/heads/1.x-branch-dev
  Home:   https://github.com/oracle/dtrace-utils
  Commit: c7a39f492293d09d57ec4afb78724d9e1cd63931
      https://github.com/oracle/dtrace-utils/commit/c7a39f492293d09d57ec4afb78724d9e1cd63931
  Author: Kris Van Hees <kris.van.hees at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M libdtrace/dt_cc.c

  Log Message:
  -----------
  Fix double declaration of yylineno

The yylineno variable was declared in dt_cc.c *and* in dt_lex.c (which
is generated from dt_lex.l).  This poses a compilation issue with some
newer compilers.  The declaration in dt_cc,c has been changed into an
extern declaration, while the one in dt_lex.c is kept as the actual
variable declaration.

Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>

  Commit: 8208ebbeb1b0fc96eeec02c5863af0b5b7746a5b
      https://github.com/oracle/dtrace-utils/commit/8208ebbeb1b0fc96eeec02c5863af0b5b7746a5b
  Author: Kris Van Hees <kris.van.hees at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M libproc/mkoffsets.sh

  Log Message:
  -----------
  libproc: remove const from declarations

The r_debug_offsets and link_map_offsets arrays were declared 'const'.
This is causing issues on some newer systems where these arrays are
stored in memory that is marked RO.  Performing memcpy() and memmove()
on these areas triggers on error.

Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>
Reviewed-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>

  Commit: 002fc5bcc106b6d61b4989f595e403d556ecbe3e
      https://github.com/oracle/dtrace-utils/commit/002fc5bcc106b6d61b4989f595e403d556ecbe3e
  Author: Kris Van Hees <kris.van.hees at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M GNUmakefile
    M Makeconfig

  Log Message:
  -----------
  Detect whether libctf is available (from binutils)

The mechanism introduced here will detect (at build time) whether the
libctf library (from binutils) is available.  If so, HAVE_LIBCTF will
be defined in config.h and config.mk.  The latter is included in the
main GNUmakefile so that it can be used in Build component files.

Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>

  Commit: 188bf56e11a226f1e8063ebe6f5eec435db2b714
      https://github.com/oracle/dtrace-utils/commit/188bf56e11a226f1e8063ebe6f5eec435db2b714
  Author: Kris Van Hees <kris.van.hees at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M cmd/Build
    A include/sys/ctf-api.h
    M libdtrace/Build
    A libdtrace/dt_bpf_maps.h
    M libdtrace/dt_decl.h
    M libdtrace/dt_ident.h
    M libdtrace/dt_impl.h
    M libdtrace/dt_module.c
    M libdtrace/dt_parser.h
    M libdtrace/dt_printf.h
    M libdtrace/dt_xlator.h
    M libdtrace/dtrace.h
    M libproc/libproc.h

  Log Message:
  -----------
  Build against libctf (from binutils) if available

CTF suypport can be provided by the libdtrace-ctf library (originally
part of the legacy DTrace implementation) or by the libctf that is
included in newer binutils versions.  The config detection mechanism
added to determine whether libctf is available provides a HAVE_LIBCTF
define to use in source code and also a HAVE_LIBCTF symbol that can be
tested for in makefiles.

This patch provides the changes in the DTrace v2 source tree to build
against libctf when it is available.  When it is not available, building
agaist libdtrace-ctf is attempted (and if that is not found a build
error will result).

Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>

  Commit: 4e5912ee411347abd6dc3441dfb42a7fdab8963e
      https://github.com/oracle/dtrace-utils/commit/4e5912ee411347abd6dc3441dfb42a7fdab8963e
  Author: Kris Van Hees <kris.van.hees at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M uts/common/sys/dtrace_types.h

  Log Message:
  -----------
  Ensure all datatypes are available for compilation

The changes to make building against libctf as an alternative to
libdtrace-ctf caused a compilation failure if libdtrace-ctf was not
installed.  The cause was a dependency on some datatype declarations
defined by the libdtrace-ctf include files.  These declarations have
been moved to dtrace_types.h to ensure that they are available
regardless of which CTF library is used during compilation.

Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>

  Commit: 02b481dbe4adae86804ac64728124fc7dce814db
      https://github.com/oracle/dtrace-utils/commit/02b481dbe4adae86804ac64728124fc7dce814db
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M test/unittest/offsetof/err.D_UNKNOWN.OffsetofNULL.r

  Log Message:
  -----------
  tests: adjust for new libctf error message

As of the deduplicator series, a bunch of libctf error messages
changed for clarity: one change affects the DTrace testsuite.

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>

  Commit: 35646889bf7dcfb4bf2bf8de340d54f56e993e0b
      https://github.com/oracle/dtrace-utils/commit/35646889bf7dcfb4bf2bf8de340d54f56e993e0b
  Author: Kris Van Hees <kris.van.hees at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M runtest.sh

  Log Message:
  -----------
  Support both old and new error message from CTF

As mentioned in an earier commit (6ca7f54a 'tests: adjust for new
libctf error message'), CTF changed the wording of an error message.
The runtest.sh script will now substitute the new message for the old
one in captured error output to ensure that tests will work correctly
against older and newer CTF libraries.

Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>

  Commit: c0180e6b175423b386b660ec2f8eb69f4baa451b
      https://github.com/oracle/dtrace-utils/commit/c0180e6b175423b386b660ec2f8eb69f4baa451b
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M libdtrace/dt_parser.c

  Log Message:
  -----------
  parser: support slices of typedefs, etc

Slices in CTF are almost totally transparent to the user, *except* that
ctf_type_resolve will halt when it encounters a slice and will drill no
further, since going further would lose information about the encoding
of the slice.  Since you can have slices of typedefs, this means you
can't rely on ctf_type_resolve getting rid of all typedefs any more.

This change drills through to base types where necessary -- but I
suspect that what we actually want is a ctf_type_resolve_me_harder that
drills through slices too... but I have no idea what to call it
(ctf_type_resolve_base?), and user code will still need adjustment
because you cannot safely call ctf_type_encoding on a type returned from
such a function (the slice, if any, has been sheared away).

This is in aid of trying to get libctf working with DTrace v1, but
should apply to v2 as well, or something similar should, if we want to
do things this way rather than adding a ctf_type_resolve_base to libctf
and using that.

  Commit: fcfefeea8b0e71f29548d20d0e06882db3da90f1
      https://github.com/oracle/dtrace-utils/commit/fcfefeea8b0e71f29548d20d0e06882db3da90f1
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    R include/sys/ctf-api.h
    M runtest.sh
    A uts/common/sys/ctf-api.h

  Log Message:
  -----------
  test: fix CTF-using tests in the absence of libctf

Some tests include <sys/ctf-api.h>, so we have to make sure the
compatibility wrapper for this header is somewhere the tests can see it,
and that config.h is visible to them too (for HAVE_LIBCTF).

None of these headers are installed, so there's no impact on users.

  Commit: f25f5efb53823399c9fe7af27fa553a9cca10be4
      https://github.com/oracle/dtrace-utils/commit/f25f5efb53823399c9fe7af27fa553a9cca10be4
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M libdtrace/procfs.d.in

  Log Message:
  -----------
  translators: procfs.d: rename projid_t

The Linux kernel already has a type named projid_t, with a definition
that conflicts with that in use (for an unused field) by this
translator.  libctf correctly diagnoses this as a conflict (where
libdtrace-ctf didn't): rename the type in the translator.

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>

  Commit: 917254256be326abebf4c7ab3defbae91c13495f
      https://github.com/oracle/dtrace-utils/commit/917254256be326abebf4c7ab3defbae91c13495f
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M libdtrace/dt_cg.c
    M libdtrace/dt_decl.c
    M libdtrace/dt_ident.c
    M libdtrace/dt_impl.h
    M libdtrace/dt_module.c
    M libdtrace/dt_parser.c
    M libdtrace/dt_parser.h
    M libdtrace/dt_printf.c
    M libdtrace/dt_xlator.c
    M libdtrace/dt_xlator.h

  Log Message:
  -----------
  parser, xlators, printf, cg, ident: resolve forwards to real types

In contrast to libdtrace-ctf (which didn't reliably handle conflicted
structs and unions at all and often just assumed they weren't
conflicting, leading to corrupted type graphs), libctf's deduplicator
reliably identifies conflicted structs and unions, and avoids getting
caught in cycles while maximizing the number of shared types by
routinely converting conflicted structs and unions into forwards and
leaving the forward in the shared dict as a marker.

This means that forwards can and do appear in the shared graph in places
where they would be forbidden in C: you can and do see things like
arrays of forwards.  We'd rather not inflict this on our users, so
introduce a new dt_type_resolve which is like ctf_type_resolve except
that rather than returning a forward when called on a type in the shared
dict, it hunts through the module list for the first non-forward with a
matching name and suitable kind (so a forward to 'struct foo' will only
match a real 'struct foo', not 'union foo').  It only returns a forward
if no suitable is found in any module at all.

This search can be relatively expensive, because it might involve
loading lots of modules, but it only happens when conflicted forwards
are found, which isn't all that common: and correctness trumps
performance anyway.

This is made a bit more complex because there were several places where
we weren't bothering to call ctf_type_resolve at all but where a
resolution is nonetheless required in any case (e.g. dt_idsize_type)
and a number of places where we were just *assuming* that
ctf_type_resolve succeeded (most unwise).  There are also a number of
places where we need to track both the fp of the original type and the
fp of the resolved type, notably in xlator-related code.

A couple of places have to *avoid* searching for real types, in
particular if a forward is found in the C and D dicts: dt_type_resolve
takes a flag to turn this off.  This was spotted heuristically (i.e.
a test failed): I don't *think* there are any places where this needs
doing that I missed, but I could be wrong.

(In the future some of this commit may be unwindable, if and when libctf
gains the ability to search for forwards in archives in the same way we
are doing here.  It seems like a worthwhile API addition.)

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>

  Commit: a8c9a6b3b4ff601ea88179af152304140b9842b7
      https://github.com/oracle/dtrace-utils/commit/a8c9a6b3b4ff601ea88179af152304140b9842b7
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    A test/internals/libproc/tst.plddGrab.r.p

  Log Message:
  -----------
  test, libproc: edit out ksplice preload if seen

ksplice LD_PRELOADs libksplice_helper.so, which itself depends on
libdl.so.2.  Drop them both if seen (since the plddGrab trigger
doesn't use either).

(Ancient patch, accidentally left around in my working tree...)

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>

  Commit: 290f493d15f81d78d6266b27198c43d0be299237
      https://github.com/oracle/dtrace-utils/commit/290f493d15f81d78d6266b27198c43d0be299237
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    A break.gdb
    M runtest.sh

  Log Message:
  -----------
  runtest: allow error messages to end in periods

libctf errors now end this way.

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>

  Commit: 02abf706bfcc52d09b4d6e8d103080c01d108252
      https://github.com/oracle/dtrace-utils/commit/02abf706bfcc52d09b4d6e8d103080c01d108252
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-04-12 (Mon, 12 Apr 2021)

  Changed paths:
    M test/unittest/union/err.D_DECL_INCOMPLETE.circular.r
    M test/unittest/union/err.D_DECL_INCOMPLETE.order.r
    A test/unittest/union/err.D_DECL_INCOMPLETE.order.r.p
    A test/unittest/union/err.D_DECL_INCOMPLETE.recursive.r.p
    M test/unittest/union/err.D_DECL_INCOMPLETE.simple.r
    A test/unittest/union/err.D_DECL_INCOMPLETE.simple.r.p

  Log Message:
  -----------
  test, union: adjust libdtrace-ctf errors to libctf ones

The incomplete union tests trip a libdtrace-ctf bug (fixed in libctf in
binutils commit b4b6ea46807ec9c01ed4f) which misprint forwards to 'union
foo' as if they were 'struct foo'.

Adjust the .r from affected tests to use the right error, and introduce
.r.p's to translate the wrong error to the right one.

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>

  Commit: 9044806730593d97fecbf28c2544c94e93d93347
      https://github.com/oracle/dtrace-utils/commit/9044806730593d97fecbf28c2544c94e93d93347
  Author: Kris Van Hees <kris.van.hees at oracle.com>
  Date:   2021-10-06 (Wed, 06 Oct 2021)

  Changed paths:
    M libdtrace/procfs.d.in

  Log Message:
  -----------
  Update procfs.d.in for T->state to T->__state change

Starting with v5.14 kernels, the 'state' member in the task_struct has
been renamed as '__state'.

Signed-off-by: Kris Van Hees <kris.van.hees at oracle.com>
Reviewed-by: Eugene Loh <eugene.loh at oracle.com>

  Commit: 9ffab9707a10b779255f7baca99d82bf17890151
      https://github.com/oracle/dtrace-utils/commit/9ffab9707a10b779255f7baca99d82bf17890151
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-10-06 (Wed, 06 Oct 2021)

  Changed paths:
    M libdtrace/io.d.in

  Log Message:
  -----------
  translators: io.d: more bio changes

The bdev is more easily accessible now, like it was in kernels < 4.14.
The bi_disk is gone, moved into the bd_disk.

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>

  Commit: 0449a78c3ecb09ff80b2aa4c66ebd327b2a48d19
      https://github.com/oracle/dtrace-utils/commit/0449a78c3ecb09ff80b2aa4c66ebd327b2a48d19
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2021-10-06 (Wed, 06 Oct 2021)

  Changed paths:
    M uts/common/sys/dtrace_types.h

  Log Message:
  -----------
  uts: work with recent kernel uapi headers

These headers define dtrace_id_t themselves, but set
_DTRACE_ID_T_DEFINED when they do so.  Avoid redefining it ourselves in
this case.

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>

Compare: https://github.com/oracle/dtrace-utils/compare/cc2ce5893884...0449a78c3ecb