[DTrace-devel] [oracle/dtrace-utils] 03ac5b: lexer: fix top-level probe ending in wildcard ambi...

[euloh]

  Branch: refs/heads/devel
  Home:   https://github.com/oracle/dtrace-utils
  Commit: 03ac5b8b7f2a012148a36f82eedb2e09d373ef48
      https://github.com/oracle/dtrace-utils/commit/03ac5b8b7f2a012148a36f82eedb2e09d373ef48
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M libdtrace/dt_lex.l

  Log Message:
  -----------
  lexer: fix top-level probe ending in wildcard ambiguity handling

The lexer contains code to determine whether

foo*bar

at the top level of a script is a wildcarded probe name or a variable
declaration (which would need a semicolon or an = or something else too,
but maybe the lexer hasn't got to it yet). Back in the Solaris days this
used to be done by just blindly looking up the stuff before the * as a
type and considering it not a type if it wasn't, but this forces loading
of all CTF basically no matter what (even the hardwired definition of
NULL forces it). I rejigged this code way back in 5a7483d86 in 2012 to
stop it looking up quite so many types unnecessarily, but in the process
broke the code that puts the parts of the token after the * back on
the parser stack again.  You need to be very unlucky for this to trigger
anything, but more recent flex has started actually doing what it has
long promised and made unput() *actually* corrupt yytext: put the two
together and bugs jump out and you start getting garbage or repeated
junk after the * in cases like int*x (seen in
test/unittest/options/err.pspec-default.d).

Getting it right is actually simpler than getting it wrong: we don't
need to take a temporary copy of the yytext to avoid the problem of its
getting corrupted if we use yyless(), which does not corrupt yytext,
rather than unput(); doing this simplifies the code that preserves the
content beyond the * as well.  (The rest of the lexer is still using
unput() everywhere and needs fixing later.)

This does not fix err.pspec-default.d, but it's a prerequisite for a fix
that works (so err.pspec-default.d will serve as a test for it).

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: c5f17d9ccbce9bb94bc38c4af66270b310ba83bc
      https://github.com/oracle/dtrace-utils/commit/c5f17d9ccbce9bb94bc38c4af66270b310ba83bc
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M libdtrace/dt_cc.c
    M libdtrace/dt_lex.l
    M libdtrace/dt_parser.h
    M libdtrace/dt_pcb.h

  Log Message:
  -----------
  lexer, parser: remove manual implementation of parsing from a string

dt_lex.l has had code (in one form or another) to parse input from a
string since the beginning.  I rewrote it in 2013 to use a dt_input
function rather than overriding YY_INPUT or simply overriding input()
(which flex has never allowed), but the code was still there.

This is ridiculous.  flex has had code to parse from strings since 1994,
so long ago that it predates any extant flex tarballs that I've been
able to find, and long before DTrace's lexer was written (but of course
it was written for an AT&T lex which had hardly been touched since the
1980s).  It's a bit different from what DTrace is doing now -- you have
to call yy_scan_string(), which gives you back a yy_buffer_state handle
that you free with yy_delete_buffer() -- but it works, and it means we
can rip out half the code in dt_input, leaving only the code to do
buffered reads from actual files.

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: 95140dae9d30e22558e94263db13338303739932
      https://github.com/oracle/dtrace-utils/commit/95140dae9d30e22558e94263db13338303739932
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M test/unittest/options/err.pspec-default.r
    R test/unittest/options/err.pspec-default.r.p

  Log Message:
  -----------
  Revert "test, options: work around a subtle lexer bug"

This reverts commit ae9f2a137438b2e3798ff532bd986fe3abfd9c8a.

(Technically this is not a strict reversion: we fix the line number too,
because the previous fixes slightly change the reported error.  But
conceptually it is one.)

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: d21ee3df07eabcc33838452c5270443fef281b7e
      https://github.com/oracle/dtrace-utils/commit/d21ee3df07eabcc33838452c5270443fef281b7e
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M libdtrace/dt_lex.l

  Log Message:
  -----------
  lexer: allow `-prepended identifiers to start with numerals

This is necessary to be able to cite names in some modules by explicit
module name, since module names can and do start with numerals.

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: 5ec88d36c405de06240d45abe9bb32394e038aae
      https://github.com/oracle/dtrace-utils/commit/5ec88d36c405de06240d45abe9bb32394e038aae
  Author: Nick Alcock <nick.alcock at oracle.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M GNUmakefile
    A dtprobed/50-dtprobed.preset
    M dtprobed/Build
    M dtrace.spec

  Log Message:
  -----------
  dtprobed: use presets properly for daemon restarting

It turns out that systemd supports per-package drop-ins for presets,
so we can use presets to enable the daemon and leave by-hand enabling/
disabling/restarting for the individual sysadmin (as might be the
intent in a world with presets: I'm not sure, this stuff is very
lightly documented).

As a bonus, doing it this way does not depend on the state of the old
package at all: the restarting is done by setting a property on the
requested service, then triggering a reload/restart of everything with that
property set (see /usr/lib/systemd/systemd-update-helper).  So we can
switch to using %systemd_postun_with_restart in %postun, and everything
will just work (including from previous versions of this package, though
upgrades from .12 will always produce a harmless error message: nothing
we can do, the bug is in .12).

Signed-off-by: Nick Alcock <nick.alcock at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: 3ad8eb84050b79d0fb3dca9755c909752df94df0
      https://github.com/oracle/dtrace-utils/commit/3ad8eb84050b79d0fb3dca9755c909752df94df0
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M libdtrace/Build

  Log Message:
  -----------
  Suppress pedantic warnings for dt_btf.c

The file includes include/linux/btf.h, which has an unnamed
union.  Suppress the warning
    ISO C99 doesn?t support unnamed structs/unions [-Wpedantic]

Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


  Commit: 7e159efa0343e6003666348b1aba2dc39045c43a
      https://github.com/oracle/dtrace-utils/commit/7e159efa0343e6003666348b1aba2dc39045c43a
  Author: Eugene Loh <eugene.loh at oracle.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M libdtrace/dt_options.c
    A test/unittest/misc/err.lockmem-pragma.r
    A test/unittest/misc/err.lockmem-pragma.sh
    R test/unittest/misc/tst.lockmem-pragma.r
    R test/unittest/misc/tst.lockmem-pragma.sh
    R test/unittest/misc/tst.lockmem-pragma.x

  Log Message:
  -----------
  Forbid setting lockmem value with a pragma

With commit
5a12c51c983f "options: ensure lockmem is set before retrieving probe info",
lockmem is set early.  Therefore, forbid setting its value in a pragma and
modify the associated test.

Signed-off-by: Eugene Loh <eugene.loh at oracle.com>
Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>


Compare: https://github.com/oracle/dtrace-utils/compare/b1897656147b...7e159efa0343