[DTrace-devel] [PATCH] Set lockmem limit before checking BPF helper functions

[Kris Van Hees]

On Wed, Oct 15, 2025 at 12:47:52AM -0400, eugene.loh at oracle.com wrote:
> From: Eugene Loh <eugene.loh at oracle.com>
> 
> In dtrace_init(), we set the locked-memory limit, either to the
> user-specified value (if any) or to unlimited (by default).  We also check
> to make sure that certain BPF helper functions are available, falling
> over to alternatives or indicating they are not available in case of
> problems.
> 
> It is possible, however, that the limit is too low when dtrace starts,
> causing problems with the helper-function tests before dtrace_init()
> even has a chance to reset the limit.
> 
> Switch the order to set the limit before checking the helper functions.
> 
> A test is added.  The underlying problem, however, depends on kernel
> version, how locked memory is handled, the behavior of fallback
> functions, and so on.  So the test could easily pass on some systems
> even if the fix is not employed.
> 
> Signed-off-by: Eugene Loh <eugene.loh at oracle.com>

Reviewed-by: Kris Van Hees <kris.van.hees at oracle.com>

> ---
>  libdtrace/dt_open.c                    | 14 +++++++-------
>  test/unittest/misc/tst.lockmem-init.r  |  3 +++
>  test/unittest/misc/tst.lockmem-init.sh | 20 ++++++++++++++++++++
>  3 files changed, 30 insertions(+), 7 deletions(-)
>  create mode 100644 test/unittest/misc/tst.lockmem-init.r
>  create mode 100755 test/unittest/misc/tst.lockmem-init.sh
> 
> diff --git a/libdtrace/dt_open.c b/libdtrace/dt_open.c
> index 17dfbf9a6..54adec02a 100644
> --- a/libdtrace/dt_open.c
> +++ b/libdtrace/dt_open.c
> @@ -1213,13 +1213,6 @@ dtrace_init(dtrace_hdl_t *dtp)
>  		return dt_set_errno(dtp, EDT_CTF);
>  	}
>  
> -	/*
> -	 * Initialize the BPF library handling.
> -	 */
> -	dt_bpf_init(dtp);
> -	dt_btf_get_module_ids(dtp);
> -	dt_dlib_init(dtp);
> -
>  	/*
>  	 * Set the locked-memory limit.
>  	 */
> @@ -1228,6 +1221,13 @@ dtrace_init(dtrace_hdl_t *dtp)
>  	rl.rlim_cur = rl.rlim_max = lockmem;
>  	setrlimit(RLIMIT_MEMLOCK, &rl);
>  
> +	/*
> +	 * Initialize the BPF library handling.
> +	 */
> +	dt_bpf_init(dtp);
> +	dt_btf_get_module_ids(dtp);
> +	dt_dlib_init(dtp);
> +
>  	/*
>  	 * Initialize consume handling.
>  	 */
> diff --git a/test/unittest/misc/tst.lockmem-init.r b/test/unittest/misc/tst.lockmem-init.r
> new file mode 100644
> index 000000000..da8b2a5f9
> --- /dev/null
> +++ b/test/unittest/misc/tst.lockmem-init.r
> @@ -0,0 +1,3 @@
> +|Delay in ns needed in delay env|
> +-- @@stderr --
> +Delay in ns needed in delay env var.
> diff --git a/test/unittest/misc/tst.lockmem-init.sh b/test/unittest/misc/tst.lockmem-init.sh
> new file mode 100755
> index 000000000..1b4dcdeea
> --- /dev/null
> +++ b/test/unittest/misc/tst.lockmem-init.sh
> @@ -0,0 +1,20 @@
> +#!/bin/bash
> +#
> +# Oracle Linux DTrace.
> +# Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> +# Licensed under the Universal Permissive License v 1.0 as shown at
> +# http://oss.oracle.com/licenses/upl.
> +
> +dtrace=$1
> +
> +# Check that dtrace runs by default even if ulimit -l is very low.
> +ulimit -l 1
> +
> +$dtrace $dt_flags -c test/triggers/delaydie -qn '
> +syscall::write:entry
> +/pid == $target/
> +{
> +	printf("|%s|", copyinstr(arg1, 32));
> +}'
> +
> +exit $?
> -- 
> 2.47.3
>