[DTrace-devel] changes needed for PIE executables

[Kris Van Hees]

On Thu, Nov 21, 2019 at 11:19:45PM +0000, Nick Alcock wrote:
> On 21 Nov 2019, David Mc Lean verbalised:
> 
> > The question now is how do we adapt to this new file format when running 
> > our USDT ?
> 
> This led me to do a bunch of digging around in code I have never really
> had reason to look at before in libdtrace/dt_link.c, in particular
> prepare_elf64() (and, I suppose, prepare_elf32(), though that is more or
> less legacy now). So this is mostly speculation on my part, but at least
> vaguely semi-half-informed speculation, honest. However, experimentation
> is needed to see how much of what I wrote below is accidental fiction.
> 
> When a final link is done with dtrace -G, dump_elf64() is eventually
> called. This calls prepare_elf64() to compute relocations and then
> writes them out in a normal SHT_RELA reloc+addend relocation table.
> prepare_elf64() derives the relocations to each usdt probe from the
> relocs in the DOF (DOF section DOF_SECT_URELHDR), and translates them
> into suitable relocs so that ld.so will fix up the DOF for us when the
> program is loaded. (The only alternative to that would be doing the same
> work ld.so is doing to fix up relocs *ourselves*, which is obviously
> pointless if ld.so can do it.)

Hm, no, the point at which we are creating the relocations is when we generate
the provider .o ELF object that holds the DOF that describes the probes in the
executable (or shared library).  So, the relocations that are created in
prepare_elf64() are based on relocations to (fake) probe functions (with a very
specific name that we can recognize).  Those relocations identify the probe
locations in the code.  THe DOF contains information about the offset of probes
in each function that contains probes, but since we won't know the address (or
offset within a section/segment) until the final linking of the executable we
need to generate relocations for references to the functions for which we have
probes.  The relocations point to fields in the DOF, and exist so that the
linker will fill in the correct information for those functions (symbols) in
the DOF.
> 
> Right now, on x86_64, we are writing out relocs of type
> R_X86_64_GLOB_DAT:
> 
>                 for (j = 0; j < nrel; j++) {
> #if defined(__i386) || defined(__amd64)
>                         rel->r_offset = s->dofs_offset +
>                             dofr[j].dofr_offset;
>                         rel->r_info = ELF64_R_INFO(count + dep->de_global,
>                             R_X86_64_GLOB_DAT);
> [...]
> 
> A GLOB_DAT simply returns the symbol value as a word64, which is why we
> set it here by adding the offset of the DOF section to the offset
> recorded in the DOF itself.

I think you are confusing offset and value.  Offset refers to the location
in the DOF section where this relocation needs to fill in a value.  The value
will be the final address of the symbol (based on the GLOB_DAT reloc type).

> This works for executables, but breaks completely for PIE executables
> (and I can't imagine it works too well for shared libraries, either).
> The problem is that this is an *absolute value*, but we want a
> section-relative value which ld.so can then adjust by the offset of the
> section.

We want a section-relative value that the DTrace kernel code can then adjust
based on the load address of the section.

> So we probably want to use R_X86_64_PC64, which also takes a word64 but
> subtracts the section offset from it (and adds an addend, which in this
> case we already hardwire to zero so we can ignore it).
> 
> The resulting value will then be the value relative to the start of the
> section, rather than the value relative to the start of the program.
> We can then fairly easily modify the kernel-side code to add the section
> base address to that (well, the segment base address, but for DOF I'm
> fairly sure we always put each DOF section in its own unique segment:
> please tell me if I'm wrong...)

Nothing needs to be modified on the kernel side because we already have code
that handles relocating DOF information based on a load address.  We just need
to ensure that we simply always pass in the load address of the code segment
along with the DOF.

> This is of course a semantic change to DOF, so this should probably bump
> the DOF version so the kernel code can distinguish old from new
> semantics and determine whether to add the base address or not.

No change in semantics as far as DOF is concerned because it is already designed
to support this.

> (Other supported architectures will no doubt need similar changes. Any
> arch supporting shared libraries should already have a suitable reloc
> type.)

We'll need to investigate that but yes, it is likely that some changes may be
needed in hos the DOF relocations are generated for each arch.

	Kris