[DTrace-devel] fail to build rpm package for dtrace-uitls

Tue Apr 24 02:12:05 PDT 2018

Hmm good question.

I think that we build more things than we ship in the end. For example
we need debug info for CTF generation.

Our configuration/specfiles live inside uek-rpm/ol7/...

Even for non-uek builds, I start with UEK configs as they are our supported
target. We are unable to test every possible kernel config option 
combination.

---
cp uek-rpm/ol7/config-x86_64 .config
make olddefconfig
make menuconfig <-- I usually turn of signatures that require OL 
certificates.
make
make ctf <-- important to get CTF data for kernel
make modules_install && make install
---

basically the contens of uek-rpm/ol7 are brought over to 
rpmbuild/SOURCES together
with kernel sources tar ball and build from there. I don't suggest this 
as it contains
kABI checks, signing, ... and it may depend on some OL packages that you 
have to
install first.

However the kernel-uek.spec should hold most answers for your questions.

Tomas

On 04/24/2018 10:55 AM, Fred Liu wrote:
> Tomas,
>
> Thanks! I will try my best.
>
> BTW, I notice the kernel package I built with all default options has
> the size more 400MB but yours is about 50MB in UEKR5 repo.
> Are they all about the parameters you fed to kernel configuration when
> you built it? Even, you added *uek*  strings which mine doesn't have
> to kernel name.
> Are these configurations in source packages too?
>
> Thanks.
>
> Fred
>
> 2018-04-24 16:44 GMT+08:00 Tomas Jedlicka <tomas.jedlicka at oracle.com>:
>> Hi Fred,
>>
>> I am glad you made it.
>>
>>
>> On 04/24/2018 06:22 AM, Fred Liu wrote:
>>> 2018-04-23 20:18 GMT+08:00 Tomas Jedlicka <tomas.jedlicka at oracle.com>:
>>>> Oh, I see. We will fix the readme. Thanks for bringing this up.
>>>>
>>>> The problem you see is caused by missing "kernel-uek-devel" package
>>>> installed with same version as spec file requires. This is a problem as
>>>> UEK5 has not been released yet and thus there is no such package
>>>> available yet (unless you build your own kernel-uek RPMs from github).
>>>>
>>>> You need to bootstrap the build and that is a bit tricky at this moment
>>>> as without dtrace-utils-devel you don't have DTrace headers installed.
>>>> At the same time you are trying to build dtrace-utils-devel.
>>>>
>>>> I am doing this boostrap procedure:
>>>>
>>>>       - Get your kernel (UEK5 or generic 4.14. with our patches) built and
>>>> booted into.
>>>>       - Install by hand DTrace headers from
>>>> <kernel>/include/uapi/linux/dtrace/* to the /usr/include/linux/dtrace/*
>>>>       - Use rpmbuild with the "build_local 1" define. This should pick
>>>> headers
>>>> from system location just fine
>>>>
>>>> If you hit problems with dtrace_kernels (other kernel-uek-devel packages
>>>> required to
>>>> produce translators) then you should start with reducing value of RPM
>>>> macro
>>>> dtrace_kernels
>>>> to only one you are currently running. Values of this macro are used to
>>>> search for installed kernel
>>>> devel sources in /lib/modules/<......>/build or /usr/src/kernels/<.....>.
>>> Wow, that is almost the same as my hacking before I asked in this mail!.
>>> I can use UEK5 master to generate rpm packages with all default kernel
>>> options:
>>>
>>> -rw-r--r--+ 1 root root 509370368 Apr 23 15:52
>>> kernel-4.14.32+-2.x86_64.rpm
>>> -rw-r--r--+ 1 root root 142234976 Apr 23 15:54
>>> kernel-devel-4.14.32+-2.x86_64.rpm
>>> -rw-r--r--+ 1 root root   1183044 Apr 23 15:52
>>> kernel-headers-4.14.32+-2.x86_64.rpm
>>>
>>> And it looks like kernel-headers-4.14.32+-2.x86_64.rpm has conflict
>>> and no use in installation.
>>
>> Yes that is correct. The OL comes with two kernels - RedHat(3.1) and UEK
>> (4.1/4.14)
>> Thus we have 2 kernel header packages. Installed are kernel-headers from
>> RedHat.
>> The kernel-uek-headers are not installed normally on a machine so we need to
>> deliver
>> our DTrace headers via separate package.
>>
>> On a signle-kernel distro or DTrace being upstream we could use
>> kernel-headers directly
>> and have simpler life :)
>>
>>> After booting into this new kernel I continue to build dtrace-utils
>>> rpm packages like your instruction with extra following hackings
>>>
>>> 1): ln -s /usr/src/kernels/4.14.32+(from
>>> kernel-devel-4.14.32+-2.x86_64.rpm)
>>> /usr/src/kernels/4.14.32-1.el7.centosuek.
>>> 2): updating spec file:
>>>
>>> %ifarch x86_64
>>> %if 0%{?oraclelinux} == 6
>>> %{!?build_kernel: %define build_kernel 4.1.12-112.14.10%{?dist}uek}
>>> %{!?dtrace_kernels: %define dtrace_kernels %{build_kernel}
>>> 3.8.13-118.19.4%{?dist}uek}
>>> %else
>>> %{!?build_kernel: %define build_kernel 4.14.32-1%{?dist}uek}
>>> %{!?dtrace_kernels: %define dtrace_kernels 4.14.32-1%{?dist}uek}  #fred
>>> %endif
>>> %else
>>
>> Yes this is what those tuneables are for. The "build_kernel" selects which
>> kernel-devel
>> is used to build dtrace-utils from (to pick dtrace headers).
>>
>> The second one is list of kernels to produce translators from (various .d
>> files).
>>
>> By using "local_build 1" you basically ask the packaging to ignore the
>> "build_kernel" and
>> use whatever you have in /usr/include/linux/dtrace/*
>>
>> You should be able to fine tun eboth from command line of rpmbuild too.
>>
>>
>>> And I finally get successful build:
>>>
>>> [root at pvg03 dtrace-utils]# rpmbuild -ba --define 'local_build 1'
>>> dtrace-utils.spec
>>> Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.vSnmST
>>> + umask 022
>>> + cd /root/rpmbuild/BUILD
>>> + cd /root/rpmbuild/BUILD
>>> + rm -rf dtrace-utils-1.0.0
>>> + /usr/bin/tar -xf /root/rpmbuild/SOURCES/dtrace-utils-1.0.0.tar.bz2
>>> + cd dtrace-utils-1.0.0
>>> + /usr/bin/chmod -Rf a+rX,u+w,g-w,o-w .
>>> + mkdir -p
>>> /root/rpmbuild/BUILD/dtrace-utils-1.0.0/usr/include/linux/dtrace
>>> + cp -dr --preserve=mode,timestamps /usr/include/linux/dtrace
>>> /root/rpmbuild/BUILD/dtrace-utils-1.0.0/usr/include/linux/
>>> + exit 0
>>> Executing(%build): /bin/sh -e /var/tmp/rpm-tmp.XVXEjx
>>> + umask 022
>>> + cd /root/rpmbuild/BUILD
>>> + cd dtrace-utils-1.0.0
>>> ++ getconf _NPROCESSORS_ONLN
>>> + make -j 4 VERSION=1.0.0 KERNELDIRPREFIX=/usr/src/kernels
>>> KERNELDIRSUFFIX= 'KERNELS=4.14.32-1.el7.centosuek '
>>> HDRPREFIX=/root/rpmbuild/BUILD/dtrace-utils-1.0.0/usr/include
>>> SED: libdtrace/signal.d
>>> AWK: libdtrace/errno.d
>>> CC: libdtrace/drti.c
>>> CC-32: libdtrace/drti.c
>>> YACC: libdtrace/dt_grammar.c
>>> MKNAMES: libdtrace/dt_names.c
>>> libdtrace/drti.c:52:1: warning: constructor priorities from 0 to 100
>>> are reserved for the implementation [enabled by default]
>>>    {
>>>    ^
>>> libdtrace/drti.c:52:1: warning: constructor priorities from 0 to 100
>>> are reserved for the implementation [enabled by default]
>>>    {
>>>    ^
>>> M4: libdtrace/procfs.d.in
>>> .........................
>>> ........................
>>> ........................
>>> equires: /bin/bash ld-linux-x86-64.so.2()(64bit)
>>> ld-linux-x86-64.so.2(GLIBC_2.3)(64bit) libc.so.6()(64bit)
>>> libc.so.6(GLIBC_2.14)(64bit) libc.so.6(GLIBC_2.2.5)(64bit)
>>> libc.so.6(GLIBC_2.3)(64bit) libc.so.6(GLIBC_2.4)(64bit)
>>> libc.so.6(GLIBC_2.5)(64bit) libc.so.6(GLIBC_2.8)(64bit)
>>> libdl.so.2()(64bit) libdtrace-ctf.so.1()(64bit)
>>> libdtrace-ctf.so.1(LIBDTRACE_CTF_1.0)(64bit)
>>> libdtrace-ctf.so.1(LIBDTRACE_CTF_1.3)(64bit) libdtrace.so.1()(64bit)
>>> libdtrace.so.1(LIBDTRACE_1.0)(64bit) libelf.so.1()(64bit)
>>> libelf.so.1(ELFUTILS_1.0)(64bit) libelf.so.1(ELFUTILS_1.5)(64bit)
>>> libelf.so.1(ELFUTILS_1.6)(64bit) libm.so.6()(64bit)
>>> libm.so.6(GLIBC_2.2.5)(64bit) libpthread.so.0()(64bit)
>>> libpthread.so.0(GLIBC_2.2.5)(64bit)
>>> libpthread.so.0(GLIBC_2.3.2)(64bit)
>>> libpthread.so.0(GLIBC_2.3.3)(64bit) librt.so.1()(64bit)
>>> librt.so.1(GLIBC_2.2.5)(64bit) libz.so.1()(64bit) rtld(GNU_HASH)
>>> Processing files: dtrace-utils-devel-1.0.0-10.el7.centos.x86_64
>>> Provides: dtrace-headers = 1.0.0 dtrace-utils-devel =
>>> 1.0.0-10.el7.centos dtrace-utils-devel(x86-64) = 1.0.0-10.el7.centos
>>> Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1
>>> rpmlib(FileDigests) <= 4.6.0-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
>>> Requires: libc.so.6()(64bit) libc.so.6(GLIBC_2.2.5)(64bit)
>>> libdtrace-ctf.so.1()(64bit) libdtrace.so.1()(64bit)
>>> libdtrace.so.1(LIBDTRACE_PRIVATE)(64bit) libelf.so.1()(64bit)
>>> libelf.so.1(ELFUTILS_1.0)(64bit) libelf.so.1(ELFUTILS_1.5)(64bit)
>>> rtld(GNU_HASH)
>>> Obsoletes: dtrace-modules-shared-headers dtrace-modules-provider-headers
>>> Processing files: dtrace-utils-testsuite-1.0.0-10.el7.centos.x86_64
>>> Provides: dtrace-utils-testsuite = 1.0.0-10.el7.centos
>>> dtrace-utils-testsuite(x86-64) = 1.0.0-10.el7.centos
>>> libproc-dlmlib.so.0()(64bit) libproc-lookup-victim-lib.so.0()(64bit)
>>> Requires(rpmlib): rpmlib(FileDigests) <= 4.6.0-1
>>> rpmlib(PayloadFilesHavePrefix) <= 4.0-1 rpmlib(CompressedFileNames) <=
>>> 3.0.4-1
>>> Checking for unpackaged file(s): /usr/lib/rpm/check-files
>>> /root/rpmbuild/BUILDROOT/dtrace-utils-1.0.0-10.el7.centos.x86_64
>>> Wrote: /root/rpmbuild/SRPMS/dtrace-utils-1.0.0-10.el7.centos.src.rpm
>>> Wrote:
>>> /root/rpmbuild/RPMS/x86_64/dtrace-utils-1.0.0-10.el7.centos.x86_64.rpm
>>> Wrote:
>>> /root/rpmbuild/RPMS/x86_64/dtrace-utils-devel-1.0.0-10.el7.centos.x86_64.rpm
>>> Wrote:
>>> /root/rpmbuild/RPMS/x86_64/dtrace-utils-testsuite-1.0.0-10.el7.centos.x86_64.rpm
>>> Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.MDSiyw
>>> + umask 022
>>> + cd /root/rpmbuild/BUILD
>>> + cd dtrace-utils-1.0.0
>>> + '[' /root/rpmbuild/BUILDROOT/dtrace-utils-1.0.0-10.el7.centos.x86_64
>>> '!=' / ']'
>>> + rm -rf /root/rpmbuild/BUILDROOT/dtrace-utils-1.0.0-10.el7.centos.x86_64
>>> + rm -rf /root/rpmbuild/BUILD/dtrace-utils-1.0.0
>>> + exit 0
>>> [root at pvg03 dtrace-utils]#
>>>
>>>
>>> Thanks!
>>>
>>> Fred
>>>
>> If you have any idea how to improve the build system/packaging, please, let
>> us know!
>>
>> Tomas