Repository navigation
ENT-14716: Fixed building core from source on RHEL 8 and 9 in ci scripts - #6472
aleksandrychev wants to merge 1 commit into
Conversation
craigcomstock
left a comment
craigcomstock
left a comment
There was a problem hiding this comment.
This will work but I think we are dirtying up what should be fairly straightforward scripts that either work or don't. So I would argue that rhel-8 should maybe just fail due to age and if need be we could refactor libntech to macro ifdef around the pcre2 differences.
The better solution to fixing things here, I think, is to migrate away from requiring CFEngine to be installed on build hosts. See https://northerntech.atlassian.net/browse/ENT-14330
|
|
||
| # rhel-8 has pcre2 10.32 but libntech needs at least 10.34 (PCRE2_SUBSTITUTE_MATCHED, PCRE2_SUBSTITUTE_LITERAL). | ||
| # Build it into /opt/pcre2, away from the system pcre2, and ci/configure.sh will use it from there. | ||
| build_pcre2() { |
There was a problem hiding this comment.
If libntech needs a particular version of pcre2 then libntech should specify that and we should use that specification here.
There was a problem hiding this comment.
We should just move this to buildscripts/ci/setup-cfengine-build-host.sh so we have the version there if we need to use it.
| echo building pcre2 in "$tmpdir" | ||
| ( | ||
| cd "$tmpdir" | ||
| curl --fail --location --silent --show-error --output pcre2.tar.gz https://github.com/PCRE2Project/pcre2/releases/download/pcre2-10.49/pcre2-10.49.tar.gz |
There was a problem hiding this comment.
If we are using a version from buildscripts and want to stay aligned we should grab the version FROM there instead of hard-coding it here and allowing the version to rot.
build_lmdb installed liblmdb into /usr/lib, which the runtime linker does not search on lib64 distributions, so cf-key failed on RHEL 9. It now installs into /usr/lib64 when that exists. RHEL 8 ships pcre2 10.32, which lacks declarations libntech uses. ensure_pcre2 reads them from libntech's AC_CHECK_DECLS line, and when the system pcre2 lacks one, build_pcre2 builds the release pinned in buildscripts deps-packaging into /opt/pcre2, which ci/configure.sh then passes as --with-pcre2. Ticket: ENT-14716 Changelog: none Signed-off-by: Ihor Aleksandrychiev <ihor.aleksandrychiev@northern.tech>
1ec3406 to
3a5f8b1
Compare
|
|
||
| # libntech's configure.ac lists the pcre2 declarations it needs (AC_CHECK_DECLS([PCRE2_...])). | ||
| # Build a newer pcre2 when the system one lacks any of them (rhel-8 has 10.32). | ||
| ensure_pcre2() { |
There was a problem hiding this comment.
I think any "guard" code should live in buildscripts/ci/setup-cfengine-build-host.sh and not here.
Here in core/ci/dependencies.sh we should just let things roll and fail if they will.
|
|
||
| # rhel-8 has pcre2 10.32 but libntech needs at least 10.34 (PCRE2_SUBSTITUTE_MATCHED, PCRE2_SUBSTITUTE_LITERAL). | ||
| # Build it into /opt/pcre2, away from the system pcre2, and ci/configure.sh will use it from there. | ||
| build_pcre2() { |
There was a problem hiding this comment.
We should just move this to buildscripts/ci/setup-cfengine-build-host.sh so we have the version there if we need to use it.
Ticket: ENT-14716
Merge together:
NorthernTechHQ/libntech#300
#6472
Depends on NorthernTechHQ/libntech#300, which makes libntech's
configure.accheck for the PCRE2 declarations it uses (AC_CHECK_DECLS). After it merges, this PR also needs alibntechsubmodule bump to pick it up.Follow-up to #6470. With RHEL 8/9 allowed in
ci/dependencies.sh, the golden-ami bake of the RHEL 8/9 arm64 build hosts got past the dependency install but then failed inci/install.sh:cf-keyfailed withliblmdb.so.1: cannot open shared object file.build_lmdbranmake install prefix=/usr, and lmdb's Makefile then puts the library in/usr/lib, which the runtime linker doesn't search on lib64 distributions. (librsync was fine; its cmake install already uses/usr/lib64.)buffer.cwith'PCRE2_SUBSTITUTE_MATCHED' undeclaredand'PCRE2_SUBSTITUTE_LITERAL' undeclared. These were added in pcre2 10.34, and RHEL 8 ships 10.32.RHEL 10 never hit either, because cf-remote finds a real el10 package there and the source build doesn't run.
Changes:
build_lmdbinstalls into/usr/lib64when that directory exists, and into/usr/libotherwise. Only RHEL and SUSE call it, and both are lib64.ensure_pcre2reads the list of pcre2 declarations libntech needs from theAC_CHECK_DECLS([PCRE2_...])line inlibntech/configure.ac(currentlyPCRE2_SUBSTITUTE_MATCHEDandPCRE2_SUBSTITUTE_LITERAL). It then checks the systempcre2.hfor them withgcc -E -dM. Only if one is missing doesbuild_pcre2build pcre2 into/opt/pcre2, leaving the system pcre2 alone. If the line is missing from libntech, the script fails instead of skipping the check.build_pcre2takes the pcre2 release from buildscripts'deps-packaging/pcre2/sourceanddistfiles(the same release the CFEngine packages bundle) and checks the download withsha256sum --check.ci/configure.shpasses--with-pcre2=/opt/pcre2when that directory exists, and core's configure adds an rpath for it.No pcre2 version is hard-coded in core: the required declarations come from libntech, and the version to build comes from buildscripts.
Testing: ran the full
ci/install.sh(dependencies, configure, build, install,cf-key) on core master with these scripts and libntech#300'sconfigure.acin a freshrockylinux/rockylinux:8container (amd64):Configuring libntech against the system pcre2 10.32 now stops with
configure: error: This release of CFEngine requires PCRE2 with PCRE2_SUBSTITUTE_MATCHED and PCRE2_SUBSTITUTE_LITERAL (10.34+).An earlier run of the lmdb fix in
rockylinux/rockylinux:9completedci/install.shwithliblmdb.so.1 => /lib64/liblmdb.so.1. On RHEL 9 the system pcre2 (10.40) has both declarations, so it's used as before (checkedensure_pcre2inrockylinux/rockylinux:9: no pcre2 build).The real arm64 RHEL 8/9 check is the next golden-ami bake, which clones core master.
🤖 Generated with Claude Code