Name: bigtop-hadoop Version: 3.4.0 Release: 1%{?dist} Summary: Build-service wrapper that builds Apache Bigtop Hadoop RPMs via Gradle License: Apache-2.0 URL: https://bigtop.apache.org %global debug_package %{nil} # --- Bigtop source ------------------------------------------------------------ %global bigtop_ref branch-3.4 %global bigtop_archive_name bigtop-%{bigtop_ref} Source0: https://github.com/apache/bigtop/archive/%{bigtop_ref}.tar.gz # --- Build arch --------------------------------------------------------------- BuildArch: noarch # --- Build requirements ------------------------------------------------------- # Core toolchain BuildRequires: java-1.8.0-openjdk-devel BuildRequires: java-11-openjdk-devel # Build tools BuildRequires: autoconf BuildRequires: automake BuildRequires: bash BuildRequires: cmake BuildRequires: curl BuildRequires: coreutils BuildRequires: diffutils BuildRequires: findutils BuildRequires: gawk BuildRequires: gcc BuildRequires: gcc-c++ BuildRequires: git BuildRequires: grep BuildRequires: gzip BuildRequires: hostname BuildRequires: libtool BuildRequires: make BuildRequires: nasm BuildRequires: patch BuildRequires: rsync BuildRequires: sed BuildRequires: tar BuildRequires: unzip BuildRequires: wget BuildRequires: which BuildRequires: yasm # Documentation tools BuildRequires: asciidoc BuildRequires: xmlto # Compression / archive libraries BuildRequires: bzip2-devel BuildRequires: libzstd-devel BuildRequires: lzo-devel BuildRequires: snappy-devel BuildRequires: zlib-devel BuildRequires: zstd # Crypto / auth BuildRequires: cyrus-sasl-devel BuildRequires: krb5-devel BuildRequires: openldap-devel BuildRequires: openssl-devel # Network / RPC BuildRequires: libcurl-devel BuildRequires: libtirpc-devel # FUSE BuildRequires: fuse BuildRequires: fuse-devel BuildRequires: fuse-libs BuildRequires: fuse3 BuildRequires: fuse3-devel BuildRequires: fuse3-libs # ISA-L (Intel Storage Acceleration) — required for Hadoop erasure coding # isal-devel provides the prebuilt headers/lib; if absent the build # bootstraps from source below, but prefer the system package when available. BuildRequires: isa-l-devel # Miscellaneous devel BuildRequires: boost-devel BuildRequires: bison BuildRequires: cppunit-devel BuildRequires: flex BuildRequires: gmp-devel BuildRequires: libffi-devel BuildRequires: libevent-devel BuildRequires: libuuid-devel BuildRequires: libxml2-devel BuildRequires: libxslt-devel BuildRequires: libyaml-devel BuildRequires: mariadb-connector-c-devel BuildRequires: ncurses-devel BuildRequires: readline-devel BuildRequires: sqlite-devel BuildRequires: xfsprogs-devel # Python BuildRequires: python3 BuildRequires: python3-devel BuildRequires: python3-flake8 BuildRequires: python3-setuptools BuildRequires: python3-wheel # Perl helpers BuildRequires: perl-Digest-SHA BuildRequires: perl-FindBin # RPM / repo tooling BuildRequires: createrepo BuildRequires: net-tools BuildRequires: redhat-rpm-config BuildRequires: rpm-build %description This package is a COPR build wrapper to run Apache Bigtop's Gradle packaging entry point for Hadoop (hadoop-pkg) and export generated RPMs. It emulates the Bigtop Puppet toolchain bootstrap without root or container access, as required by Fedora COPR's rootless build environment. The spec follows the parameter convention from the Ambari Bigtop Stack build guide: ./gradlew hadoop-pkg -PparentDir=/usr/bigtop -PpkgSuffix -PbuildThreads=2C # ============================================================================= %prep # ============================================================================= %setup -q -n %{bigtop_archive_name} # --------------------------------------------------------------------------- # Patch do-component-build: the "mvn site site:stage" invocation does not # honor EXTRA_GOALS (only the "mvn ... install" line does), but maven-site's # javadoc aggregation still walks into hadoop-yarn-applications-catalog-webapp # and re-triggers the frontend-maven-plugin / yarn install there, hitting the # same color@5.0.3 (Node>=18) incompatibility with the pinned Node 12.22.1. # Append ${EXTRA_GOALS} to the site call as well so our -pl exclusion applies # to both Maven invocations. # --------------------------------------------------------------------------- sed -i \ 's|^mvn site site:stage ${MAVEN_OPTS} \$@|mvn site site:stage ${MAVEN_OPTS} ${EXTRA_GOALS} $@|' \ bigtop-packages/src/common/hadoop/do-component-build # Sanity check: fail fast in %prep if the pattern wasn't found (e.g. upstream # do-component-build changed) rather than silently building an unpatched copy. grep -q 'mvn site site:stage \${MAVEN_OPTS} \${EXTRA_GOALS} \$@' \ bigtop-packages/src/common/hadoop/do-component-build # ============================================================================= %build # ============================================================================= # --------------------------------------------------------------------------- # Locale / umask # --------------------------------------------------------------------------- export LANG=C export LC_ALL=C umask 022 # --------------------------------------------------------------------------- # Writable home directories for tools that expect $HOME to be writable # (Gradle, Maven, Ivy all write caches here; COPR's build user is not root) # --------------------------------------------------------------------------- export HOME=%{_builddir}/home mkdir -p "$HOME" \ %{_builddir}/.gradle \ %{_builddir}/.m2 \ %{_builddir}/.ivy2 # --------------------------------------------------------------------------- # Java selection # # Hadoop's *native* (JNI/C) components must be compiled with JDK 8. # The JVM bytecode target is controlled separately by the Gradle build scripts. # We pick the JDK 8 path explicitly; the readlink dance is intentionally # avoided to keep the logic transparent and easy to debug in COPR logs. # --------------------------------------------------------------------------- if [ -d /usr/lib/jvm/java-1.8.0-openjdk ]; then export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk elif [ -d /usr/lib/jvm/java-8-openjdk ]; then export JAVA_HOME=/usr/lib/jvm/java-8-openjdk elif [ -d /usr/lib/jvm/java-1.8.0 ]; then export JAVA_HOME=/usr/lib/jvm/java-1.8.0 else echo "ERROR: JDK 8 not found — required for Hadoop native build" >&2 exit 1 fi export PATH="$JAVA_HOME/bin:$PATH" export BIGTOP_JDK=8 export BIGTOP_PREFERRED_JAVA_VERSION=8 # --------------------------------------------------------------------------- # Toolchain bootstrap # # Puppet manifests (gradle.pp, groovy.pp, maven.pp, ant.pp, isal.pp) normally # run as root and install under /usr/local. COPR forbids root, so we install # everything under $TOOLCHAIN_HOME inside the build directory instead and # adjust PATH / environment variables accordingly. # --------------------------------------------------------------------------- export TOOLCHAIN_HOME=%{_builddir}/toolchain export MAVEN_HOME="$TOOLCHAIN_HOME/maven" export ANT_HOME="$TOOLCHAIN_HOME/ant" export GROOVY_HOME="$TOOLCHAIN_HOME/groovy" export PROTOBUF_HOME="$TOOLCHAIN_HOME/protobuf" export ISAL_HOME="$TOOLCHAIN_HOME/isal" mkdir -p "$TOOLCHAIN_HOME" # Versions — keep in sync with bigtop_toolchain/manifests/*.pp # NOTE: Gradle version is NOT here — gradlew owns that; see gradle-wrapper.properties. MAVEN_VERSION=3.9.11 ANT_VERSION=1.9.16 GROOVY_VERSION=4.0.24 PROTOBUF_VERSION=3.7.1 # -- Maven (maven.pp) ---------------------------------------------------------- if [ ! -x "$MAVEN_HOME/bin/mvn" ]; then echo "--- Bootstrapping Maven ${MAVEN_VERSION} ---" curl -fL "https://archive.apache.org/dist/maven/maven-3/${MAVEN_VERSION}/binaries/apache-maven-${MAVEN_VERSION}-bin.tar.gz" \ -o "%{_builddir}/apache-maven-${MAVEN_VERSION}-bin.tar.gz" mkdir -p "$MAVEN_HOME" tar -xzf "%{_builddir}/apache-maven-${MAVEN_VERSION}-bin.tar.gz" \ -C "$MAVEN_HOME" --strip-components=1 fi # -- Ant (ant.pp) -------------------------------------------------------------- if [ ! -x "$ANT_HOME/bin/ant" ]; then echo "--- Bootstrapping Ant ${ANT_VERSION} ---" curl -fL "https://archive.apache.org/dist/ant/binaries/apache-ant-${ANT_VERSION}-bin.tar.gz" \ -o "%{_builddir}/apache-ant-${ANT_VERSION}-bin.tar.gz" mkdir -p "$ANT_HOME" tar -xzf "%{_builddir}/apache-ant-${ANT_VERSION}-bin.tar.gz" \ -C "$ANT_HOME" --strip-components=1 fi # -- Groovy (groovy.pp) -------------------------------------------------------- if [ ! -x "$GROOVY_HOME/bin/groovy" ]; then echo "--- Bootstrapping Groovy ${GROOVY_VERSION} ---" curl -fL "https://archive.apache.org/dist/groovy/${GROOVY_VERSION}/distribution/apache-groovy-binary-${GROOVY_VERSION}.zip" \ -o "%{_builddir}/apache-groovy-binary-${GROOVY_VERSION}.zip" mkdir -p "$GROOVY_HOME" unzip -q "%{_builddir}/apache-groovy-binary-${GROOVY_VERSION}.zip" \ -d "%{_builddir}/groovy-tmp" mv "%{_builddir}/groovy-tmp"/groovy-"${GROOVY_VERSION}"/* "$GROOVY_HOME/" rm -rf "%{_builddir}/groovy-tmp" fi # -- Protobuf (protobuf.pp) ---------------------------------------------------- if [ ! -x "$PROTOBUF_HOME/bin/protoc" ]; then echo "--- Bootstrapping Protobuf ${PROTOBUF_VERSION} ---" curl -fL "https://github.com/protocolbuffers/protobuf/archive/refs/tags/v${PROTOBUF_VERSION}.tar.gz" \ -o "%{_builddir}/protobuf-${PROTOBUF_VERSION}.tar.gz" mkdir -p "%{_builddir}/protobuf-src" tar -xzf "%{_builddir}/protobuf-${PROTOBUF_VERSION}.tar.gz" \ -C "%{_builddir}/protobuf-src" --strip-components=1 pushd "%{_builddir}/protobuf-src" ./autogen.sh ./configure --prefix="$PROTOBUF_HOME" --disable-shared --with-pic make -j"$(nproc)" make install popd fi # -- ISA-L (isal.pp) ----------------------------------------------------------- # Required for Hadoop native erasure-coding support. # Use the system package (isa-l-devel, BuildRequires above) when available; # fall back to a source build only if the header is absent. if pkg-config --exists libisal 2>/dev/null || [ -f /usr/include/isa-l.h ]; then echo "--- ISA-L: using system package ---" export ISAL_HOME=/usr else echo "--- Bootstrapping ISA-L from source ---" ISAL_VERSION=2.31.0 curl -fL "https://github.com/intel/isa-l/archive/refs/tags/v${ISAL_VERSION}.tar.gz" \ -o "%{_builddir}/isa-l-${ISAL_VERSION}.tar.gz" mkdir -p "%{_builddir}/isal-src" tar -xzf "%{_builddir}/isa-l-${ISAL_VERSION}.tar.gz" \ -C "%{_builddir}/isal-src" --strip-components=1 pushd "%{_builddir}/isal-src" ./autogen.sh ./configure --prefix="$ISAL_HOME" make -j"$(nproc)" make install popd export PKG_CONFIG_PATH="$ISAL_HOME/lib/pkgconfig:${PKG_CONFIG_PATH:-}" fi # --------------------------------------------------------------------------- # PATH — expose all bootstrapped tools; keep /usr/bin for system tools # --------------------------------------------------------------------------- export PATH="%{_builddir}/bin:$GROOVY_HOME/bin:$MAVEN_HOME/bin:$ANT_HOME/bin:$PROTOBUF_HOME/bin:$PATH" # --------------------------------------------------------------------------- # Fake sudo — Bigtop's Gradle configuration phase calls sudo in a few places # even for non-root tasks. Redirect it to /usr/bin/true so those calls # silently succeed without actually elevating privileges. # --------------------------------------------------------------------------- mkdir -p %{_builddir}/bin ln -sf /usr/bin/true %{_builddir}/bin/sudo # --------------------------------------------------------------------------- # Cache directories # --------------------------------------------------------------------------- export GRADLE_USER_HOME=%{_builddir}/.gradle export MAVEN_OPTS="-Dmaven.repo.local=%{_builddir}/.m2 -Xmx2g -XX:ReservedCodeCacheSize=512m" export IVY_HOME=%{_builddir}/.ivy2 # Suppress the Bigtop build timestamp to keep package names deterministic. export BIGTOP_BUILD_STAMP=1 # --------------------------------------------------------------------------- # Skip the YARN Application Catalog webapp during the Maven build. # # do-component-build passes EXTRA_GOALS verbatim to the mvn invocation. # The catalog webapp (hadoop-yarn-applications-catalog-webapp) fails because # its frontend-maven-plugin pins Node.js 12.22.1, but the transitive npm # dependency color@5.0.3 now requires Node>=18. The catalog webapp is an # optional UI component not required for core Hadoop or Ambari integration. # --------------------------------------------------------------------------- export EXTRA_GOALS="-pl !hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-catalog/hadoop-yarn-applications-catalog-webapp" # --------------------------------------------------------------------------- # Installation prefix # # -PparentDir matches the Ambari Bigtop Stack build guide convention. # /usr/bigtop does not exist in the COPR chroot, so we redirect to a writable # path under _builddir and copy the RPM outputs out afterwards. # --------------------------------------------------------------------------- export BIGTOP_OUTPUT=%{_builddir}/bigtop-output mkdir -p "$BIGTOP_OUTPUT" # --------------------------------------------------------------------------- # Sanity-check: print versions to the COPR log for easy debugging # --------------------------------------------------------------------------- echo "=== Build environment ===" java -version "$MAVEN_HOME/bin/mvn" --version "$ANT_HOME/bin/ant" -version "$GROOVY_HOME/bin/groovy" --version "$PROTOBUF_HOME/bin/protoc" --version # gradlew is the authoritative Gradle entry point; its version is set by # gradle/wrapper/gradle-wrapper.properties (currently 5.6.4). ./gradlew --version echo "=========================" # --------------------------------------------------------------------------- # Build # # Parameters follow the Ambari Bigtop Stack documentation: # -PparentDir installation prefix (Ambari convention: /usr/bigtop) # -PpkgSuffix appends the Bigtop version to package names # (e.g. hadoop_3_3_0) to satisfy Ambari service specs; # the flag is intentionally passed without a value — Bigtop # derives the suffix from its own version metadata # -PbuildThreads 2C = 2 × CPU cores # # allclean is omitted: COPR always starts from a clean chroot. # --------------------------------------------------------------------------- ./gradlew --no-daemon --stacktrace \ -Dbuildwithdeps=true \ -PparentDir=/usr/bigtop \ -PpkgSuffix \ -PbuildThreads=2C \ hadoop-pkg \ hbase-pkg \ zookeeper-pkg # --------------------------------------------------------------------------- # Export RPMs for COPR pickup # Bigtop writes output under output//; search broadly for safety. # --------------------------------------------------------------------------- find . -type f -name "*.rpm" -path "*/output/*" \ -exec cp -v -- {} %{_rpmdir}/ \; # ============================================================================= %changelog # ============================================================================= * Fri Jun 12 2026 Detlef Eppers - 3.4.0-5 - Patch do-component-build in %prep: append ${EXTRA_GOALS} to the "mvn site site:stage" call too. javadoc aggregation in that step still walks into hadoop-yarn-applications-catalog-webapp and re-triggers the same Node 12 / color@5.0.3 yarn install failure that -pl already fixed for the main "mvn ... install" invocation. * Fri Jun 12 2026 Detlef Eppers - 3.4.0-4 - Fix EXTRA_GOALS: drop literal single quotes around -pl argument. do-component-build expands EXTRA_GOALS unquoted as a plain bash string, so quote characters were passed to mvn literally and broke project lookup. * Fri Jun 12 2026 Detlef Eppers - 3.4.0-3 - Skip catalog webapp via EXTRA_GOALS: color@5.0.3 requires Node>=18 but frontend-maven-plugin pins Node 12.22.1; optional UI not needed for Ambari - Fix changelog day-of-week (Jun 12 2026 is Friday, not Thursday) * Fri Jun 12 2026 Detlef Eppers - 3.4.0-2 - Revert Source1/file:// Gradle pre-seed; all toolchain downloads are uniform at build time — consistent with Maven, Ant, Groovy, Protobuf, ISA-L * Fri Jun 12 2026 Detlef Eppers - 3.4.0-1 - Incorporate rootless toolchain bootstrap for COPR (no Puppet/sudo/root) - Add Groovy bootstrap (groovy.pp equivalent) required by Hadoop build scripts - Add ISA-L bootstrap (isal.pp equivalent) for Hadoop erasure-coding native libs - Switch JAVA_HOME to JDK 8 (required for Hadoop native/JNI compilation) - Remove BIGTOP_JDK=11 / BIGTOP_PREFERRED_JAVA_VERSION=11 (wrong for Hadoop) - Redirect -PparentDir to writable _builddir path; /usr/bigtop absent in chroot - Drop allclean task (redundant in COPR clean chroot; saves one Gradle config run) - Document -PpkgSuffix and -PbuildThreads per Ambari Bigtop Stack build guide - Add fake-sudo shim earlier in PATH to silence Gradle config-phase sudo calls