[2026-02-12 16:31:17,079][ INFO][PID:1256856] Marking build as starting [2026-02-12 16:31:17,160][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:31:17,169][ INFO][PID:1256856] VM allocation process starts [2026-02-12 16:31:17,232][ INFO][PID:1256856] Trying to allocate VM: ResallocHost, ticket_id=1301328, requested_tags=['copr_builder', 'arch_x86_64'] [2026-02-12 16:31:20,271][ INFO][PID:1256856] Allocated host ResallocHost, ticket_id=1301328, hostname=98.92.98.0, name=aws_x86_64_spot_prod_02224728_20260212_162832, requested_tags=['copr_builder', 'arch_x86_64'] [2026-02-12 16:31:20,272][ INFO][PID:1256856] Allocating ssh connection to builder [2026-02-12 16:31:20,272][ INFO][PID:1256856] Checking that builder machine is OK [2026-02-12 16:31:20,630][ INFO][PID:1256856] Running remote command: copr-builder-ready srpm-builds [2026-02-12 16:31:20,895][ INFO][PID:1256856] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2026-02-12 16:31:20,900][ INFO][PID:1256856] Filling build.info file with builder info [2026-02-12 16:31:20,903][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:31:20,908][ INFO][PID:1256856] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "lucamagrone", "project_name": "el10-pi", "project_dirname": "el10-pi", "submitter": null, "ended_on": null, "started_on": 1770913880.9084241, "submitted_on": null, "status": 3, "chroot": "srpm-builds", "arch": "x86_64", "buildroot_pkgs": null, "task_id": "10125356", "build_id": 10125356, "package_name": "kernel", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 9, "source_json": "{\"script\": \"#! /bin/bash\\n\\nexport PROJ=61887861\\n\\nexport URL=$(curl -Ls -o /dev/null -w '%{url_effective}' https://gitlab.com/Leuca/centos-stream-10/-/releases/permalink/latest)\\n\\necho $URL\\n\\nexport REL=$(echo $URL | cut -d\\\"/\\\" -f 8)\\nexport VREL=$(echo $URL | cut -d\\\"-\\\" -f 6 | cut -d\\\".\\\" -f 1)\\nexport RCOUNT=$(echo $URL | cut -d'.' -f 5)\\nexport VER=\\\"$VREL.$RCOUNT\\\"\\n\\ncurl -O https://gitlab.com/api/v4/projects/$PROJ/packages/generic/kernel/$VER/$REL.src.rpm\\ncurl -O https://gitlab.com/api/v4/projects/$PROJ/packages/generic/kernel/$VER/$REL.src.rpm.sha512\\n\\nshasum -a 512 -c $REL.src.rpm.sha512\\n\\nrpm2cpio $REL.src.rpm | cpio -idmv\\n\", \"chroot\": \"fedora-44-x86_64\", \"builddeps\": \"curl cut rpm cpio perl-Digest-SHA\", \"resultdir\": \"\", \"repos\": \"\"}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "lucamagrone/el10-pi--07aa7e60-5d42-4b35-87aa-a1006e128abb", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": true, "destdir": "/var/lib/copr/public_html/results/lucamagrone/el10-pi", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/lucamagrone/el10-pi", "result_dir": "10125356", "built_packages": "", "tags": [ "arch_x86_64" ], "id": 10125356, "mockchain_macros": { "copr_username": "lucamagrone", "copr_projectname": "el10-pi", "vendor": "Fedora Project COPR (lucamagrone/el10-pi)" } } ] } [2026-02-12 16:31:20,973][ INFO][PID:1256856] Sending fedora-messaging bus message in build.start [2026-02-12 16:31:22,014][ INFO][PID:1256856] Sending fedora-messaging bus message in chroot.start [2026-02-12 16:31:22,055][ INFO][PID:1256856] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10125356 --detached [2026-02-12 16:31:22,433][ INFO][PID:1256856] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '98.92.98.0' (ED25519) to the list of known hosts. [2026-02-12 16:31:22,436][ INFO][PID:1256856] Downloading the builder-live.log file, attempt 1 [2026-02-12 16:31:22,438][ INFO][PID:1256856] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.98.0 copr-rpmbuild-log [2026-02-12 16:31:27,448][ INFO][PID:1256856] Periodic builder liveness probe: alive [2026-02-12 16:31:27,450][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:31:32,451][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:31:37,216][ ERROR][PID:1256856] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-02-12 16:31:37,454][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:31:42,458][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:31:47,222][WARNING][PID:1256856] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-02-12 16:31:47,461][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:31:52,463][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:31:57,225][ INFO][PID:1256856] Downloading the builder-live.log file, attempt 2 [2026-02-12 16:31:57,230][ INFO][PID:1256856] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.98.0 copr-rpmbuild-log [2026-02-12 16:31:57,465][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:02,469][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:07,471][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:12,475][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:17,483][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:22,501][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:27,503][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:32,507][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:37,512][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:42,514][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:47,518][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:52,690][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:32:57,300][ ERROR][PID:1256856] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-02-12 16:32:57,705][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:02,711][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:07,386][WARNING][PID:1256856] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-02-12 16:33:07,744][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:12,750][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:17,387][ INFO][PID:1256856] Downloading the builder-live.log file, attempt 3 [2026-02-12 16:33:17,390][ INFO][PID:1256856] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.98.0 copr-rpmbuild-log [2026-02-12 16:33:17,753][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:22,759][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:27,763][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:32,765][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:37,767][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:42,806][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:47,827][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:52,833][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:33:57,835][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:02,836][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:07,838][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:12,840][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:17,459][ ERROR][PID:1256856] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-02-12 16:34:17,843][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:22,847][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:27,459][WARNING][PID:1256856] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-02-12 16:34:27,862][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:32,875][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:37,460][ INFO][PID:1256856] Downloading the builder-live.log file, attempt 4 [2026-02-12 16:34:37,465][ INFO][PID:1256856] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.98.0 copr-rpmbuild-log [2026-02-12 16:34:37,877][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:42,887][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:47,900][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:52,907][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:34:57,916][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:02,922][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:07,930][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:12,941][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:17,948][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:22,951][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:27,955][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:32,957][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:37,544][ ERROR][PID:1256856] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-02-12 16:35:37,958][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:43,068][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:47,672][ ERROR][PID:1256856] Giving up for unstable SSH, failures: 1770913907.2226734, 1770913987.3862932, 1770914067.45991, 1770914147.6725552 [2026-02-12 16:35:47,675][ ERROR][PID:1256856] Re-try request for task on 'ResallocHost, ticket_id=1301328, hostname=98.92.98.0, name=aws_x86_64_spot_prod_02224728_20260212_162832, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-02-12 16:35:47,679][ INFO][PID:1256856] Releasing VM back to pool [2026-02-12 16:35:47,738][ INFO][PID:1256856] Retry #1 (on other host) [2026-02-12 16:35:47,741][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:47,750][ INFO][PID:1256856] VM allocation process starts [2026-02-12 16:35:47,805][ INFO][PID:1256856] Trying to allocate VM: ResallocHost, ticket_id=1301566, requested_tags=['copr_builder', 'arch_x86_64'] [2026-02-12 16:35:52,817][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:35:57,821][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:02,826][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:07,830][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:12,835][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:17,837][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:22,882][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:27,958][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:32,966][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:35,994][ INFO][PID:1256856] Allocated host ResallocHost, ticket_id=1301566, hostname=100.31.122.39, name=aws_x86_64_normalreserved_prod_02225156_20260212_163457, requested_tags=['copr_builder', 'arch_x86_64'] [2026-02-12 16:36:35,995][ INFO][PID:1256856] Allocating ssh connection to builder [2026-02-12 16:36:35,995][ INFO][PID:1256856] Checking that builder machine is OK [2026-02-12 16:36:36,396][ INFO][PID:1256856] Running remote command: copr-builder-ready srpm-builds [2026-02-12 16:36:36,713][ INFO][PID:1256856] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2026-02-12 16:36:36,715][ INFO][PID:1256856] Filling build.info file with builder info [2026-02-12 16:36:36,749][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:36,751][ INFO][PID:1256856] Not re-notifying FE and msg buses for the new host. [2026-02-12 16:36:36,752][ INFO][PID:1256856] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10125356 --detached [2026-02-12 16:36:37,171][ INFO][PID:1256856] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '100.31.122.39' (ED25519) to the list of known hosts. [2026-02-12 16:36:37,177][ INFO][PID:1256856] Downloading the builder-live.log file, attempt 1 [2026-02-12 16:36:37,182][ INFO][PID:1256856] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.31.122.39 copr-rpmbuild-log [2026-02-12 16:36:42,181][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:47,183][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:52,187][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:36:57,189][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:37:02,193][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:37:07,194][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:37:12,198][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:37:17,202][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:37:22,204][ INFO][PID:1256856] Checking for cancel request [2026-02-12 16:37:27,079][ INFO][PID:1256856] Downloading results from builder [2026-02-12 16:37:27,080][ INFO][PID:1256856] rsyncing of mockbuilder@100.31.122.39:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356 started [2026-02-12 16:37:27,086][ INFO][PID:1256856] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@100.31.122.39:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356/ &> /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356/build-10125356.rsync.log [2026-02-12 16:37:28,633][ INFO][PID:1256856] rsyncing finished. [2026-02-12 16:37:28,634][ INFO][PID:1256856] Releasing VM back to pool [2026-02-12 16:37:28,646][ INFO][PID:1256856] Searching for 'success' file in resultdir [2026-02-12 16:37:28,647][ INFO][PID:1256856] Getting build details [2026-02-12 16:37:28,647][ INFO][PID:1256856] Retrieving SRPM info from /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356 [2026-02-12 16:37:28,648][ INFO][PID:1256856] SRPM URL: https://download.copr.fedorainfracloud.org/results/lucamagrone/el10-pi/srpm-builds/10125356/kernel-6.12.0-204.rpi64.1.src.rpm [2026-02-12 16:37:28,648][ INFO][PID:1256856] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/lucamagrone/el10-pi/srpm-builds/10125356/kernel-6.12.0-204.rpi64.1.src.rpm', 'pkg_name': 'kernel', 'pkg_version': '6.12.0-204.rpi64.1'} [2026-02-12 16:37:28,649][ INFO][PID:1256856] Finished build: id=10125356 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/lucamagrone/el10-pi chroot=srpm-builds [2026-02-12 16:37:28,650][ INFO][PID:1256856] Worker succeeded build, took 51.89955186843872 [2026-02-12 16:37:28,651][ INFO][PID:1256856] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "lucamagrone", "project_name": "el10-pi", "project_dirname": "el10-pi", "submitter": null, "ended_on": 1770914248.650832, "started_on": 1770914196.75128, "submitted_on": null, "status": 1, "chroot": "srpm-builds", "arch": "x86_64", "buildroot_pkgs": null, "task_id": "10125356", "build_id": 10125356, "package_name": "kernel", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 9, "source_json": "{\"script\": \"#! /bin/bash\\n\\nexport PROJ=61887861\\n\\nexport URL=$(curl -Ls -o /dev/null -w '%{url_effective}' https://gitlab.com/Leuca/centos-stream-10/-/releases/permalink/latest)\\n\\necho $URL\\n\\nexport REL=$(echo $URL | cut -d\\\"/\\\" -f 8)\\nexport VREL=$(echo $URL | cut -d\\\"-\\\" -f 6 | cut -d\\\".\\\" -f 1)\\nexport RCOUNT=$(echo $URL | cut -d'.' -f 5)\\nexport VER=\\\"$VREL.$RCOUNT\\\"\\n\\ncurl -O https://gitlab.com/api/v4/projects/$PROJ/packages/generic/kernel/$VER/$REL.src.rpm\\ncurl -O https://gitlab.com/api/v4/projects/$PROJ/packages/generic/kernel/$VER/$REL.src.rpm.sha512\\n\\nshasum -a 512 -c $REL.src.rpm.sha512\\n\\nrpm2cpio $REL.src.rpm | cpio -idmv\\n\", \"chroot\": \"fedora-44-x86_64\", \"builddeps\": \"curl cut rpm cpio perl-Digest-SHA\", \"resultdir\": \"\", \"repos\": \"\"}", "pkg_name": "kernel", "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": "https://download.copr.fedorainfracloud.org/results/lucamagrone/el10-pi/srpm-builds/10125356/kernel-6.12.0-204.rpi64.1.src.rpm", "uses_devel_repo": null, "sandbox": "lucamagrone/el10-pi--07aa7e60-5d42-4b35-87aa-a1006e128abb", "results": { "name": "kernel", "epoch": null, "version": "6.12.0", "release": "204.rpi64.1", "exclusivearch": [ "noarch", "x86_64", "s390x", "aarch64", "ppc64le", "riscv64" ], "excludearch": [] }, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": true, "destdir": "/var/lib/copr/public_html/results/lucamagrone/el10-pi", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/lucamagrone/el10-pi", "result_dir": "10125356", "built_packages": "", "tags": [ "arch_x86_64" ], "pkg_version": "6.12.0-204.rpi64.1", "id": 10125356, "mockchain_macros": { "copr_username": "lucamagrone", "copr_projectname": "el10-pi", "vendor": "Fedora Project COPR (lucamagrone/el10-pi)" } } ] } [2026-02-12 16:37:28,716][ INFO][PID:1256856] Sending fedora-messaging bus message in build.end [2026-02-12 16:37:28,745][ INFO][PID:1256856] Compressing /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356/builder-live.log by gzip [2026-02-12 16:37:28,747][ INFO][PID:1256856] Running command 'gzip /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356/builder-live.log' as PID 1327396 [2026-02-12 16:37:28,763][ INFO][PID:1256856] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356/builder-live.log) [2026-02-12 16:37:28,763][ INFO][PID:1256856] Compressing /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356/backend.log by gzip [2026-02-12 16:37:28,764][ INFO][PID:1256856] Running command 'gzip /var/lib/copr/public_html/results/lucamagrone/el10-pi/srpm-builds/10125356/backend.log' as PID 1327398