[2026-06-02 10:33:12,136][ INFO][PID:1035245] Marking build as starting [2026-06-02 10:33:12,181][ INFO][PID:1035245] Checking that https://download.copr.fedorainfracloud.org/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386//repodata/repomd.xml exists [2026-06-02 10:33:12,509][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:12,523][ INFO][PID:1035245] rpmeta: no HW info for arch i386, skipping [2026-06-02 10:33:12,524][ INFO][PID:1035245] VM allocation process starts [2026-06-02 10:33:12,543][ INFO][PID:1035245] Trying to allocate VM: ResallocHost, ticket_id=3180357, requested_tags=['arch_i386', 'copr_builder'] [2026-06-02 10:33:15,565][ INFO][PID:1035245] Allocated host ResallocHost, ticket_id=3180357, hostname=3.237.91.223, name=aws_x86_64_spot_prod_10449518_20260602_102429, requested_tags=['arch_i386', 'copr_builder'] [2026-06-02 10:33:15,575][ INFO][PID:1035245] Allocating ssh connection to builder [2026-06-02 10:33:15,576][ INFO][PID:1035245] Checking that builder machine is OK [2026-06-02 10:33:15,901][ INFO][PID:1035245] Installed copr-rpmbuild version: 1.8 [2026-06-02 10:33:15,902][ INFO][PID:1035245] Running remote command: copr-builder-ready fedora-rawhide-i386 [2026-06-02 10:33:16,131][ INFO][PID:1035245] Red Hat subscription not needed for fedora-rawhide-i386 Builder is ready to be used [2026-06-02 10:33:16,132][ INFO][PID:1035245] Filling build.info file with builder info [2026-06-02 10:33:16,133][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:16,135][ INFO][PID:1035245] Sending build state back to frontend: { "builds": [ { "timeout": 115200, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "jwakely", "project_name": "boost-190-system-deps.checker", "project_dirname": "boost-190-system-deps.checker", "submitter": "jwakely", "ended_on": null, "started_on": 1780396396.1349087, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-i386", "arch": "i386", "buildroot_pkgs": [], "task_id": "10536389-fedora-rawhide-i386", "build_id": 10536389, "package_name": "ledger", "package_version": "3.3.2-5", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/jwakely/boost-190-system-deps.checker/ledger", "git_hash": "c118fafdd3252d981a2d1e1c851799aa9ee2c666", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "jwakely/boost-190-system-deps.checker--jwakely", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_i386" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/jwakely/boost-190-system-deps.checker", "result_dir": "10536389-ledger", "built_packages": "", "rpmeta_prediction": null, "id": 10536389, "mockchain_macros": { "copr_username": "jwakely", "copr_projectname": "boost-190-system-deps.checker", "vendor": "Fedora Project COPR (jwakely/boost-190-system-deps.checker)" } } ] } [2026-06-02 10:33:16,185][ INFO][PID:1035245] Sending fedora-messaging bus message in build.start [2026-06-02 10:33:16,753][ INFO][PID:1035245] Sending fedora-messaging bus message in chroot.start [2026-06-02 10:33:16,784][ INFO][PID:1035245] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10536389-fedora-rawhide-i386 --chroot fedora-rawhide-i386 --detached [2026-06-02 10:33:17,225][ INFO][PID:1035245] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.237.91.223' (ED25519) to the list of known hosts. [2026-06-02 10:33:17,228][ INFO][PID:1035245] Downloading the builder-live.log file, attempt 1 [2026-06-02 10:33:17,231][ INFO][PID:1035245] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.237.91.223 copr-rpmbuild-log [2026-06-02 10:33:22,236][ INFO][PID:1035245] Periodic builder liveness probe: alive [2026-06-02 10:33:22,237][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:22,691][ ERROR][PID:1035245] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-06-02 10:33:27,238][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:32,266][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:32,702][WARNING][PID:1035245] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-06-02 10:33:37,269][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:42,302][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:42,704][ INFO][PID:1035245] Downloading the builder-live.log file, attempt 2 [2026-06-02 10:33:42,707][ INFO][PID:1035245] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.237.91.223 copr-rpmbuild-log [2026-06-02 10:33:47,305][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:52,345][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:33:57,356][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:02,369][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:07,371][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:12,376][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:17,377][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:22,380][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:27,381][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:32,383][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:37,385][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:42,389][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:42,777][ ERROR][PID:1035245] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-06-02 10:34:47,392][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:52,394][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:34:52,806][WARNING][PID:1035245] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-06-02 10:34:57,396][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:02,399][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:02,808][ INFO][PID:1035245] Downloading the builder-live.log file, attempt 3 [2026-06-02 10:35:02,810][ INFO][PID:1035245] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.237.91.223 copr-rpmbuild-log [2026-06-02 10:35:07,400][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:12,401][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:17,403][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:22,404][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:27,406][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:32,424][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:37,425][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:42,426][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:47,428][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:52,429][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:35:57,431][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:02,432][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:02,882][ ERROR][PID:1035245] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-06-02 10:36:07,434][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:12,435][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:12,920][WARNING][PID:1035245] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-06-02 10:36:17,436][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:22,438][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:22,921][ INFO][PID:1035245] Downloading the builder-live.log file, attempt 4 [2026-06-02 10:36:22,923][ INFO][PID:1035245] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.237.91.223 copr-rpmbuild-log [2026-06-02 10:36:27,439][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:32,441][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:37,444][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:42,475][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:47,477][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:52,478][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:36:57,480][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:02,482][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:07,483][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:12,485][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:17,520][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:22,521][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:22,990][ ERROR][PID:1035245] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-06-02 10:37:27,522][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:32,524][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:33,076][ ERROR][PID:1035245] Giving up for unstable SSH, failures: 1780396412.7020807, 1780396492.80692, 1780396572.9203227, 1780396653.0559256 [2026-06-02 10:37:33,119][ ERROR][PID:1035245] Re-try request for task on 'ResallocHost, ticket_id=3180357, hostname=3.237.91.223, name=aws_x86_64_spot_prod_10449518_20260602_102429, requested_tags=['arch_i386', 'copr_builder']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-06-02 10:37:33,153][ INFO][PID:1035245] VM Release request [2026-06-02 10:37:33,258][ INFO][PID:1035245] Retry #1 (on other host) [2026-06-02 10:37:33,265][ INFO][PID:1035245] Checking that https://download.copr.fedorainfracloud.org/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386//repodata/repomd.xml exists [2026-06-02 10:37:34,602][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:34,682][ INFO][PID:1035245] rpmeta: no HW info for arch i386, skipping [2026-06-02 10:37:34,682][ INFO][PID:1035245] VM allocation process starts [2026-06-02 10:37:34,798][ INFO][PID:1035245] Trying to allocate VM: ResallocHost, ticket_id=3180438, requested_tags=['arch_i386', 'copr_builder'] [2026-06-02 10:37:37,979][ INFO][PID:1035245] Allocated host ResallocHost, ticket_id=3180438, hostname=54.86.109.186, name=aws_x86_64_spot_prod_10449747_20260602_103544, requested_tags=['arch_i386', 'copr_builder'] [2026-06-02 10:37:37,995][ INFO][PID:1035245] Allocating ssh connection to builder [2026-06-02 10:37:37,998][ INFO][PID:1035245] Checking that builder machine is OK [2026-06-02 10:37:38,342][ INFO][PID:1035245] Installed copr-rpmbuild version: 1.8 [2026-06-02 10:37:38,358][ INFO][PID:1035245] Running remote command: copr-builder-ready fedora-rawhide-i386 [2026-06-02 10:37:38,615][ INFO][PID:1035245] Red Hat subscription not needed for fedora-rawhide-i386 Builder is ready to be used [2026-06-02 10:37:38,616][ INFO][PID:1035245] Filling build.info file with builder info [2026-06-02 10:37:38,780][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:38,788][ INFO][PID:1035245] Not re-notifying FE and msg buses for the new host. [2026-06-02 10:37:38,792][ INFO][PID:1035245] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10536389-fedora-rawhide-i386 --chroot fedora-rawhide-i386 --detached [2026-06-02 10:37:39,363][ INFO][PID:1035245] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '54.86.109.186' (ED25519) to the list of known hosts. [2026-06-02 10:37:39,364][ INFO][PID:1035245] Downloading the builder-live.log file, attempt 1 [2026-06-02 10:37:39,365][ INFO][PID:1035245] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.109.186 copr-rpmbuild-log [2026-06-02 10:37:44,364][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:49,366][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:54,367][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:37:59,368][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:04,370][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:09,380][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:14,381][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:19,383][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:24,385][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:29,390][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:34,398][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:39,400][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:44,402][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:49,406][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:54,408][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:38:59,409][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:39:04,426][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:39:09,436][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:39:14,439][ INFO][PID:1035245] Checking for cancel request [2026-06-02 10:39:16,243][ INFO][PID:1035245] Downloading results from builder [2026-06-02 10:39:16,348][ INFO][PID:1035245] rsyncing of mockbuilder@54.86.109.186:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/10536389-ledger started [2026-06-02 10:39:16,373][ INFO][PID:1035245] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.86.109.186:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/10536389-ledger/ &> /var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/10536389-ledger/build-10536389.rsync.log [2026-06-02 10:39:16,891][ INFO][PID:1035245] rsyncing finished. [2026-06-02 10:39:16,899][ INFO][PID:1035245] VM Release request [2026-06-02 10:39:16,947][ INFO][PID:1035245] Searching for 'success' file in resultdir [2026-06-02 10:39:16,951][ ERROR][PID:1035245] Build failed: Backend process error: No success file => build failure [2026-06-02 10:39:17,644][ INFO][PID:1035245] Finished build: id=10536389 failed=True timeout=115200 destdir=/var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker chroot=fedora-rawhide-i386 [2026-06-02 10:39:18,284][ ERROR][PID:1035245] Backend process error: No success file => build failure [2026-06-02 10:39:18,376][ INFO][PID:1035245] Worker failed build, took 99.59385347366333 [2026-06-02 10:39:18,396][ INFO][PID:1035245] Sending build state back to frontend: { "builds": [ { "timeout": 115200, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "jwakely", "project_name": "boost-190-system-deps.checker", "project_dirname": "boost-190-system-deps.checker", "submitter": "jwakely", "ended_on": 1780396758.3761215, "started_on": 1780396658.782268, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-i386", "arch": "i386", "buildroot_pkgs": [], "task_id": "10536389-fedora-rawhide-i386", "build_id": 10536389, "package_name": "ledger", "package_version": "3.3.2-5", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/jwakely/boost-190-system-deps.checker/ledger", "git_hash": "c118fafdd3252d981a2d1e1c851799aa9ee2c666", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "jwakely/boost-190-system-deps.checker--jwakely", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_i386" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/jwakely/boost-190-system-deps.checker", "result_dir": "10536389-ledger", "built_packages": "", "rpmeta_prediction": null, "id": 10536389, "mockchain_macros": { "copr_username": "jwakely", "copr_projectname": "boost-190-system-deps.checker", "vendor": "Fedora Project COPR (jwakely/boost-190-system-deps.checker)" } } ] } [2026-06-02 10:39:18,595][ INFO][PID:1035245] Sending fedora-messaging bus message in build.end [2026-06-02 10:39:21,545][ INFO][PID:1035245] Compressing /var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/10536389-ledger/builder-live.log by gzip [2026-06-02 10:39:21,564][ INFO][PID:1035245] Running command 'gzip /var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/10536389-ledger/builder-live.log' as PID 1074767 [2026-06-02 10:39:21,668][ INFO][PID:1035245] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/10536389-ledger/builder-live.log) [2026-06-02 10:39:21,701][ INFO][PID:1035245] Compressing /var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/10536389-ledger/backend.log by gzip [2026-06-02 10:39:21,705][ INFO][PID:1035245] Running command 'gzip /var/lib/copr/public_html/results/jwakely/boost-190-system-deps.checker/fedora-rawhide-i386/10536389-ledger/backend.log' as PID 1074787