[2026-01-26 12:19:31,907][ INFO][PID:3933741] Marking build as starting [2026-01-26 12:19:31,985][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:19:31,989][ INFO][PID:3933741] VM allocation process starts [2026-01-26 12:19:32,017][ INFO][PID:3933741] Trying to allocate VM: ResallocHost, ticket_id=1021755, requested_tags=['copr_builder', 'arch_noarch'] [2026-01-26 12:19:37,023][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:19:38,083][ INFO][PID:3933741] Allocated host ResallocHost, ticket_id=1021755, hostname=44.221.46.102, name=aws_x86_64_spot_prod_01570880_20260126_121412, requested_tags=['copr_builder', 'arch_noarch'] [2026-01-26 12:19:38,086][ INFO][PID:3933741] Allocating ssh connection to builder [2026-01-26 12:19:38,087][ INFO][PID:3933741] Checking that builder machine is OK [2026-01-26 12:19:38,451][ INFO][PID:3933741] Running remote command: copr-builder-ready srpm-builds [2026-01-26 12:19:38,746][ INFO][PID:3933741] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2026-01-26 12:19:38,747][ INFO][PID:3933741] Filling build.info file with builder info [2026-01-26 12:19:38,748][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:19:38,750][ INFO][PID:3933741] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "lbalhar", "project_name": "python-packaging_rawhide_control", "project_dirname": "python-packaging_rawhide_control", "submitter": "lbalhar", "ended_on": null, "started_on": 1769429978.7498279, "submitted_on": null, "status": 3, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "10062078", "build_id": 10062078, "package_name": "python-flask-babel", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 10, "source_json": "{\"clone_url\": \"https://src.fedoraproject.org/rpms/python-flask-babel.git\", \"distgit\": \"fedora\"}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "lbalhar/python-packaging_rawhide_control--lbalhar", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": true, "destdir": "/var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control", "result_dir": "10062078", "built_packages": "", "tags": [ "arch_noarch" ], "id": 10062078, "mockchain_macros": { "copr_username": "lbalhar", "copr_projectname": "python-packaging_rawhide_control", "vendor": "Fedora Project COPR (lbalhar/python-packaging_rawhide_control)" } } ] } [2026-01-26 12:19:38,865][ INFO][PID:3933741] Sending fedora-messaging bus message in build.start [2026-01-26 12:19:39,788][ INFO][PID:3933741] Sending fedora-messaging bus message in chroot.start [2026-01-26 12:19:39,838][ INFO][PID:3933741] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10062078 --detached [2026-01-26 12:19:40,210][ INFO][PID:3933741] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '44.221.46.102' (ED25519) to the list of known hosts. [2026-01-26 12:19:40,211][ INFO][PID:3933741] Downloading the builder-live.log file, attempt 1 [2026-01-26 12:19:40,213][ INFO][PID:3933741] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.221.46.102 copr-rpmbuild-log [2026-01-26 12:19:42,105][ ERROR][PID:3933741] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-26 12:19:45,220][ INFO][PID:3933741] Periodic builder liveness probe: alive [2026-01-26 12:19:45,220][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:19:50,223][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:19:52,105][WARNING][PID:3933741] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-26 12:19:55,229][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:00,230][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:02,106][ INFO][PID:3933741] Downloading the builder-live.log file, attempt 2 [2026-01-26 12:20:02,107][ INFO][PID:3933741] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.221.46.102 copr-rpmbuild-log [2026-01-26 12:20:05,231][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:10,232][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:15,233][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:20,234][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:25,235][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:30,236][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:35,242][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:40,250][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:45,287][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:50,288][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:20:55,290][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:00,301][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:02,176][ ERROR][PID:3933741] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-26 12:21:05,302][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:10,303][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:12,181][WARNING][PID:3933741] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-26 12:21:15,305][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:20,313][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:22,184][ INFO][PID:3933741] Downloading the builder-live.log file, attempt 3 [2026-01-26 12:21:22,188][ INFO][PID:3933741] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.221.46.102 copr-rpmbuild-log [2026-01-26 12:21:25,314][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:30,316][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:35,336][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:40,337][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:45,338][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:50,339][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:21:55,340][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:00,346][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:05,349][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:10,355][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:15,356][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:20,357][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:22,227][ ERROR][PID:3933741] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-26 12:22:25,358][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:30,360][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:32,238][WARNING][PID:3933741] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-26 12:22:35,361][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:40,362][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:42,239][ INFO][PID:3933741] Downloading the builder-live.log file, attempt 4 [2026-01-26 12:22:42,240][ INFO][PID:3933741] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.221.46.102 copr-rpmbuild-log [2026-01-26 12:22:45,363][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:50,364][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:22:55,365][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:00,366][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:05,368][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:10,369][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:15,370][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:20,371][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:25,372][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:30,373][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:35,374][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:40,379][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:42,308][ ERROR][PID:3933741] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-26 12:23:45,380][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:50,382][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:52,315][ ERROR][PID:3933741] Giving up for unstable SSH, failures: 1769429992.1058705, 1769430072.1810768, 1769430152.2381852, 1769430232.3150783 [2026-01-26 12:23:52,316][ ERROR][PID:3933741] Re-try request for task on 'ResallocHost, ticket_id=1021755, hostname=44.221.46.102, name=aws_x86_64_spot_prod_01570880_20260126_121412, requested_tags=['copr_builder', 'arch_noarch']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-01-26 12:23:52,316][ INFO][PID:3933741] Releasing VM back to pool [2026-01-26 12:23:52,331][ INFO][PID:3933741] Retry #1 (on other host) [2026-01-26 12:23:52,331][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:52,332][ INFO][PID:3933741] VM allocation process starts [2026-01-26 12:23:52,344][ INFO][PID:3933741] Trying to allocate VM: ResallocHost, ticket_id=1021917, requested_tags=['copr_builder', 'arch_noarch'] [2026-01-26 12:23:55,385][ INFO][PID:3933741] Allocated host ResallocHost, ticket_id=1021917, hostname=18.206.174.53, name=aws_aarch64_normalreserved_prod_01571367_20260126_122115, requested_tags=['copr_builder', 'arch_noarch'] [2026-01-26 12:23:55,387][ INFO][PID:3933741] Allocating ssh connection to builder [2026-01-26 12:23:55,388][ INFO][PID:3933741] Checking that builder machine is OK [2026-01-26 12:23:55,818][ INFO][PID:3933741] Running remote command: copr-builder-ready srpm-builds [2026-01-26 12:23:56,068][ INFO][PID:3933741] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2026-01-26 12:23:56,069][ INFO][PID:3933741] Filling build.info file with builder info [2026-01-26 12:23:56,095][ INFO][PID:3933741] Checking for cancel request [2026-01-26 12:23:56,096][ INFO][PID:3933741] Not re-notifying FE and msg buses for the new host. [2026-01-26 12:23:56,097][ INFO][PID:3933741] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10062078 --detached [2026-01-26 12:23:56,492][ INFO][PID:3933741] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '18.206.174.53' (ED25519) to the list of known hosts. [2026-01-26 12:23:56,493][ INFO][PID:3933741] Downloading the builder-live.log file, attempt 1 [2026-01-26 12:23:56,495][ INFO][PID:3933741] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.206.174.53 copr-rpmbuild-log [2026-01-26 12:23:59,429][ INFO][PID:3933741] Downloading results from builder [2026-01-26 12:23:59,432][ INFO][PID:3933741] rsyncing of mockbuilder@18.206.174.53:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078 started [2026-01-26 12:23:59,437][ INFO][PID:3933741] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@18.206.174.53:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/ &> /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/build-10062078.rsync.log [2026-01-26 12:23:59,872][ INFO][PID:3933741] rsyncing finished. [2026-01-26 12:23:59,873][ INFO][PID:3933741] Releasing VM back to pool [2026-01-26 12:23:59,885][ INFO][PID:3933741] Searching for 'success' file in resultdir [2026-01-26 12:23:59,887][ INFO][PID:3933741] Getting build details [2026-01-26 12:23:59,888][ INFO][PID:3933741] Retrieving SRPM info from /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078 [2026-01-26 12:23:59,890][ INFO][PID:3933741] SRPM URL: https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/python-flask-babel-4.1.0-2.src.rpm [2026-01-26 12:23:59,890][ INFO][PID:3933741] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/python-flask-babel-4.1.0-2.src.rpm', 'pkg_name': 'python-flask-babel', 'pkg_version': '4.1.0-2'} [2026-01-26 12:23:59,890][ INFO][PID:3933741] Finished build: id=10062078 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control chroot=srpm-builds [2026-01-26 12:23:59,892][ INFO][PID:3933741] Worker succeeded build, took 3.7956154346466064 [2026-01-26 12:23:59,893][ INFO][PID:3933741] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "lbalhar", "project_name": "python-packaging_rawhide_control", "project_dirname": "python-packaging_rawhide_control", "submitter": "lbalhar", "ended_on": 1769430239.8924303, "started_on": 1769430236.0968149, "submitted_on": null, "status": 1, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "10062078", "build_id": 10062078, "package_name": "python-flask-babel", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 10, "source_json": "{\"clone_url\": \"https://src.fedoraproject.org/rpms/python-flask-babel.git\", \"distgit\": \"fedora\"}", "pkg_name": "python-flask-babel", "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": "https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/python-flask-babel-4.1.0-2.src.rpm", "uses_devel_repo": null, "sandbox": "lbalhar/python-packaging_rawhide_control--lbalhar", "results": { "name": "python-flask-babel", "epoch": null, "version": "4.1.0", "release": "2", "exclusivearch": [], "excludearch": [] }, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": true, "destdir": "/var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control", "result_dir": "10062078", "built_packages": "", "tags": [ "arch_noarch" ], "pkg_version": "4.1.0-2", "id": 10062078, "mockchain_macros": { "copr_username": "lbalhar", "copr_projectname": "python-packaging_rawhide_control", "vendor": "Fedora Project COPR (lbalhar/python-packaging_rawhide_control)" } } ] } [2026-01-26 12:23:59,970][ INFO][PID:3933741] Sending fedora-messaging bus message in build.end [2026-01-26 12:24:00,001][ INFO][PID:3933741] Compressing /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/builder-live.log by gzip [2026-01-26 12:24:00,002][ INFO][PID:3933741] Running command 'gzip /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/builder-live.log' as PID 3971198 [2026-01-26 12:24:00,004][ INFO][PID:3933741] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/builder-live.log) [2026-01-26 12:24:00,004][ INFO][PID:3933741] Compressing /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/backend.log by gzip [2026-01-26 12:24:00,005][ INFO][PID:3933741] Running command 'gzip /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062078/backend.log' as PID 3971199