[2026-01-26 12:19:31,547][ INFO][PID:3933673] Marking build as starting [2026-01-26 12:19:31,648][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:19:31,653][ INFO][PID:3933673] VM allocation process starts [2026-01-26 12:19:31,680][ INFO][PID:3933673] Trying to allocate VM: ResallocHost, ticket_id=1021751, requested_tags=['copr_builder', 'arch_noarch'] [2026-01-26 12:19:36,684][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:19:37,726][ INFO][PID:3933673] Allocated host ResallocHost, ticket_id=1021751, hostname=98.92.166.57, name=aws_x86_64_spot_prod_01570878_20260126_121412, requested_tags=['copr_builder', 'arch_noarch'] [2026-01-26 12:19:37,728][ INFO][PID:3933673] Allocating ssh connection to builder [2026-01-26 12:19:37,729][ INFO][PID:3933673] Checking that builder machine is OK [2026-01-26 12:19:38,088][ INFO][PID:3933673] Running remote command: copr-builder-ready srpm-builds [2026-01-26 12:19:38,358][ INFO][PID:3933673] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2026-01-26 12:19:38,381][ INFO][PID:3933673] Filling build.info file with builder info [2026-01-26 12:19:38,383][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:19:38,389][ INFO][PID:3933673] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "lbalhar", "project_name": "python-packaging_rawhide_control", "project_dirname": "python-packaging_rawhide_control", "submitter": "lbalhar", "ended_on": null, "started_on": 1769429978.3893483, "submitted_on": null, "status": 3, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "10062074", "build_id": 10062074, "package_name": "python-hdfs", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 10, "source_json": "{\"clone_url\": \"https://src.fedoraproject.org/rpms/python-hdfs.git\", \"distgit\": \"fedora\"}", "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": null, "sandbox": "lbalhar/python-packaging_rawhide_control--lbalhar", "results": null, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": true, "destdir": "/var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control", "result_dir": "10062074", "built_packages": "", "tags": [ "arch_noarch" ], "id": 10062074, "mockchain_macros": { "copr_username": "lbalhar", "copr_projectname": "python-packaging_rawhide_control", "vendor": "Fedora Project COPR (lbalhar/python-packaging_rawhide_control)" } } ] } [2026-01-26 12:19:38,498][ INFO][PID:3933673] Sending fedora-messaging bus message in build.start [2026-01-26 12:19:40,020][ INFO][PID:3933673] Sending fedora-messaging bus message in chroot.start [2026-01-26 12:19:40,053][ INFO][PID:3933673] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10062074 --detached [2026-01-26 12:19:40,394][ INFO][PID:3933673] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '98.92.166.57' (ED25519) to the list of known hosts. [2026-01-26 12:19:40,395][ INFO][PID:3933673] Downloading the builder-live.log file, attempt 1 [2026-01-26 12:19:40,397][ INFO][PID:3933673] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.166.57 copr-rpmbuild-log [2026-01-26 12:19:42,070][ ERROR][PID:3933673] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-26 12:19:45,402][ INFO][PID:3933673] Periodic builder liveness probe: alive [2026-01-26 12:19:45,403][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:19:50,405][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:19:52,071][WARNING][PID:3933673] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-26 12:19:55,410][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:00,412][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:02,072][ INFO][PID:3933673] Downloading the builder-live.log file, attempt 2 [2026-01-26 12:20:02,073][ INFO][PID:3933673] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.166.57 copr-rpmbuild-log [2026-01-26 12:20:05,413][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:10,414][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:15,415][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:20,416][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:25,417][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:30,418][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:35,422][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:40,426][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:45,536][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:50,537][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:20:55,538][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:00,555][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:02,172][ ERROR][PID:3933673] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-26 12:21:05,563][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:10,564][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:12,181][WARNING][PID:3933673] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-26 12:21:15,566][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:20,567][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:22,183][ INFO][PID:3933673] Downloading the builder-live.log file, attempt 3 [2026-01-26 12:21:22,187][ INFO][PID:3933673] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.166.57 copr-rpmbuild-log [2026-01-26 12:21:25,571][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:30,572][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:35,577][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:40,578][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:45,580][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:50,580][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:21:55,582][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:00,584][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:05,596][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:10,599][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:15,600][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:20,601][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:22,195][ ERROR][PID:3933673] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-26 12:22:25,603][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:30,604][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:32,210][WARNING][PID:3933673] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-26 12:22:35,605][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:40,606][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:42,211][ INFO][PID:3933673] Downloading the builder-live.log file, attempt 4 [2026-01-26 12:22:42,212][ INFO][PID:3933673] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.166.57 copr-rpmbuild-log [2026-01-26 12:22:45,607][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:50,608][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:22:55,609][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:00,610][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:05,611][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:10,614][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:15,615][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:20,617][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:25,618][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:30,619][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:35,620][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:40,621][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:42,281][ ERROR][PID:3933673] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-26 12:23:45,622][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:50,623][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:52,296][ ERROR][PID:3933673] Giving up for unstable SSH, failures: 1769429992.0715826, 1769430072.181064, 1769430152.2102978, 1769430232.296786 [2026-01-26 12:23:52,297][ ERROR][PID:3933673] Re-try request for task on 'ResallocHost, ticket_id=1021751, hostname=98.92.166.57, name=aws_x86_64_spot_prod_01570878_20260126_121412, requested_tags=['copr_builder', 'arch_noarch']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-01-26 12:23:52,298][ INFO][PID:3933673] Releasing VM back to pool [2026-01-26 12:23:52,308][ INFO][PID:3933673] Retry #1 (on other host) [2026-01-26 12:23:52,309][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:52,310][ INFO][PID:3933673] VM allocation process starts [2026-01-26 12:23:52,323][ INFO][PID:3933673] Trying to allocate VM: ResallocHost, ticket_id=1021916, requested_tags=['copr_builder', 'arch_noarch'] [2026-01-26 12:23:55,374][ INFO][PID:3933673] Allocated host ResallocHost, ticket_id=1021916, hostname=3.235.171.124, name=aws_x86_64_normalreserved_prod_01571082_20260126_121658, requested_tags=['copr_builder', 'arch_noarch'] [2026-01-26 12:23:55,375][ INFO][PID:3933673] Allocating ssh connection to builder [2026-01-26 12:23:55,375][ INFO][PID:3933673] Checking that builder machine is OK [2026-01-26 12:23:55,739][ INFO][PID:3933673] Running remote command: copr-builder-ready srpm-builds [2026-01-26 12:23:55,964][ INFO][PID:3933673] Red Hat subscription not needed for srpm-builds Builder is ready to be used [2026-01-26 12:23:55,965][ INFO][PID:3933673] Filling build.info file with builder info [2026-01-26 12:23:55,992][ INFO][PID:3933673] Checking for cancel request [2026-01-26 12:23:55,993][ INFO][PID:3933673] Not re-notifying FE and msg buses for the new host. [2026-01-26 12:23:55,993][ INFO][PID:3933673] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10062074 --detached [2026-01-26 12:23:56,315][ INFO][PID:3933673] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.235.171.124' (ED25519) to the list of known hosts. [2026-01-26 12:23:56,316][ INFO][PID:3933673] Downloading the builder-live.log file, attempt 1 [2026-01-26 12:23:56,317][ INFO][PID:3933673] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.235.171.124 copr-rpmbuild-log [2026-01-26 12:23:58,525][ INFO][PID:3933673] Downloading results from builder [2026-01-26 12:23:58,525][ INFO][PID:3933673] rsyncing of mockbuilder@3.235.171.124:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074 started [2026-01-26 12:23:58,526][ INFO][PID:3933673] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@3.235.171.124:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/ &> /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/build-10062074.rsync.log [2026-01-26 12:23:58,762][ INFO][PID:3933673] rsyncing finished. [2026-01-26 12:23:58,763][ INFO][PID:3933673] Releasing VM back to pool [2026-01-26 12:23:58,773][ INFO][PID:3933673] Searching for 'success' file in resultdir [2026-01-26 12:23:58,774][ INFO][PID:3933673] Getting build details [2026-01-26 12:23:58,774][ INFO][PID:3933673] Retrieving SRPM info from /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074 [2026-01-26 12:23:58,775][ INFO][PID:3933673] SRPM URL: https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/python-hdfs-2.7.3-13.src.rpm [2026-01-26 12:23:58,775][ INFO][PID:3933673] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/python-hdfs-2.7.3-13.src.rpm', 'pkg_name': 'python-hdfs', 'pkg_version': '2.7.3-13'} [2026-01-26 12:23:58,775][ INFO][PID:3933673] Finished build: id=10062074 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control chroot=srpm-builds [2026-01-26 12:23:58,777][ INFO][PID:3933673] Worker succeeded build, took 2.7844078540802 [2026-01-26 12:23:58,778][ INFO][PID:3933673] Sending build state back to frontend: { "builds": [ { "timeout": 108000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": null, "enable_net": true, "project_owner": "lbalhar", "project_name": "python-packaging_rawhide_control", "project_dirname": "python-packaging_rawhide_control", "submitter": "lbalhar", "ended_on": 1769430238.777568, "started_on": 1769430235.9931602, "submitted_on": null, "status": 1, "chroot": "srpm-builds", "arch": null, "buildroot_pkgs": null, "task_id": "10062074", "build_id": 10062074, "package_name": "python-hdfs", "package_version": null, "git_repo": null, "git_hash": null, "git_branch": null, "source_type": 10, "source_json": "{\"clone_url\": \"https://src.fedoraproject.org/rpms/python-hdfs.git\", \"distgit\": \"fedora\"}", "pkg_name": "python-hdfs", "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": "https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/python-hdfs-2.7.3-13.src.rpm", "uses_devel_repo": null, "sandbox": "lbalhar/python-packaging_rawhide_control--lbalhar", "results": { "name": "python-hdfs", "epoch": null, "version": "2.7.3", "release": "13", "exclusivearch": [], "excludearch": [ "i386", "i486", "i586", "i686", "pentium3", "pentium4", "athlon", "geode" ] }, "appstream": false, "allow_user_ssh": null, "ssh_public_keys": null, "storage": null, "repos": [], "background": true, "destdir": "/var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/lbalhar/python-packaging_rawhide_control", "result_dir": "10062074", "built_packages": "", "tags": [ "arch_noarch" ], "pkg_version": "2.7.3-13", "id": 10062074, "mockchain_macros": { "copr_username": "lbalhar", "copr_projectname": "python-packaging_rawhide_control", "vendor": "Fedora Project COPR (lbalhar/python-packaging_rawhide_control)" } } ] } [2026-01-26 12:23:58,821][ INFO][PID:3933673] Sending fedora-messaging bus message in build.end [2026-01-26 12:23:58,851][ INFO][PID:3933673] Compressing /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/builder-live.log by gzip [2026-01-26 12:23:58,853][ INFO][PID:3933673] Running command 'gzip /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/builder-live.log' as PID 3971004 [2026-01-26 12:23:58,854][ INFO][PID:3933673] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/builder-live.log) [2026-01-26 12:23:58,855][ INFO][PID:3933673] Compressing /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/backend.log by gzip [2026-01-26 12:23:58,856][ INFO][PID:3933673] Running command 'gzip /var/lib/copr/public_html/results/lbalhar/python-packaging_rawhide_control/srpm-builds/10062074/backend.log' as PID 3971005