[2025-12-12 15:57:05,663][ INFO][PID:1532150] Marking build as starting [2025-12-12 15:57:05,752][ INFO][PID:1532150] Checking that /var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/repodata/repomd.xml exists [2025-12-12 15:57:05,753][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:05,754][ INFO][PID:1532150] VM allocation process starts [2025-12-12 15:57:05,769][ INFO][PID:1532150] Trying to allocate VM: ResallocHost, ticket_id=307241, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-12 15:57:10,771][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:15,772][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:20,773][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:25,782][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:30,790][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:35,791][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:40,793][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:45,794][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:50,795][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:57:55,797][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:00,798][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:05,799][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:10,800][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:15,801][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:20,803][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:25,804][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:30,809][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:35,812][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:40,814][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:45,815][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:50,817][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:58:55,823][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:00,969][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:05,970][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:10,973][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:15,975][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:20,976][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:25,977][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:30,978][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:35,984][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:40,986][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:45,987][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:50,988][ INFO][PID:1532150] Checking for cancel request [2025-12-12 15:59:55,990][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:00,991][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:05,992][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:10,993][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:15,994][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:20,995][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:25,999][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:31,000][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:36,001][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:41,003][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:46,004][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:51,005][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:00:56,007][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:01,009][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:06,010][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:11,011][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:16,013][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:21,014][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:26,016][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:31,019][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:36,020][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:41,021][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:46,022][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:51,024][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:01:56,026][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:01,027][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:06,028][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:11,029][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:16,030][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:21,032][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:26,039][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:31,045][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:36,046][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:41,047][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:46,048][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:51,050][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:02:56,056][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:01,062][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:06,064][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:11,071][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:16,073][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:21,074][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:26,078][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:31,079][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:36,080][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:41,082][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:46,084][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:51,087][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:03:56,088][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:01,089][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:06,091][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:11,093][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:16,094][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:21,095][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:26,096][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:31,099][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:36,101][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:41,102][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:46,103][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:51,104][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:04:56,105][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:01,106][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:06,107][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:11,108][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:16,110][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:21,111][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:26,112][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:31,113][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:36,114][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:41,115][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:46,116][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:51,118][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:05:56,122][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:01,123][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:06,124][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:11,125][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:16,127][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:21,129][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:24,181][ INFO][PID:1532150] Allocated host ResallocHost, ticket_id=307241, hostname=54.144.113.24, name=aws_x86_64_spot_prod_00503797_20251212_160421, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-12 16:06:24,197][ INFO][PID:1532150] Allocating ssh connection to builder [2025-12-12 16:06:24,214][ INFO][PID:1532150] Checking that builder machine is OK [2025-12-12 16:06:25,031][ INFO][PID:1532150] Running remote command: copr-builder-ready epel-10-x86_64 [2025-12-12 16:06:25,278][ INFO][PID:1532150] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-12-12 16:06:25,342][ INFO][PID:1532150] Filling build.info file with builder info [2025-12-12 16:06:25,398][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:25,517][ INFO][PID:1532150] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "networkmanager", "project_name": "NetworkManager-1.48-debug", "project_dirname": "NetworkManager-1.48-debug", "submitter": null, "ended_on": null, "started_on": 1765555585.4274485, "submitted_on": null, "status": 3, "chroot": "epel-10-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9903163-epel-10-x86_64", "build_id": 9903163, "package_name": "NetworkManager", "package_version": "1:1.48.17-33403.copr.247f9cfda8", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/networkmanager/NetworkManager-1.48-debug/NetworkManager", "git_hash": "9b601f30d98c5526341ad3b7c61acaaa8a13911c", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "networkmanager/NetworkManager-1.48-debug--07849acc-d72a-4836-b31c-940682bf32b8", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/nmstate/nm-build-deps/epel-10-$basearch/", "id": "https_download_copr_fedorainfracloud_org_results_nmstate_nm_build_deps_epel_10_basearch", "name": "Additional repo https_download_copr_fedorainfracloud_org_results_nmstate_nm_build_deps_epel_10_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-1.48-debug", "result_dir": "09903163-NetworkManager", "built_packages": "", "id": 9903163, "mockchain_macros": { "copr_username": "networkmanager", "copr_projectname": "NetworkManager-1.48-debug", "vendor": "Fedora Project COPR (networkmanager/NetworkManager-1.48-debug)" } } ] } [2025-12-12 16:06:29,024][ INFO][PID:1532150] Sending fedora-messaging bus message in build.start [2025-12-12 16:06:44,308][ INFO][PID:1532150] Sending fedora-messaging bus message in chroot.start [2025-12-12 16:06:44,338][ INFO][PID:1532150] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9903163-epel-10-x86_64 --chroot epel-10-x86_64 --detached [2025-12-12 16:06:44,806][ INFO][PID:1532150] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '54.144.113.24' (ED25519) to the list of known hosts. [2025-12-12 16:06:44,807][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 1 [2025-12-12 16:06:44,810][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.144.113.24 copr-rpmbuild-log [2025-12-12 16:06:49,819][ INFO][PID:1532150] Periodic builder liveness probe: alive [2025-12-12 16:06:49,820][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:54,824][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:06:59,826][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:04,834][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:09,835][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:14,838][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:19,842][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:24,845][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:29,850][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:34,851][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:39,853][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:44,854][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:49,856][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:54,857][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:07:59,858][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:04,859][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:09,861][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:14,863][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:19,865][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:24,870][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:29,872][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:34,873][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:39,876][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:44,877][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:49,880][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:54,882][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:08:59,885][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:04,889][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:09,893][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:14,897][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:19,911][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:24,913][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:29,915][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:34,917][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:39,919][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:44,924][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:49,928][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:54,930][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:09:59,931][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:04,933][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:09,934][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:14,935][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:19,936][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:24,940][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:29,943][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:34,946][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:39,948][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:44,949][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:49,952][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:54,953][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:10:59,954][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:04,955][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:09,957][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:14,958][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:17,720][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:11:19,963][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:24,965][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:27,721][WARNING][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:11:29,966][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:34,968][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:37,723][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 2 [2025-12-12 16:11:37,730][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.144.113.24 copr-rpmbuild-log [2025-12-12 16:11:37,752][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:11:39,969][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:44,970][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:47,753][WARNING][PID:1532150] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:11:49,973][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:54,977][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:11:57,754][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 3 [2025-12-12 16:11:57,756][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.144.113.24 copr-rpmbuild-log [2025-12-12 16:11:59,979][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:04,982][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:09,991][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:14,994][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:19,995][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:24,997][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:30,000][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:35,001][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:40,002][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:45,003][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:50,005][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:55,006][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:12:57,767][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:13:00,007][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:05,008][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:07,771][WARNING][PID:1532150] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:13:10,009][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:15,011][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:17,772][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 4 [2025-12-12 16:13:17,774][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.144.113.24 copr-rpmbuild-log [2025-12-12 16:13:20,014][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:25,017][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:30,021][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:35,023][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:40,026][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:45,028][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:50,029][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:13:55,031][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:00,037][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:05,039][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:10,040][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:15,041][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:17,845][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:14:20,042][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:25,043][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:27,855][ ERROR][PID:1532150] Giving up for unstable SSH, failures: 1765555887.721782, 1765555907.7536664, 1765555987.77121, 1765556067.8492293 [2025-12-12 16:14:27,861][ ERROR][PID:1532150] Re-try request for task on 'ResallocHost, ticket_id=307241, hostname=54.144.113.24, name=aws_x86_64_spot_prod_00503797_20251212_160421, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-12 16:14:27,862][ INFO][PID:1532150] Releasing VM back to pool [2025-12-12 16:14:27,893][ INFO][PID:1532150] Retry #1 (on other host) [2025-12-12 16:14:27,894][ INFO][PID:1532150] Checking that /var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/repodata/repomd.xml exists [2025-12-12 16:14:27,896][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:27,898][ INFO][PID:1532150] VM allocation process starts [2025-12-12 16:14:27,914][ INFO][PID:1532150] Trying to allocate VM: ResallocHost, ticket_id=307575, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-12 16:14:32,917][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:37,919][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:42,921][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:47,922][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:52,923][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:14:57,924][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:02,926][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:07,929][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:12,930][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:17,931][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:22,933][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:27,934][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:32,935][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:37,936][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:42,937][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:47,939][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:52,942][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:15:57,945][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:02,946][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:07,947][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:12,948][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:17,949][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:22,950][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:27,953][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:32,954][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:37,955][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:42,958][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:47,966][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:52,968][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:16:57,969][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:02,970][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:07,973][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:12,974][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:17,975][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:22,976][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:27,977][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:32,978][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:37,979][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:42,980][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:47,983][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:52,984][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:17:57,985][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:02,989][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:07,991][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:12,992][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:17,993][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:22,994][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:27,995][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:32,996][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:37,997][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:42,999][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:48,000][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:53,004][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:18:58,006][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:03,007][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:08,008][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:13,010][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:18,017][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:23,019][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:28,020][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:33,021][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:38,022][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:43,024][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:48,026][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:53,028][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:19:58,030][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:03,034][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:08,037][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:13,039][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:18,042][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:23,045][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:28,047][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:33,055][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:38,056][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:43,057][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:48,059][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:53,066][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:20:58,068][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:03,069][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:08,074][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:13,075][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:18,077][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:23,079][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:28,080][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:33,081][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:38,082][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:43,084][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:48,086][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:53,088][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:21:58,094][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:03,095][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:08,096][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:13,098][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:18,102][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:23,105][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:28,109][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:33,112][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:38,114][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:43,121][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:46,111][ INFO][PID:1532150] Allocated host ResallocHost, ticket_id=307575, hostname=44.211.30.62, name=aws_x86_64_spot_prod_00504648_20251212_162021, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-12 16:22:46,111][ INFO][PID:1532150] Allocating ssh connection to builder [2025-12-12 16:22:46,114][ INFO][PID:1532150] Checking that builder machine is OK [2025-12-12 16:22:46,473][ INFO][PID:1532150] Running remote command: copr-builder-ready epel-10-x86_64 [2025-12-12 16:22:46,732][ INFO][PID:1532150] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-12-12 16:22:46,733][ INFO][PID:1532150] Filling build.info file with builder info [2025-12-12 16:22:46,735][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:46,737][ INFO][PID:1532150] Not re-notifying FE and msg buses for the new host. [2025-12-12 16:22:46,740][ INFO][PID:1532150] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9903163-epel-10-x86_64 --chroot epel-10-x86_64 --detached [2025-12-12 16:22:47,121][ INFO][PID:1532150] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '44.211.30.62' (ED25519) to the list of known hosts. [2025-12-12 16:22:47,122][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 1 [2025-12-12 16:22:47,137][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.211.30.62 copr-rpmbuild-log [2025-12-12 16:22:52,151][ INFO][PID:1532150] Periodic builder liveness probe: alive [2025-12-12 16:22:52,155][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:22:57,167][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:02,171][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:07,175][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:12,176][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:17,183][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:22,184][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:27,190][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:32,192][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:37,204][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:42,209][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:47,212][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:52,214][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:23:57,222][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:02,236][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:07,239][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:12,241][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:17,245][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:22,247][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:27,253][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:32,260][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:37,263][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:42,265][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:47,268][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:52,272][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:24:57,275][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:02,279][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:07,288][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:12,299][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:17,301][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:22,302][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:27,306][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:32,311][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:37,325][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:42,330][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:47,337][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:52,342][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:25:57,777][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:02,779][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:07,782][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:12,797][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:17,798][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:17,840][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:26:22,800][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:27,804][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:27,850][WARNING][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:26:32,805][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:37,807][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:37,852][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 2 [2025-12-12 16:26:37,861][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.211.30.62 copr-rpmbuild-log [2025-12-12 16:26:42,810][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:47,812][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:52,822][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:26:57,826][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:02,829][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:07,832][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:12,840][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:17,842][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:22,843][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:27,845][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:32,846][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:37,847][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:37,943][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:27:42,851][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:47,855][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:47,947][WARNING][PID:1532150] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:27:52,859][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:57,862][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:27:57,952][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 3 [2025-12-12 16:27:57,956][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.211.30.62 copr-rpmbuild-log [2025-12-12 16:28:02,866][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:07,868][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:12,869][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:17,870][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:22,871][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:27,873][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:32,874][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:37,876][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:42,877][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:47,878][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:52,879][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:57,880][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:28:58,030][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:29:02,881][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:07,882][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:08,032][WARNING][PID:1532150] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:29:12,883][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:17,884][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:18,033][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 4 [2025-12-12 16:29:18,034][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.211.30.62 copr-rpmbuild-log [2025-12-12 16:29:22,886][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:27,887][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:32,889][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:37,902][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:42,904][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:47,908][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:52,915][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:29:57,922][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:02,923][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:07,924][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:12,927][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:17,928][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:18,106][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:30:22,929][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:27,930][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:28,114][ ERROR][PID:1532150] Giving up for unstable SSH, failures: 1765556787.8507245, 1765556867.9476626, 1765556948.0323887, 1765557028.1103792 [2025-12-12 16:30:28,123][ ERROR][PID:1532150] Re-try request for task on 'ResallocHost, ticket_id=307575, hostname=44.211.30.62, name=aws_x86_64_spot_prod_00504648_20251212_162021, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-12 16:30:28,126][ INFO][PID:1532150] Releasing VM back to pool [2025-12-12 16:30:28,147][ INFO][PID:1532150] Retry #2 (on other host) [2025-12-12 16:30:28,147][ INFO][PID:1532150] Checking that /var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/repodata/repomd.xml exists [2025-12-12 16:30:28,150][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:28,150][ INFO][PID:1532150] VM allocation process starts [2025-12-12 16:30:28,168][ INFO][PID:1532150] Trying to allocate VM: ResallocHost, ticket_id=307871, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-12 16:30:33,177][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:38,183][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:43,184][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:48,190][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:53,191][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:30:58,198][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:03,200][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:08,202][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:13,203][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:18,204][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:23,205][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:28,267][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:33,268][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:38,269][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:43,272][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:48,274][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:53,277][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:31:58,278][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:03,280][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:08,289][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:13,290][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:18,296][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:23,298][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:28,302][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:33,304][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:38,306][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:43,307][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:46,312][ INFO][PID:1532150] Allocated host ResallocHost, ticket_id=307871, hostname=3.239.84.20, name=aws_x86_64_spot_prod_00505040_20251212_163044, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-12 16:32:46,313][ INFO][PID:1532150] Allocating ssh connection to builder [2025-12-12 16:32:46,314][ INFO][PID:1532150] Checking that builder machine is OK [2025-12-12 16:32:46,685][ INFO][PID:1532150] Running remote command: copr-builder-ready epel-10-x86_64 [2025-12-12 16:32:46,910][ INFO][PID:1532150] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-12-12 16:32:46,912][ INFO][PID:1532150] Filling build.info file with builder info [2025-12-12 16:32:47,029][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:47,031][ INFO][PID:1532150] Not re-notifying FE and msg buses for the new host. [2025-12-12 16:32:47,032][ INFO][PID:1532150] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9903163-epel-10-x86_64 --chroot epel-10-x86_64 --detached [2025-12-12 16:32:47,373][ INFO][PID:1532150] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.239.84.20' (ED25519) to the list of known hosts. [2025-12-12 16:32:47,376][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 1 [2025-12-12 16:32:47,378][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.239.84.20 copr-rpmbuild-log [2025-12-12 16:32:52,389][ INFO][PID:1532150] Periodic builder liveness probe: alive [2025-12-12 16:32:52,669][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:32:57,671][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:02,673][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:07,674][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:12,675][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:17,676][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:22,679][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:27,680][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:32,681][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:37,682][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:42,686][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:47,688][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:52,690][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:33:57,694][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:02,695][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:07,697][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:12,703][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:17,704][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:22,710][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:27,713][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:32,715][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:37,716][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:42,717][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:47,718][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:52,720][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:34:57,721][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:02,722][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:07,724][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:12,725][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:17,726][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:22,728][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:27,730][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:32,731][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:37,733][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:42,735][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:47,736][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:52,737][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:35:57,739][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:02,741][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:07,744][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:12,745][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:17,746][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:22,748][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:27,749][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:32,752][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:37,754][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:42,757][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:47,762][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:52,764][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:36:57,773][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:02,774][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:07,776][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:12,780][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:17,782][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:22,784][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:27,790][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:32,797][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:37,802][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:42,804][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:47,807][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:52,829][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:37:57,839][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:02,842][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:07,843][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:12,846][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:17,852][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:22,855][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:27,858][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:32,864][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:37,869][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:42,871][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:47,875][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:52,877][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:38:57,879][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:02,884][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:07,889][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:12,908][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:17,909][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:22,916][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:27,923][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:32,927][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:37,941][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:42,942][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:47,944][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:52,946][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:39:57,947][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:02,953][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:07,956][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:12,958][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:17,959][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:22,962][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:27,963][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:32,964][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:37,966][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:42,976][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:47,978][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:52,980][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:40:57,983][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:02,986][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:07,995][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:12,997][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:18,008][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:23,011][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:28,013][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:33,016][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:38,022][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:43,025][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:48,073][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:53,075][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:41:58,080][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:03,088][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:08,099][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:13,108][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:18,110][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:23,114][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:28,116][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:33,120][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:38,121][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:43,132][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:42:48,136][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:07,231][ INFO][PID:1532150] Periodic builder liveness probe: alive [2025-12-12 16:43:07,779][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:12,789][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:17,791][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:22,794][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:27,795][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:32,800][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:37,803][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:42,804][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:47,814][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:52,821][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:43:57,822][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:02,827][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:07,829][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:12,835][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:17,847][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:22,848][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:27,850][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:32,852][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:37,856][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:42,860][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:47,871][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:52,873][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:44:57,886][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:02,888][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:07,890][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:12,892][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:17,899][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:22,901][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:27,904][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:32,911][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:37,918][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:42,925][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:47,927][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:52,929][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:45:57,932][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:02,936][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:07,937][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:12,940][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:17,941][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:22,944][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:27,946][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:32,948][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:37,949][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:42,951][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:47,952][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:52,953][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:46:57,954][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:02,955][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:07,956][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:12,959][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:17,960][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:22,961][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:27,963][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:32,964][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:37,966][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:42,969][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:47,971][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:52,976][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:47:57,977][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:02,979][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:07,980][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:12,981][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:17,982][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:22,983][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:27,984][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:32,985][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:37,986][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:42,987][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:47,988][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:52,989][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:48:57,990][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:02,991][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:07,993][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:12,994][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:17,995][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:22,996][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:27,998][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:32,999][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:38,000][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:43,001][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:48,002][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:53,004][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:49:58,005][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:03,007][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:08,010][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:13,014][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:18,017][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:23,018][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:28,023][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:33,024][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:38,027][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:43,030][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:48,034][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:53,036][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:50:58,038][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:03,040][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:08,041][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:13,042][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:18,043][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:23,044][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:28,046][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:33,047][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:38,048][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:43,049][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:48,050][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:53,051][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:51:58,053][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:03,054][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:06,199][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:52:08,055][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:13,056][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:16,224][WARNING][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:52:18,057][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:23,058][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:26,235][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 2 [2025-12-12 16:52:26,373][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.239.84.20 copr-rpmbuild-log [2025-12-12 16:52:28,059][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:33,060][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:38,061][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:43,062][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:48,063][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:53,064][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:52:58,064][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:03,065][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:08,250][ INFO][PID:1532150] Periodic builder liveness probe: alive [2025-12-12 16:53:08,262][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:13,263][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:18,264][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:23,265][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:26,422][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:53:28,266][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:33,267][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:36,428][WARNING][PID:1532150] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:53:38,268][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:43,269][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:46,428][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 3 [2025-12-12 16:53:46,431][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.239.84.20 copr-rpmbuild-log [2025-12-12 16:53:48,270][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:53,271][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:53:58,272][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:03,273][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:08,274][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:13,275][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:18,276][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:23,277][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:28,278][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:33,278][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:38,280][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:43,281][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:46,479][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:54:48,282][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:53,282][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:54:56,483][WARNING][PID:1532150] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-12 16:54:58,284][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:03,285][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:06,484][ INFO][PID:1532150] Downloading the builder-live.log file, attempt 4 [2025-12-12 16:55:06,486][ INFO][PID:1532150] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.239.84.20 copr-rpmbuild-log [2025-12-12 16:55:08,286][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:13,287][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:18,288][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:23,289][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:28,290][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:33,291][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:38,292][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:43,293][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:48,294][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:53,296][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:55:58,296][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:56:03,297][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:56:06,553][ ERROR][PID:1532150] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-12 16:56:08,298][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:56:13,299][ INFO][PID:1532150] Checking for cancel request [2025-12-12 16:56:16,584][ ERROR][PID:1532150] Giving up for unstable SSH, failures: 1765558336.2219758, 1765558416.4280758, 1765558496.4838214, 1765558576.554528 [2025-12-12 16:56:16,609][ ERROR][PID:1532150] Re-try request for task on 'ResallocHost, ticket_id=307871, hostname=3.239.84.20, name=aws_x86_64_spot_prod_00505040_20251212_163044, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-12 16:56:16,612][ INFO][PID:1532150] Releasing VM back to pool [2025-12-12 16:56:16,641][ ERROR][PID:1532150] Backend process error: Three host tried without success: {'54.144.113.24', '3.239.84.20', '44.211.30.62'} [2025-12-12 16:56:16,652][WARNING][PID:1532150] Switching not-finished job state to 'failed' [2025-12-12 16:56:16,655][ INFO][PID:1532150] Worker failed build, took 1409.6153457164764 [2025-12-12 16:56:16,678][ INFO][PID:1532150] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "networkmanager", "project_name": "NetworkManager-1.48-debug", "project_dirname": "NetworkManager-1.48-debug", "submitter": null, "ended_on": 1765558576.6468105, "started_on": 1765557167.0314648, "submitted_on": null, "status": 0, "chroot": "epel-10-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9903163-epel-10-x86_64", "build_id": 9903163, "package_name": "NetworkManager", "package_version": "1:1.48.17-33403.copr.247f9cfda8", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/networkmanager/NetworkManager-1.48-debug/NetworkManager", "git_hash": "9b601f30d98c5526341ad3b7c61acaaa8a13911c", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "networkmanager/NetworkManager-1.48-debug--07849acc-d72a-4836-b31c-940682bf32b8", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/nmstate/nm-build-deps/epel-10-$basearch/", "id": "https_download_copr_fedorainfracloud_org_results_nmstate_nm_build_deps_epel_10_basearch", "name": "Additional repo https_download_copr_fedorainfracloud_org_results_nmstate_nm_build_deps_epel_10_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-1.48-debug", "result_dir": "09903163-NetworkManager", "built_packages": "", "id": 9903163, "mockchain_macros": { "copr_username": "networkmanager", "copr_projectname": "NetworkManager-1.48-debug", "vendor": "Fedora Project COPR (networkmanager/NetworkManager-1.48-debug)" } } ] } [2025-12-12 16:56:17,835][ INFO][PID:1532150] Sending fedora-messaging bus message in build.end [2025-12-12 16:56:18,299][ INFO][PID:1532150] Compressing /var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/09903163-NetworkManager/builder-live.log by gzip [2025-12-12 16:56:18,306][ INFO][PID:1532150] Running command 'gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/09903163-NetworkManager/builder-live.log' as PID 2001504 [2025-12-12 16:56:18,339][ INFO][PID:1532150] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/09903163-NetworkManager/builder-live.log) [2025-12-12 16:56:18,349][ INFO][PID:1532150] Compressing /var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/09903163-NetworkManager/backend.log by gzip [2025-12-12 16:56:18,350][ INFO][PID:1532150] Running command 'gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-1.48-debug/epel-10-x86_64/09903163-NetworkManager/backend.log' as PID 2001509