[2025-07-27 07:11:37,554][ INFO][PID:1546524] Marking build as starting [2025-07-27 07:11:37,609][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:11:37,610][ INFO][PID:1546524] VM allocation process starts [2025-07-27 07:11:37,621][ INFO][PID:1546524] Trying to allocate VM: ResallocHost, ticket_id=4183063, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-07-27 07:11:40,661][ INFO][PID:1546524] Allocated host ResallocHost, ticket_id=4183063, hostname=140.211.168.9, name=copr_osuosl_p09_prod_05903536_20250727_070429, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-07-27 07:11:40,662][ INFO][PID:1546524] Allocating ssh connection to builder [2025-07-27 07:11:40,662][ INFO][PID:1546524] Checking that builder machine is OK [2025-07-27 07:11:41,930][ INFO][PID:1546524] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2025-07-27 07:11:42,974][ INFO][PID:1546524] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2025-07-27 07:11:42,975][ INFO][PID:1546524] Filling build.info file with builder info [2025-07-27 07:11:42,975][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:11:42,976][ INFO][PID:1546524] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "openblas-0.3.23", "project_dirname": "openblas-0.3.23", "submitter": "https://src.fedoraproject.org/user/releng", "ended_on": null, "started_on": 1753600302.9764838, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9319972-fedora-rawhide-ppc64le", "build_id": 9319972, "package_name": "korganizer", "package_version": "25.07.80-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/korganizer", "git_hash": "308d66f349b70775687d5f7c1ddddb11e6f9a99c", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/releng", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23", "result_dir": "09319972-korganizer", "built_packages": "", "id": 9319972, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "openblas-0.3.23", "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)" } } ] } [2025-07-27 07:11:43,045][ INFO][PID:1546524] Sending fedora-messaging bus message in build.start [2025-07-27 07:11:43,697][ INFO][PID:1546524] Sending fedora-messaging bus message in chroot.start [2025-07-27 07:11:43,723][ INFO][PID:1546524] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9319972-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2025-07-27 07:11:45,027][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 1 [2025-07-27 07:11:45,029][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.9 copr-rpmbuild-log [2025-07-27 07:11:50,034][ INFO][PID:1546524] Periodic builder liveness probe: alive [2025-07-27 07:11:50,035][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:11:55,036][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:00,037][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:05,038][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:10,039][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:15,040][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:20,041][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:25,042][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:30,043][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:35,045][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:40,046][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:45,047][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:50,048][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:12:55,049][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:00,050][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:05,051][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:10,052][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:15,053][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:20,054][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:25,055][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:30,057][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:35,058][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:40,059][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:45,060][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:50,061][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:13:55,063][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:00,065][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:05,066][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:10,067][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:15,069][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:20,070][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:25,071][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:30,072][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:35,073][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:40,074][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:45,075][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:50,076][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:14:55,077][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:00,078][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:05,079][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:10,080][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:15,081][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:20,082][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:25,083][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:30,084][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:35,085][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:40,087][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:45,088][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:50,089][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:15:55,090][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:00,091][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:05,092][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:10,093][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:14,139][ ERROR][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-27 07:16:15,094][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:20,095][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:24,140][WARNING][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-27 07:16:25,096][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:30,097][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:34,140][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 2 [2025-07-27 07:16:34,142][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.9 copr-rpmbuild-log [2025-07-27 07:16:35,098][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:40,099][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:45,100][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:50,101][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:16:55,102][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:00,103][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:05,104][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:10,105][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:15,106][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:20,107][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:25,108][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:30,109][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:34,279][ ERROR][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-27 07:17:35,110][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:40,111][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:44,284][WARNING][PID:1546524] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-27 07:17:45,113][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:50,114][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:17:54,285][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 3 [2025-07-27 07:17:54,286][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.9 copr-rpmbuild-log [2025-07-27 07:17:55,115][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:00,116][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:05,117][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:10,118][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:15,120][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:20,121][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:25,122][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:30,123][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:35,124][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:40,125][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:45,126][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:50,127][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:18:54,423][ ERROR][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-27 07:18:55,128][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:00,129][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:04,427][WARNING][PID:1546524] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-27 07:19:05,130][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:10,131][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:14,428][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 4 [2025-07-27 07:19:14,429][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.9 copr-rpmbuild-log [2025-07-27 07:19:15,132][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:20,133][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:25,134][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:30,135][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:35,136][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:40,137][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:45,138][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:50,139][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:19:55,140][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:00,141][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:05,142][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:10,143][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:14,563][ ERROR][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-27 07:20:15,144][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:20,145][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:24,568][ ERROR][PID:1546524] Giving up for unstable SSH, failures: 1753600584.1399875, 1753600664.2845547, 1753600744.4279032, 1753600824.568595 [2025-07-27 07:20:24,570][ ERROR][PID:1546524] Re-try request for task on 'ResallocHost, ticket_id=4183063, hostname=140.211.168.9, name=copr_osuosl_p09_prod_05903536_20250727_070429, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-07-27 07:20:24,570][ INFO][PID:1546524] Releasing VM back to pool [2025-07-27 07:20:24,580][ INFO][PID:1546524] Retry #1 (on other host) [2025-07-27 07:20:24,581][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:24,581][ INFO][PID:1546524] VM allocation process starts [2025-07-27 07:20:24,616][ INFO][PID:1546524] Trying to allocate VM: ResallocHost, ticket_id=4183095, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-07-27 07:20:29,617][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:34,618][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:39,619][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:44,620][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:49,621][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:54,622][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:20:59,623][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:04,624][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:09,625][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:14,626][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:19,627][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:24,628][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:29,629][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:34,630][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:39,631][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:44,632][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:49,633][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:54,634][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:21:59,635][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:04,636][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:09,637][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:14,638][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:19,639][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:24,640][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:29,641][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:34,642][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:39,643][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:44,644][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:49,645][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:54,646][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:22:59,647][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:04,648][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:09,649][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:14,650][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:19,651][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:24,652][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:29,653][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:34,654][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:39,655][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:44,656][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:49,657][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:54,658][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:23:59,659][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:04,660][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:09,661][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:14,662][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:19,663][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:24,664][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:29,665][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:34,666][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:39,667][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:44,668][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:49,669][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:54,670][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:24:59,671][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:04,673][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:09,674][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:14,675][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:19,676][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:24,677][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:29,678][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:34,679][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:39,680][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:44,681][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:49,682][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:54,683][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:25:59,684][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:04,685][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:09,686][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:14,687][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:19,689][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:24,690][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:29,691][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:34,692][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:39,693][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:44,694][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:49,695][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:54,696][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:26:59,697][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:04,698][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:09,699][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:14,700][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:19,702][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:24,703][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:29,704][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:34,705][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:39,705][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:42,723][ INFO][PID:1546524] Allocated host ResallocHost, ticket_id=4183095, hostname=140.211.168.163, name=copr_osuosl_p09_prod_05903638_20250727_072400, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-07-27 07:27:42,724][ INFO][PID:1546524] Allocating ssh connection to builder [2025-07-27 07:27:42,724][ INFO][PID:1546524] Checking that builder machine is OK [2025-07-27 07:27:44,051][ INFO][PID:1546524] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2025-07-27 07:27:45,069][ INFO][PID:1546524] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2025-07-27 07:27:45,069][ INFO][PID:1546524] Filling build.info file with builder info [2025-07-27 07:27:45,074][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:45,074][ INFO][PID:1546524] Not re-notifying FE and msg buses for the new host. [2025-07-27 07:27:45,075][ INFO][PID:1546524] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9319972-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2025-07-27 07:27:46,414][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 1 [2025-07-27 07:27:46,415][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.163 copr-rpmbuild-log [2025-07-27 07:27:51,420][ INFO][PID:1546524] Periodic builder liveness probe: alive [2025-07-27 07:27:51,421][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:27:56,422][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:01,423][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:06,424][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:11,425][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:16,426][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:21,427][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:26,428][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:31,429][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:36,430][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:41,431][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:46,432][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:51,433][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:28:56,434][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:01,435][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:06,436][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:11,437][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:16,438][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:21,439][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:26,440][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:31,441][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:36,442][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:41,443][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:46,444][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:51,445][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:29:56,446][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:01,447][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:06,448][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:11,450][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:16,451][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:21,452][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:26,453][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:31,454][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:36,455][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:41,456][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:46,457][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:51,458][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:30:56,459][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:01,460][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:06,461][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:11,462][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:16,464][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:21,465][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:26,466][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:31,467][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:36,468][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:41,469][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:46,470][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:51,472][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:31:56,474][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:01,476][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:06,477][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:11,478][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:16,479][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:21,480][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:26,481][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:31,482][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:36,483][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:41,484][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:46,485][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:51,486][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:32:56,487][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:01,488][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:06,489][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:11,490][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:16,491][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:21,492][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:26,494][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:31,495][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:36,496][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:41,497][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:46,498][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:51,499][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:33:56,500][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:01,501][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:06,502][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:11,503][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:16,504][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:21,505][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:26,506][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:31,507][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:36,508][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:41,509][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:46,510][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:51,511][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:34:56,512][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:01,513][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:06,514][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:11,515][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:16,516][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:21,517][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:26,518][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:31,519][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:36,520][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:41,521][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:46,522][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:51,523][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:35:56,524][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:01,525][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:06,526][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:11,527][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:16,528][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:21,529][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:26,530][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:31,532][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:36,532][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:41,534][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:46,535][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:51,536][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:36:56,537][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:01,538][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:06,539][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:11,540][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:16,541][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:21,542][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:26,543][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:31,544][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:36,545][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:41,546][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:46,548][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:51,556][ INFO][PID:1546524] Periodic builder liveness probe: alive [2025-07-27 07:37:51,557][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:37:56,558][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:01,559][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:06,560][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:11,561][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:16,562][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:21,563][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:26,564][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:31,565][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:36,567][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:41,567][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:46,568][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:51,569][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:38:56,571][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:01,572][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:06,573][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:11,574][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:16,575][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:21,576][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:26,577][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:31,578][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:36,580][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:41,581][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:46,582][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:51,583][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:39:56,584][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:01,585][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:06,586][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:11,587][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:16,588][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:21,589][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:26,590][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:31,591][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:36,593][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:41,594][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:46,595][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:51,596][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:40:56,597][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:01,598][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:06,599][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:11,600][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:16,601][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:21,602][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:26,607][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:31,611][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:36,612][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:41,613][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:46,614][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:51,615][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:41:56,616][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:01,617][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:06,618][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:11,620][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:16,621][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:21,622][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:26,623][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:31,624][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:36,625][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:41,626][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:46,628][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:51,629][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:42:56,630][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:01,631][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:06,632][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:11,633][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:16,634][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:21,635][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:26,636][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:31,637][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:36,638][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:41,639][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:46,640][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:51,641][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:43:56,642][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:01,643][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:06,644][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:11,645][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:16,646][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:21,648][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:26,649][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:31,650][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:36,651][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:41,652][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:46,653][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:51,654][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:44:56,655][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:01,656][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:06,657][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:11,658][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:16,659][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:21,660][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:26,661][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:31,662][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:36,663][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:41,664][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:46,665][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:51,666][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:45:56,667][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:01,668][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:06,669][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:11,670][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:16,671][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:21,672][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:26,673][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:31,674][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:36,675][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:41,676][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:46,678][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:51,679][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:46:56,680][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:01,681][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:06,682][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:11,683][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:16,684][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:21,685][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:26,686][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:31,687][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:36,688][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:41,689][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:46,690][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:51,696][ INFO][PID:1546524] Periodic builder liveness probe: alive [2025-07-27 07:47:51,697][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:47:56,698][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:01,699][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:06,700][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:11,701][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:16,702][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:21,703][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:26,704][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:31,705][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:36,706][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:41,707][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:46,708][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:51,709][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:48:56,710][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:01,711][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:06,712][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:11,713][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:16,714][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:21,715][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:26,716][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:31,717][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:36,718][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:41,719][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:46,720][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:51,721][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:49:56,724][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:01,725][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:06,726][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:11,728][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:16,729][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:21,730][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:26,731][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:31,732][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:36,733][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:41,734][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:46,735][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:51,736][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:50:56,737][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:01,738][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:06,739][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:11,740][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:16,741][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:21,742][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:26,743][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:28,139][ ERROR][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-27 07:51:31,744][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:36,745][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:38,139][WARNING][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-27 07:51:41,746][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:46,747][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:48,140][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 2 [2025-07-27 07:51:48,142][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.163 copr-rpmbuild-log [2025-07-27 07:51:51,748][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:51:56,749][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:01,751][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:06,752][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:11,753][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:16,754][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:21,755][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:26,757][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:31,758][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:36,759][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:41,760][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:46,761][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:48,210][ ERROR][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-27 07:52:51,762][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:56,763][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:52:58,214][WARNING][PID:1546524] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-27 07:53:01,764][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:06,765][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:08,215][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 3 [2025-07-27 07:53:08,217][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.163 copr-rpmbuild-log [2025-07-27 07:53:11,766][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:16,767][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:21,768][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:26,769][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:31,770][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:36,771][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:41,773][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:46,774][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:51,775][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:53:56,779][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:01,780][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:06,792][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:08,285][ ERROR][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-27 07:54:11,793][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:16,794][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:18,290][WARNING][PID:1546524] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-07-27 07:54:21,795][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:26,796][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:28,291][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 4 [2025-07-27 07:54:28,292][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.163 copr-rpmbuild-log [2025-07-27 07:54:31,797][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:36,798][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:41,799][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:46,800][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:51,801][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:54:56,802][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:01,803][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:06,804][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:11,805][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:16,806][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:21,807][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:26,808][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:28,360][ ERROR][PID:1546524] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-07-27 07:55:31,809][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:36,810][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:38,361][ ERROR][PID:1546524] Giving up for unstable SSH, failures: 1753602698.1398268, 1753602778.2147903, 1753602858.2905524, 1753602938.3611124 [2025-07-27 07:55:38,362][ ERROR][PID:1546524] Re-try request for task on 'ResallocHost, ticket_id=4183095, hostname=140.211.168.163, name=copr_osuosl_p09_prod_05903638_20250727_072400, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-07-27 07:55:38,362][ INFO][PID:1546524] Releasing VM back to pool [2025-07-27 07:55:38,371][ INFO][PID:1546524] Retry #2 (on other host) [2025-07-27 07:55:38,371][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:38,372][ INFO][PID:1546524] VM allocation process starts [2025-07-27 07:55:38,437][ INFO][PID:1546524] Trying to allocate VM: ResallocHost, ticket_id=4183197, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-07-27 07:55:43,438][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:48,439][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:53,440][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:55:58,441][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:03,442][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:08,443][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:13,444][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:18,445][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:23,446][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:28,447][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:33,448][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:38,449][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:43,450][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:48,451][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:53,452][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:56:58,453][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:03,454][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:08,455][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:13,456][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:18,457][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:23,458][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:28,458][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:33,460][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:38,461][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:43,462][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:48,462][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:53,463][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:57:58,464][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:03,465][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:08,466][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:13,467][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:18,468][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:23,469][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:28,470][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:33,471][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:38,472][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:43,473][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:48,474][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:53,475][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:58:56,515][ INFO][PID:1546524] Allocated host ResallocHost, ticket_id=4183197, hostname=2620:52:3:1:dead:beef:cafe:c24c, name=copr_hv_ppc64le_02_prod_05903653_20250727_072719, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-07-27 07:58:56,516][ INFO][PID:1546524] Allocating ssh connection to builder [2025-07-27 07:58:56,516][ INFO][PID:1546524] Checking that builder machine is OK [2025-07-27 07:58:59,373][ INFO][PID:1546524] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2025-07-27 07:59:00,009][ INFO][PID:1546524] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2025-07-27 07:59:00,009][ INFO][PID:1546524] Filling build.info file with builder info [2025-07-27 07:59:00,013][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:00,014][ INFO][PID:1546524] Not re-notifying FE and msg buses for the new host. [2025-07-27 07:59:00,014][ INFO][PID:1546524] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9319972-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2025-07-27 07:59:02,423][ INFO][PID:1546524] Downloading the builder-live.log file, attempt 1 [2025-07-27 07:59:02,425][ INFO][PID:1546524] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c24c copr-rpmbuild-log [2025-07-27 07:59:07,428][ INFO][PID:1546524] Periodic builder liveness probe: alive [2025-07-27 07:59:07,429][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:12,430][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:17,431][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:22,432][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:27,433][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:32,434][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:37,435][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:42,436][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:47,437][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:52,438][ INFO][PID:1546524] Checking for cancel request [2025-07-27 07:59:57,439][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:02,440][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:07,441][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:12,442][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:17,443][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:22,444][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:27,445][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:32,446][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:37,447][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:42,448][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:47,449][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:52,450][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:00:57,451][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:02,452][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:07,453][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:12,454][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:17,455][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:22,456][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:27,457][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:32,458][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:37,459][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:42,460][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:47,461][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:52,462][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:01:57,463][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:02,465][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:07,466][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:12,467][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:17,468][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:22,469][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:27,470][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:32,471][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:37,472][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:42,473][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:47,474][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:52,475][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:02:57,476][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:02,477][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:07,478][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:12,479][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:17,480][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:22,481][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:27,482][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:32,484][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:37,485][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:42,486][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:47,487][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:52,489][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:03:57,490][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:02,491][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:07,492][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:12,493][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:17,494][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:22,495][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:27,496][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:32,497][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:37,498][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:42,499][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:47,500][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:52,501][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:04:57,502][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:02,503][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:07,504][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:12,505][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:17,506][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:22,507][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:27,508][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:32,510][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:37,511][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:42,512][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:47,513][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:52,514][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:05:57,515][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:02,516][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:07,517][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:12,518][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:17,519][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:22,521][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:27,522][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:32,523][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:37,524][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:42,529][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:47,530][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:52,531][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:06:57,532][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:02,533][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:07,534][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:12,535][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:17,536][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:22,538][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:27,539][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:32,541][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:37,544][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:42,546][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:47,547][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:52,549][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:07:57,550][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:02,551][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:07,552][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:12,554][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:17,555][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:22,557][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:27,559][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:32,560][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:37,562][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:42,563][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:47,564][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:52,566][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:08:57,568][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:02,572][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:07,582][ INFO][PID:1546524] Periodic builder liveness probe: alive [2025-07-27 08:09:07,582][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:12,584][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:17,585][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:22,587][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:27,588][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:32,590][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:37,591][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:42,592][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:47,593][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:52,594][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:09:57,595][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:02,597][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:07,598][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:12,600][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:17,601][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:22,602][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:27,604][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:32,605][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:37,607][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:42,608][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:47,609][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:52,610][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:10:57,611][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:02,612][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:07,614][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:12,615][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:17,616][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:22,617][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:27,619][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:32,620][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:37,621][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:42,622][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:47,623][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:52,624][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:11:57,625][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:02,626][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:07,627][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:12,629][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:17,630][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:22,631][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:27,632][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:32,633][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:37,634][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:42,635][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:47,636][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:52,637][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:12:57,638][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:02,639][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:07,641][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:12,642][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:17,643][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:22,644][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:27,645][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:32,646][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:37,647][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:42,648][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:47,649][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:52,650][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:13:57,651][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:02,652][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:07,654][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:12,655][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:17,656][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:22,657][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:27,659][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:32,660][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:37,661][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:42,662][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:47,663][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:52,664][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:14:57,665][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:02,666][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:07,667][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:12,668][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:17,669][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:22,670][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:27,671][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:32,673][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:37,674][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:42,675][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:47,676][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:52,677][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:15:57,678][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:02,679][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:07,680][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:12,681][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:17,682][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:22,683][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:27,684][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:32,686][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:37,687][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:42,688][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:47,689][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:52,690][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:16:57,691][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:17:02,692][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:17:07,694][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:17:12,695][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:17:17,696][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:17:22,697][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:17:27,699][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:17:32,700][ INFO][PID:1546524] Checking for cancel request [2025-07-27 08:17:32,932][ INFO][PID:1546524] Downloading results from builder [2025-07-27 08:17:32,933][ INFO][PID:1546524] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c24c]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer started [2025-07-27 08:17:32,934][ INFO][PID:1546524] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c24c]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/ &> /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/build-09319972.rsync.log [2025-07-27 08:17:35,925][ INFO][PID:1546524] rsyncing finished. [2025-07-27 08:17:35,925][ INFO][PID:1546524] Releasing VM back to pool [2025-07-27 08:17:35,934][ INFO][PID:1546524] Searching for 'success' file in resultdir [2025-07-27 08:17:35,934][ INFO][PID:1546524] Going to sign pkgs from source: 9319972-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le [2025-07-27 08:17:35,935][ INFO][PID:1546524] Calling '/bin/sign -u psimovec#openblas-0.3.23@copr.fedorahosted.org -p' (attempt #1) [2025-07-27 08:17:36,262][ INFO][PID:1546524] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-25.07.80-2.fc43.src.rpm' (attempt #1) [2025-07-27 08:17:36,955][ INFO][PID:1546524] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-25.07.80-2.fc43.src.rpm [2025-07-27 08:17:36,956][ INFO][PID:1546524] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-libs-debuginfo-25.07.80-2.fc43.ppc64le.rpm' (attempt #1) [2025-07-27 08:17:38,228][ INFO][PID:1546524] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-libs-debuginfo-25.07.80-2.fc43.ppc64le.rpm [2025-07-27 08:17:38,229][ INFO][PID:1546524] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-debugsource-25.07.80-2.fc43.ppc64le.rpm' (attempt #1) [2025-07-27 08:17:39,195][ INFO][PID:1546524] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-debugsource-25.07.80-2.fc43.ppc64le.rpm [2025-07-27 08:17:39,196][ INFO][PID:1546524] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-25.07.80-2.fc43.ppc64le.rpm' (attempt #1) [2025-07-27 08:17:40,244][ INFO][PID:1546524] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-25.07.80-2.fc43.ppc64le.rpm [2025-07-27 08:17:40,245][ INFO][PID:1546524] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-libs-25.07.80-2.fc43.ppc64le.rpm' (attempt #1) [2025-07-27 08:17:41,046][ INFO][PID:1546524] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-libs-25.07.80-2.fc43.ppc64le.rpm [2025-07-27 08:17:41,046][ INFO][PID:1546524] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-debuginfo-25.07.80-2.fc43.ppc64le.rpm' (attempt #1) [2025-07-27 08:17:41,708][ INFO][PID:1546524] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/korganizer-debuginfo-25.07.80-2.fc43.ppc64le.rpm [2025-07-27 08:17:41,709][ INFO][PID:1546524] Sign done [2025-07-27 08:17:41,709][ INFO][PID:1546524] Incremental createrepo run, adding 09319972-korganizer into https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le, (auto-create-repo=True) [2025-07-27 08:17:41,710][ INFO][PID:1546524] Running command 'copr-repo --batched /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le --add 09319972-korganizer --no-appstream-metadata' as PID 1728754 [2025-07-27 08:17:52,348][ INFO][PID:1546524] Finished after 10 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le --add 09319972-korganizer --no-appstream-metadata) [2025-07-27 08:17:52,350][ INFO][PID:1546524] Getting build details [2025-07-27 08:17:52,351][ INFO][PID:1546524] Listing built binary packages in /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer [2025-07-27 08:17:52,351][ INFO][PID:1546524] Built packages: ['korganizer 25.07.80', 'korganizer-debuginfo 25.07.80', 'korganizer-libs-debuginfo 25.07.80', 'korganizer-libs 25.07.80', 'korganizer-debugsource 25.07.80'] [2025-07-27 08:17:52,351][ INFO][PID:1546524] build details: {'built_packages': 'korganizer 25.07.80\nkorganizer-debuginfo 25.07.80\nkorganizer-libs-debuginfo 25.07.80\nkorganizer-libs 25.07.80\nkorganizer-debugsource 25.07.80'} [2025-07-27 08:17:52,352][ INFO][PID:1546524] Retrieving pubkey [2025-07-27 08:17:52,352][ INFO][PID:1546524] Calling '/bin/sign -u psimovec#openblas-0.3.23@copr.fedorahosted.org -p' (attempt #1) [2025-07-27 08:17:52,678][ INFO][PID:1546524] Added pubkey for user psimovec project openblas-0.3.23 into: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/pubkey.gpg [2025-07-27 08:17:52,679][ INFO][PID:1546524] Finished build: id=9319972 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/openblas-0.3.23 chroot=fedora-rawhide-ppc64le [2025-07-27 08:17:52,681][ INFO][PID:1546524] Worker succeeded build, took 1132.6671562194824 [2025-07-27 08:17:52,682][ INFO][PID:1546524] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "openblas-0.3.23", "project_dirname": "openblas-0.3.23", "submitter": "https://src.fedoraproject.org/user/releng", "ended_on": 1753604272.6813786, "started_on": 1753603140.0142224, "submitted_on": null, "status": 1, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9319972-fedora-rawhide-ppc64le", "build_id": 9319972, "package_name": "korganizer", "package_version": "25.07.80-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/korganizer", "git_hash": "308d66f349b70775687d5f7c1ddddb11e6f9a99c", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/releng", "results": { "packages": [ { "name": "korganizer", "epoch": null, "version": "25.07.80", "release": "2.fc43", "arch": "ppc64le" }, { "name": "korganizer-debuginfo", "epoch": null, "version": "25.07.80", "release": "2.fc43", "arch": "ppc64le" }, { "name": "korganizer-libs-debuginfo", "epoch": null, "version": "25.07.80", "release": "2.fc43", "arch": "ppc64le" }, { "name": "korganizer-libs", "epoch": null, "version": "25.07.80", "release": "2.fc43", "arch": "ppc64le" }, { "name": "korganizer", "epoch": null, "version": "25.07.80", "release": "2.fc43", "arch": "src" }, { "name": "korganizer-debugsource", "epoch": null, "version": "25.07.80", "release": "2.fc43", "arch": "ppc64le" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23", "result_dir": "09319972-korganizer", "built_packages": "korganizer 25.07.80\nkorganizer-debuginfo 25.07.80\nkorganizer-libs-debuginfo 25.07.80\nkorganizer-libs 25.07.80\nkorganizer-debugsource 25.07.80", "id": 9319972, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "openblas-0.3.23", "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)" } } ] } [2025-07-27 08:17:52,728][ INFO][PID:1546524] Sending fedora-messaging bus message in build.end [2025-07-27 08:17:52,756][ INFO][PID:1546524] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/builder-live.log by gzip [2025-07-27 08:17:52,757][ INFO][PID:1546524] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/builder-live.log' as PID 1729138 [2025-07-27 08:17:52,785][ INFO][PID:1546524] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/builder-live.log) [2025-07-27 08:17:52,786][ INFO][PID:1546524] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/backend.log by gzip [2025-07-27 08:17:52,787][ INFO][PID:1546524] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09319972-korganizer/backend.log' as PID 1729141