[2025-12-16 23:12:34,503][ INFO][PID:3598975] Marking build as starting [2025-12-16 23:12:34,885][ INFO][PID:3598975] Checking that /var/lib/copr/public_html/results/rhcontainerbot/bootc/fedora-42-ppc64le/repodata/repomd.xml exists [2025-12-16 23:12:34,984][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:12:34,990][ INFO][PID:3598975] VM allocation process starts [2025-12-16 23:12:35,035][ INFO][PID:3598975] Trying to allocate VM: ResallocHost, ticket_id=376561, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-16 23:12:40,038][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:12:41,095][ INFO][PID:3598975] Allocated host ResallocHost, ticket_id=376561, hostname=2620:52:6:1161:dead:beef:cafe:c188, name=vmhost_p09_04_prod_00625266_20251216_224432, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-16 23:12:41,097][ INFO][PID:3598975] Allocating ssh connection to builder [2025-12-16 23:12:41,101][ INFO][PID:3598975] Checking that builder machine is OK [2025-12-16 23:13:41,185][ ERROR][PID:3598975] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c188 port 22: Connection timed out [2025-12-16 23:13:51,189][ ERROR][PID:3598975] Re-try request for task on 'ResallocHost, ticket_id=376561, hostname=2620:52:6:1161:dead:beef:cafe:c188, name=vmhost_p09_04_prod_00625266_20251216_224432, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts [2025-12-16 23:13:51,189][ INFO][PID:3598975] Releasing VM back to pool [2025-12-16 23:13:51,201][ INFO][PID:3598975] Retry #1 (on other host) [2025-12-16 23:13:51,202][ INFO][PID:3598975] Checking that /var/lib/copr/public_html/results/rhcontainerbot/bootc/fedora-42-ppc64le/repodata/repomd.xml exists [2025-12-16 23:13:51,202][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:13:51,203][ INFO][PID:3598975] VM allocation process starts [2025-12-16 23:13:51,213][ INFO][PID:3598975] Trying to allocate VM: ResallocHost, ticket_id=376608, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-16 23:13:54,227][ INFO][PID:3598975] Allocated host ResallocHost, ticket_id=376608, hostname=2620:52:6:1161:dead:beef:cafe:c18c, name=vmhost_p09_04_prod_00625386_20251216_225540, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-16 23:13:54,228][ INFO][PID:3598975] Allocating ssh connection to builder [2025-12-16 23:13:54,228][ INFO][PID:3598975] Checking that builder machine is OK [2025-12-16 23:13:55,268][ INFO][PID:3598975] Running remote command: copr-builder-ready fedora-42-ppc64le [2025-12-16 23:13:55,739][ INFO][PID:3598975] Red Hat subscription not needed for fedora-42-ppc64le Builder is ready to be used [2025-12-16 23:13:55,740][ INFO][PID:3598975] Filling build.info file with builder info [2025-12-16 23:13:55,741][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:13:55,742][ INFO][PID:3598975] Not re-notifying FE and msg buses for the new host. [2025-12-16 23:13:55,743][ INFO][PID:3598975] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9916198-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached [2025-12-16 23:13:56,735][ INFO][PID:3598975] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18c' (ED25519) to the list of known hosts. [2025-12-16 23:13:56,736][ INFO][PID:3598975] Downloading the builder-live.log file, attempt 1 [2025-12-16 23:13:56,737][ INFO][PID:3598975] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log [2025-12-16 23:14:01,742][ INFO][PID:3598975] Periodic builder liveness probe: alive [2025-12-16 23:14:01,743][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:06,744][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:11,746][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:16,748][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:21,750][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:26,751][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:31,752][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:36,755][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:41,759][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:46,761][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:51,762][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:14:56,768][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:01,771][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:06,772][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:11,773][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:16,774][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:21,776][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:26,777][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:31,780][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:36,781][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:41,783][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:46,784][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:51,787][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:15:56,791][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:01,792][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:06,793][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:11,794][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:16,795][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:21,796][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:26,797][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:31,798][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:36,799][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:41,800][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:46,801][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:51,802][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:16:56,803][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:01,804][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:06,805][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:11,806][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:16,807][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:21,808][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:26,809][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:31,809][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:36,810][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:41,811][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:46,812][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:51,813][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:17:56,814][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:01,815][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:06,816][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:11,817][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:16,818][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:21,819][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:26,820][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:31,821][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:36,821][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:41,822][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:46,823][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:51,824][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:18:56,825][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:01,826][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:06,827][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:11,828][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:16,829][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:21,830][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:26,831][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:31,832][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:36,833][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:41,834][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:46,835][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:51,836][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:19:56,837][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:01,838][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:06,839][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:11,842][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:16,843][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:21,844][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:26,845][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:31,846][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:36,847][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:41,848][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:46,849][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:51,850][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:20:56,851][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:01,852][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:06,853][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:11,854][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:16,855][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:21,885][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:26,886][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:31,887][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:36,888][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:41,889][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:46,890][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:51,891][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:21:56,891][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:01,892][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:06,893][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:11,894][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:16,895][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:21,896][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:26,897][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:31,898][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:36,899][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:41,900][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:46,900][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:51,901][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:22:56,902][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:01,903][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:06,904][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:11,905][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:16,906][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:21,907][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:26,908][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:31,909][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:36,909][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:41,910][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:46,911][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:51,912][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:23:56,913][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:01,919][ INFO][PID:3598975] Periodic builder liveness probe: alive [2025-12-16 23:24:01,920][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:06,920][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:11,921][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:16,922][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:21,923][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:26,924][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:31,925][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:36,926][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:41,927][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:46,928][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:51,928][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:24:56,929][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:01,931][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:06,932][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:11,932][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:16,933][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:21,934][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:26,935][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:31,936][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:36,937][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:41,938][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:46,939][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:51,940][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:25:56,941][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:01,941][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:06,942][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:11,943][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:16,944][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:21,945][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:26,946][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:31,947][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:36,948][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:41,949][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:46,949][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:51,950][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:26:56,951][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:01,952][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:06,953][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:11,954][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:16,955][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:21,956][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:26,957][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:31,958][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:36,959][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:41,959][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:46,961][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:51,962][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:56,962][ INFO][PID:3598975] Checking for cancel request [2025-12-16 23:27:56,963][ INFO][PID:3598975] calling cancel callback, and waiting [2025-12-16 23:28:57,029][ ERROR][PID:3598975] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18c port 22: Connection timed out [2025-12-16 23:30:07,098][ ERROR][PID:3598975] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18c port 22: Connection timed out [2025-12-16 23:30:20,811][ INFO][PID:3598975] Cancel request succeeded out: success err: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18c' (ED25519) to the list of known hosts. [2025-12-16 23:30:20,811][ ERROR][PID:3598975] Build was canceled [2025-12-16 23:30:20,812][ INFO][PID:3598975] Releasing VM back to pool [2025-12-16 23:30:20,820][WARNING][PID:3598975] Switching not-finished job state to 'failed' [2025-12-16 23:30:20,821][ INFO][PID:3598975] Worker failed build, took 985.0783684253693 [2025-12-16 23:30:20,821][ INFO][PID:3598975] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "rhcontainerbot", "project_name": "bootc", "project_dirname": "bootc", "submitter": "packit", "ended_on": 1765927820.8207877, "started_on": 1765926835.7424192, "submitted_on": null, "status": 0, "chroot": "fedora-42-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9916198-fedora-42-ppc64le", "build_id": 9916198, "package_name": "bootc", "package_version": "202512162308.g3c861fb27a-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/bootc/bootc", "git_hash": "b70a4a478e9841b88f964a4cba47155a3fce9caa", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "rhcontainerbot/bootc--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/fedora-42-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/bootc", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc", "result_dir": "09916198-bootc", "built_packages": "", "id": 9916198, "mockchain_macros": { "copr_username": "rhcontainerbot", "copr_projectname": "bootc", "vendor": "Fedora Project COPR (rhcontainerbot/bootc)" } } ] } [2025-12-16 23:30:20,898][ INFO][PID:3598975] Sending fedora-messaging bus message in build.end [2025-12-16 23:30:21,451][ INFO][PID:3598975] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/fedora-42-ppc64le/09916198-bootc/builder-live.log by gzip [2025-12-16 23:30:21,452][ INFO][PID:3598975] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/fedora-42-ppc64le/09916198-bootc/builder-live.log' as PID 3653099 [2025-12-16 23:30:21,547][ INFO][PID:3598975] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/fedora-42-ppc64le/09916198-bootc/builder-live.log) [2025-12-16 23:30:21,548][ INFO][PID:3598975] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/fedora-42-ppc64le/09916198-bootc/backend.log by gzip [2025-12-16 23:30:21,549][ INFO][PID:3598975] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/fedora-42-ppc64le/09916198-bootc/backend.log' as PID 3653103