[2026-04-20 13:17:47,942][  INFO][PID:1961767] Marking build as starting
[2026-04-20 13:17:48,094][  INFO][PID:1961767] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/repodata/repomd.xml exists
[2026-04-20 13:17:48,108][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:17:48,111][  INFO][PID:1961767] VM allocation process starts
[2026-04-20 13:17:48,133][  INFO][PID:1961767] Trying to allocate VM: ResallocHost, ticket_id=2421919, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']
[2026-04-20 13:17:53,135][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:17:58,136][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:00,193][  INFO][PID:1961767] Allocated host ResallocHost, ticket_id=2421919, hostname=2620:52:6:1161:dead:beef:cafe:c174, name=vmhost_p09_01_prod_07098692_20260420_104547, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']
[2026-04-20 13:18:00,194][  INFO][PID:1961767] Allocating ssh connection to builder
[2026-04-20 13:18:00,194][  INFO][PID:1961767] Checking that builder machine is OK
[2026-04-20 13:18:01,037][  INFO][PID:1961767] Installed copr-rpmbuild version: 1.8
[2026-04-20 13:18:01,040][  INFO][PID:1961767] Running remote command: copr-builder-ready centos-stream-10-ppc64le
[2026-04-20 13:18:01,696][  INFO][PID:1961767] Red Hat subscription not needed for centos-stream-10-ppc64le
Builder is ready to be used

[2026-04-20 13:18:01,697][  INFO][PID:1961767] Filling build.info file with builder info
[2026-04-20 13:18:01,698][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:01,699][  INFO][PID:1961767] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@osbuild",
            "project_name": "osbuild-composer",
            "project_dirname": "osbuild-composer",
            "submitter": "packit",
            "ended_on": null,
            "started_on": 1776691081.69906,
            "submitted_on": null,
            "status": 3,
            "chroot": "centos-stream-10-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10348571-centos-stream-10-ppc64le",
            "build_id": 10348571,
            "package_name": "osbuild-composer",
            "package_version": "169-1.20260420131710720793.main.5.g2414f89f9",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@osbuild/osbuild-composer/osbuild-composer",
            "git_hash": "601f84c74abff0a1499af288559d7a2f27f82597",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@osbuild/osbuild-composer--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@osbuild/osbuild-composer",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer",
            "result_dir": "10348571-osbuild-composer",
            "built_packages": "",
            "id": 10348571,
            "mockchain_macros": {
                "copr_username": "@osbuild",
                "copr_projectname": "osbuild-composer",
                "vendor": "Fedora Project COPR (@osbuild/osbuild-composer)"
            }
        }
    ]
}
[2026-04-20 13:18:01,788][  INFO][PID:1961767] Sending fedora-messaging bus message in build.start
[2026-04-20 13:18:02,845][  INFO][PID:1961767] Sending fedora-messaging bus message in chroot.start
[2026-04-20 13:18:02,909][  INFO][PID:1961767] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10348571-centos-stream-10-ppc64le --chroot centos-stream-10-ppc64le --detached
[2026-04-20 13:18:04,108][  INFO][PID:1961767] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c174' (ED25519) to the list of known hosts.


[2026-04-20 13:18:04,117][  INFO][PID:1961767] Downloading the builder-live.log file, attempt 1
[2026-04-20 13:18:04,121][  INFO][PID:1961767] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c174 copr-rpmbuild-log
[2026-04-20 13:18:09,128][  INFO][PID:1961767] Periodic builder liveness probe: alive
[2026-04-20 13:18:09,128][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:14,129][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:19,132][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:24,139][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:29,146][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:34,147][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:39,151][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:44,155][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:49,167][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:54,169][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:18:59,170][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:04,171][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:09,175][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:14,176][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:19,180][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:24,181][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:29,183][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:34,185][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:39,188][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:44,218][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:49,220][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:54,222][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:19:59,223][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:04,229][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:09,281][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:14,291][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:19,298][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:24,300][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:29,303][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:34,313][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:39,321][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:44,324][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:49,327][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:54,331][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:20:59,332][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:04,345][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:09,359][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:14,362][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:19,368][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:24,372][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:29,375][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:34,390][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:39,392][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:44,396][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:49,404][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:54,418][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:21:59,424][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:04,426][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:09,441][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:14,450][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:19,455][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:24,461][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:29,465][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:34,469][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:39,473][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:44,479][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:49,483][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:54,565][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:22:59,566][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:04,569][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:09,572][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:14,573][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:19,577][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:24,580][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:29,582][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:34,584][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:39,586][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:44,589][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:49,591][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:54,650][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:23:59,651][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:04,654][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:09,656][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:14,659][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:19,661][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:24,663][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:29,664][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:34,666][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:39,667][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:44,671][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:49,672][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:54,674][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:24:59,676][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:04,679][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:09,687][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:14,690][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:19,691][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:24,694][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:29,695][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:34,698][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:39,700][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:44,707][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:49,713][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:54,714][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:25:59,716][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:04,717][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:09,720][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:14,723][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:19,748][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:24,749][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:29,750][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:34,751][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:39,753][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:44,754][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:49,761][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:54,765][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:26:59,768][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:04,772][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:09,780][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:14,784][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:19,790][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:24,793][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:29,864][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:34,865][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:39,869][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:44,928][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:49,932][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:54,941][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:27:59,945][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:04,948][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:09,964][  INFO][PID:1961767] Periodic builder liveness probe: alive
[2026-04-20 13:28:09,965][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:14,966][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:20,063][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:25,067][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:30,072][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:35,074][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:40,081][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:45,090][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:50,094][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:28:55,099][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:00,101][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:05,105][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:10,108][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:15,114][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:20,117][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:25,119][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:30,158][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:35,165][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:40,178][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:45,184][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:50,188][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:29:55,191][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:00,195][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:05,201][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:10,202][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:15,204][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:20,205][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:25,209][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:30,212][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:35,215][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:40,225][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:45,241][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:50,245][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:30:55,249][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:00,251][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:05,255][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:10,262][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:15,265][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:20,267][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:25,271][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:30,278][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:35,287][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:40,290][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:45,294][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:31:48,162][  INFO][PID:1961767] Downloading results from builder
[2026-04-20 13:31:48,162][  INFO][PID:1961767] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c174]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer started
[2026-04-20 13:31:48,168][  INFO][PID:1961767] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c174]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/ &> /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/build-10348571.rsync.log
[2026-04-20 13:31:48,504][  INFO][PID:1961767] rsyncing finished.
[2026-04-20 13:31:48,599][ ERROR][PID:1961767] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:31:48,600][ ERROR][PID:1961767] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:31:58,604][  INFO][PID:1961767] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c174]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer started
[2026-04-20 13:31:58,648][  INFO][PID:1961767] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c174]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/ &> /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/build-10348571.rsync.log
[2026-04-20 13:31:58,931][  INFO][PID:1961767] rsyncing finished.
[2026-04-20 13:31:58,935][ ERROR][PID:1961767] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:31:58,935][ ERROR][PID:1961767] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:32:08,936][  INFO][PID:1961767] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c174]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer started
[2026-04-20 13:32:08,942][  INFO][PID:1961767] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c174]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/ &> /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/build-10348571.rsync.log
[2026-04-20 13:32:09,289][  INFO][PID:1961767] rsyncing finished.
[2026-04-20 13:32:09,290][ ERROR][PID:1961767] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:32:09,291][ ERROR][PID:1961767] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:32:19,294][ ERROR][PID:1961767] Re-try request for task on 'ResallocHost, ticket_id=2421919, hostname=2620:52:6:1161:dead:beef:cafe:c174, name=vmhost_p09_01_prod_07098692_20260420_104547, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']': Unable to finish after 3 SSH attempts
[2026-04-20 13:32:19,296][  INFO][PID:1961767] VM Release request
[2026-04-20 13:32:19,319][  INFO][PID:1961767] Retry #1 (on other host)
[2026-04-20 13:32:19,320][  INFO][PID:1961767] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/repodata/repomd.xml exists
[2026-04-20 13:32:19,379][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:32:19,381][  INFO][PID:1961767] VM allocation process starts
[2026-04-20 13:32:19,411][  INFO][PID:1961767] Trying to allocate VM: ResallocHost, ticket_id=2422456, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']
[2026-04-20 13:32:24,414][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:32:29,416][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:32:34,417][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:32:37,506][  INFO][PID:1961767] Allocated host ResallocHost, ticket_id=2422456, hostname=2620:52:6:1161:dead:beef:cafe:c124, name=vmhost_p09_01_prod_07114750_20260420_120115, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']
[2026-04-20 13:32:37,507][  INFO][PID:1961767] Allocating ssh connection to builder
[2026-04-20 13:32:37,507][  INFO][PID:1961767] Checking that builder machine is OK
[2026-04-20 13:32:37,804][ ERROR][PID:1961767] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c124' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@2620:52:6:1161:dead:beef:cafe:c124: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:32:47,806][ ERROR][PID:1961767] Re-try request for task on 'ResallocHost, ticket_id=2422456, hostname=2620:52:6:1161:dead:beef:cafe:c124, name=vmhost_p09_01_prod_07114750_20260420_120115, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:32:47,807][  INFO][PID:1961767] VM Release request
[2026-04-20 13:32:47,838][  INFO][PID:1961767] Retry #2 (on other host)
[2026-04-20 13:32:47,841][  INFO][PID:1961767] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/repodata/repomd.xml exists
[2026-04-20 13:32:47,843][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:32:47,849][  INFO][PID:1961767] VM allocation process starts
[2026-04-20 13:32:47,912][  INFO][PID:1961767] Trying to allocate VM: ResallocHost, ticket_id=2422484, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']
[2026-04-20 13:32:52,918][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:32:57,920][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:33:02,924][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:33:07,936][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:33:12,938][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:33:17,941][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:33:22,944][  INFO][PID:1961767] Checking for cancel request
[2026-04-20 13:33:26,156][  INFO][PID:1961767] Allocated host ResallocHost, ticket_id=2422484, hostname=2620:52:6:1161:dead:beef:cafe:c15e, name=vmhost_p09_04_prod_07115935_20260420_121036, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']
[2026-04-20 13:33:26,160][  INFO][PID:1961767] Allocating ssh connection to builder
[2026-04-20 13:33:26,164][  INFO][PID:1961767] Checking that builder machine is OK
[2026-04-20 13:33:26,503][ ERROR][PID:1961767] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c15e' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@2620:52:6:1161:dead:beef:cafe:c15e: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:33:36,504][ ERROR][PID:1961767] Re-try request for task on 'ResallocHost, ticket_id=2422484, hostname=2620:52:6:1161:dead:beef:cafe:c15e, name=vmhost_p09_04_prod_07115935_20260420_121036, requested_tags=['arch_power9', 'arch_ppc64le', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:33:36,532][  INFO][PID:1961767] VM Release request
[2026-04-20 13:33:36,547][ ERROR][PID:1961767] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c174', '2620:52:6:1161:dead:beef:cafe:c124', '2620:52:6:1161:dead:beef:cafe:c15e'}
[2026-04-20 13:33:36,547][WARNING][PID:1961767] Switching not-finished job state to 'failed'
[2026-04-20 13:33:36,548][  INFO][PID:1961767] Worker failed build, took 934.8488404750824
[2026-04-20 13:33:36,548][  INFO][PID:1961767] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@osbuild",
            "project_name": "osbuild-composer",
            "project_dirname": "osbuild-composer",
            "submitter": "packit",
            "ended_on": 1776692016.5479004,
            "started_on": 1776691081.69906,
            "submitted_on": null,
            "status": 0,
            "chroot": "centos-stream-10-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10348571-centos-stream-10-ppc64le",
            "build_id": 10348571,
            "package_name": "osbuild-composer",
            "package_version": "169-1.20260420131710720793.main.5.g2414f89f9",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@osbuild/osbuild-composer/osbuild-composer",
            "git_hash": "601f84c74abff0a1499af288559d7a2f27f82597",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@osbuild/osbuild-composer--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@osbuild/osbuild-composer",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer",
            "result_dir": "10348571-osbuild-composer",
            "built_packages": "",
            "id": 10348571,
            "mockchain_macros": {
                "copr_username": "@osbuild",
                "copr_projectname": "osbuild-composer",
                "vendor": "Fedora Project COPR (@osbuild/osbuild-composer)"
            }
        }
    ]
}
[2026-04-20 13:33:36,627][  INFO][PID:1961767] Sending fedora-messaging bus message in build.end
[2026-04-20 13:33:36,688][  INFO][PID:1961767] Compressing /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/builder-live.log by gzip
[2026-04-20 13:33:36,690][  INFO][PID:1961767] Running command 'gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/builder-live.log' as PID 2173624
[2026-04-20 13:33:36,800][  INFO][PID:1961767] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/builder-live.log)
[2026-04-20 13:33:36,802][  INFO][PID:1961767] Compressing /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/backend.log by gzip
[2026-04-20 13:33:36,803][  INFO][PID:1961767] Running command 'gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/centos-stream-10-ppc64le/10348571-osbuild-composer/backend.log' as PID 2173642