[2026-01-02 12:56:10,844][  INFO][PID:1225498] Marking build as starting
[2026-01-02 12:56:11,189][  INFO][PID:1225498] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-02 12:56:11,190][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:56:11,191][  INFO][PID:1225498] VM allocation process starts
[2026-01-02 12:56:11,202][  INFO][PID:1225498] Trying to allocate VM: ResallocHost, ticket_id=627334, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-02 12:56:14,220][  INFO][PID:1225498] Allocated host ResallocHost, ticket_id=627334, hostname=2620:52:6:1161:dead:beef:cafe:c152, name=vmhost_p09_03_prod_00891506_20260102_124940, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-02 12:56:14,220][  INFO][PID:1225498] Allocating ssh connection to builder
[2026-01-02 12:56:14,221][  INFO][PID:1225498] Checking that builder machine is OK
[2026-01-02 12:56:15,181][  INFO][PID:1225498] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-02 12:56:15,661][  INFO][PID:1225498] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2026-01-02 12:56:15,662][  INFO][PID:1225498] Filling build.info file with builder info
[2026-01-02 12:56:15,662][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:56:15,663][  INFO][PID:1225498] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@osbuild",
            "project_name": "osbuild-composer",
            "project_dirname": "osbuild-composer",
            "submitter": "packit",
            "ended_on": null,
            "started_on": 1767358575.6633315,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-42-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9969913-fedora-42-ppc64le",
            "build_id": 9969913,
            "package_name": "osbuild-composer",
            "package_version": "158-1.20260102125518286082.main.2.gbc95e77eb",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@osbuild/osbuild-composer/osbuild-composer",
            "git_hash": "1e61bd011ffe17dd0d928a7373bc096f5df8bce7",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@osbuild/osbuild-composer--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer/fedora-42-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@osbuild/osbuild-composer",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer",
            "result_dir": "09969913-osbuild-composer",
            "built_packages": "",
            "id": 9969913,
            "mockchain_macros": {
                "copr_username": "@osbuild",
                "copr_projectname": "osbuild-composer",
                "vendor": "Fedora Project COPR (@osbuild/osbuild-composer)"
            }
        }
    ]
}
[2026-01-02 12:56:15,834][  INFO][PID:1225498] Sending fedora-messaging bus message in build.start
[2026-01-02 12:56:16,337][  INFO][PID:1225498] Sending fedora-messaging bus message in chroot.start
[2026-01-02 12:56:16,363][  INFO][PID:1225498] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9969913-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2026-01-02 12:57:56,544][ ERROR][PID:1225498] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c152' (ED25519) to the list of known hosts.
Connection to 2620:52:6:1161:dead:beef:cafe:c152 port 22 timed out

[2026-01-02 12:58:06,545][ ERROR][PID:1225498] Re-try request for task on 'ResallocHost, ticket_id=627334, hostname=2620:52:6:1161:dead:beef:cafe:c152, name=vmhost_p09_03_prod_00891506_20260102_124940, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-02 12:58:06,545][  INFO][PID:1225498] Releasing VM back to pool
[2026-01-02 12:58:06,557][  INFO][PID:1225498] Retry #1 (on other host)
[2026-01-02 12:58:06,558][  INFO][PID:1225498] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-02 12:58:06,558][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:58:06,559][  INFO][PID:1225498] VM allocation process starts
[2026-01-02 12:58:06,570][  INFO][PID:1225498] Trying to allocate VM: ResallocHost, ticket_id=627352, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-02 12:58:09,581][  INFO][PID:1225498] Allocated host ResallocHost, ticket_id=627352, hostname=2620:52:6:1161:dead:beef:cafe:c190, name=vmhost_p09_04_prod_00891510_20260102_125025, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-02 12:58:09,582][  INFO][PID:1225498] Allocating ssh connection to builder
[2026-01-02 12:58:09,582][  INFO][PID:1225498] Checking that builder machine is OK
[2026-01-02 12:59:09,650][ ERROR][PID:1225498] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c190 port 22: Connection timed out

[2026-01-02 12:59:19,651][ ERROR][PID:1225498] Re-try request for task on 'ResallocHost, ticket_id=627352, hostname=2620:52:6:1161:dead:beef:cafe:c190, name=vmhost_p09_04_prod_00891510_20260102_125025, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-01-02 12:59:19,652][  INFO][PID:1225498] Releasing VM back to pool
[2026-01-02 12:59:19,660][  INFO][PID:1225498] Retry #2 (on other host)
[2026-01-02 12:59:19,660][  INFO][PID:1225498] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-42-ppc64le/repodata/repomd.xml exists
[2026-01-02 12:59:19,661][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:59:19,661][  INFO][PID:1225498] VM allocation process starts
[2026-01-02 12:59:19,715][  INFO][PID:1225498] Trying to allocate VM: ResallocHost, ticket_id=627358, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-02 12:59:22,750][  INFO][PID:1225498] Allocated host ResallocHost, ticket_id=627358, hostname=2620:52:6:1161:dead:beef:cafe:c14a, name=vmhost_p09_03_prod_00891518_20260102_125118, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-02 12:59:22,751][  INFO][PID:1225498] Allocating ssh connection to builder
[2026-01-02 12:59:22,751][  INFO][PID:1225498] Checking that builder machine is OK
[2026-01-02 12:59:23,685][  INFO][PID:1225498] Running remote command: copr-builder-ready fedora-42-ppc64le
[2026-01-02 12:59:24,250][  INFO][PID:1225498] Red Hat subscription not needed for fedora-42-ppc64le
Builder is ready to be used

[2026-01-02 12:59:24,250][  INFO][PID:1225498] Filling build.info file with builder info
[2026-01-02 12:59:24,262][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:59:24,263][  INFO][PID:1225498] Not re-notifying FE and msg buses for the new host.
[2026-01-02 12:59:24,264][  INFO][PID:1225498] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9969913-fedora-42-ppc64le --chroot fedora-42-ppc64le --detached
[2026-01-02 12:59:25,225][  INFO][PID:1225498] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14a' (ED25519) to the list of known hosts.


[2026-01-02 12:59:25,226][  INFO][PID:1225498] Downloading the builder-live.log file, attempt 1
[2026-01-02 12:59:25,227][  INFO][PID:1225498] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14a copr-rpmbuild-log
[2026-01-02 12:59:30,235][  INFO][PID:1225498] Periodic builder liveness probe: alive
[2026-01-02 12:59:30,236][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:59:35,237][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:59:40,238][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:59:45,239][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:59:50,240][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 12:59:55,241][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:00,242][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:05,243][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:10,244][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:15,245][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:20,246][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:25,247][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:30,248][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:35,249][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:40,250][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:45,251][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:50,252][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:00:55,253][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:00,254][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:05,255][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:10,256][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:15,257][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:20,259][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:25,260][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:30,261][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:35,262][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:40,263][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:45,264][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:50,265][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:01:55,266][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:00,267][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:05,268][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:10,270][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:15,270][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:20,271][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:25,272][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:30,274][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:35,275][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:40,276][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:45,277][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:50,278][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:02:55,279][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:00,280][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:05,281][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:10,282][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:15,283][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:20,284][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:25,298][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:30,299][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:35,300][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:40,301][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:45,302][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:50,303][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:03:55,304][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:00,305][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:05,306][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:10,307][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:15,308][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:20,310][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:25,311][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:30,312][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:35,313][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:40,339][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:45,569][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:50,570][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:04:55,571][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:00,572][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:05,576][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:10,580][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:15,581][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:20,583][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:25,584][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:30,585][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:35,586][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:40,587][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:45,588][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:50,590][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:05:55,591][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:00,592][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:05,593][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:10,594][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:15,608][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:20,613][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:25,614][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:30,615][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:35,617][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:40,618][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:45,619][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:50,620][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:06:55,621][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:00,622][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:05,623][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:10,627][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:15,628][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:20,632][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:25,634][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:30,638][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:35,640][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:40,641][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:45,643][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:50,650][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:07:55,656][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:00,660][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:05,662][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:10,663][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:15,666][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:20,668][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:25,673][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:30,675][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:35,677][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:40,679][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:45,682][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:50,684][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:08:55,685][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:00,686][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:05,688][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:10,692][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:15,695][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:20,701][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:25,706][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:30,719][  INFO][PID:1225498] Periodic builder liveness probe: alive
[2026-01-02 13:09:30,720][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:35,723][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:40,725][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:45,767][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:50,768][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:09:55,770][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:00,773][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:05,775][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:10,776][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:15,779][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:20,781][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:25,785][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:30,786][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:35,787][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:40,792][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:45,794][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:50,801][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:10:55,802][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:00,805][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:05,806][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:10,807][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:15,808][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:20,809][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:25,810][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:30,811][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:35,812][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:40,813][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:45,814][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:50,815][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:11:55,816][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:00,817][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:05,818][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:10,826][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:15,827][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:20,828][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:25,829][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:30,831][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:35,832][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:40,833][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:45,834][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:50,835][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:12:55,836][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:00,837][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:05,839][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:10,840][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:15,841][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:20,842][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:25,843][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:30,844][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:35,845][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:40,846][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:45,847][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:50,849][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:13:55,850][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:00,851][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:05,852][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:10,853][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:15,854][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:20,855][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:25,856][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:30,857][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:35,858][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:40,859][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:45,860][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:50,861][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:14:55,862][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:00,863][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:05,864][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:10,865][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:15,866][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:20,867][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:25,868][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:30,869][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:35,870][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:40,871][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:45,872][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:50,873][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:15:55,874][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:00,876][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:05,877][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:10,878][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:15,879][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:20,880][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:25,881][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:30,882][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:35,883][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:40,884][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:45,885][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:50,886][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:16:55,887][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:00,888][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:05,888][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:10,889][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:15,890][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:20,891][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:25,892][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:30,893][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:35,894][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:40,895][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:45,896][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:50,897][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:17:55,898][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:00,899][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:05,900][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:10,901][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:15,906][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:20,907][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:25,908][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:30,909][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:35,910][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:40,911][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:45,912][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:48,370][ ERROR][PID:1225498] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-02 13:18:50,913][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:55,914][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:18:58,371][WARNING][PID:1225498] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-02 13:19:00,915][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:19:05,916][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:19:08,372][  INFO][PID:1225498] Downloading the builder-live.log file, attempt 2
[2026-01-02 13:19:08,379][  INFO][PID:1225498] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14a copr-rpmbuild-log
[2026-01-02 13:19:10,917][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:19:15,918][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:19:20,919][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:19:25,920][  INFO][PID:1225498] Checking for cancel request
[2026-01-02 13:19:30,926][  INFO][PID:1225498] Periodic builder liveness probe: dead
[2026-01-02 13:19:30,927][  INFO][PID:1225498] calling cancel callback, and waiting
[2026-01-02 13:20:08,445][ ERROR][PID:1225498] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-02 13:20:18,457][WARNING][PID:1225498] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-02 13:20:28,458][  INFO][PID:1225498] Downloading the builder-live.log file, attempt 3
[2026-01-02 13:20:28,460][  INFO][PID:1225498] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14a copr-rpmbuild-log
[2026-01-02 13:20:30,998][ ERROR][PID:1225498] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c14a port 22: Connection timed out

[2026-01-02 13:21:28,510][ ERROR][PID:1225498] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-02 13:21:38,520][WARNING][PID:1225498] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-02 13:21:41,070][ ERROR][PID:1225498] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c14a port 22: Connection timed out

[2026-01-02 13:21:48,522][  INFO][PID:1225498] Downloading the builder-live.log file, attempt 4
[2026-01-02 13:21:48,524][  INFO][PID:1225498] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14a copr-rpmbuild-log
[2026-01-02 13:22:03,745][WARNING][PID:1225498] Can't cancel build
out:

err:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14a' (ED25519) to the list of known hosts.
Traceback (most recent call last):
  File "/usr/bin/copr-rpmbuild-cancel", line 85, in <module>
    _main()
    ~~~~~^^
  File "/usr/bin/copr-rpmbuild-cancel", line 69, in _main
    with open(pidfile, "r") as pidfd:
         ~~~~^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/copr-rpmbuild/pid'

[2026-01-02 13:24:08,818][ ERROR][PID:1225498] Re-try request for task on 'ResallocHost, ticket_id=627358, hostname=2620:52:6:1161:dead:beef:cafe:c14a, name=vmhost_p09_03_prod_00891518_20260102_125118, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED
[2026-01-02 13:24:08,825][  INFO][PID:1225498] Releasing VM back to pool
[2026-01-02 13:24:08,836][ ERROR][PID:1225498] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c152', '2620:52:6:1161:dead:beef:cafe:c14a', '2620:52:6:1161:dead:beef:cafe:c190'}
[2026-01-02 13:24:08,837][WARNING][PID:1225498] Switching not-finished job state to 'failed'
[2026-01-02 13:24:08,838][  INFO][PID:1225498] Worker failed build, took 1484.5738899707794
[2026-01-02 13:24:08,838][  INFO][PID:1225498] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@osbuild",
            "project_name": "osbuild-composer",
            "project_dirname": "osbuild-composer",
            "submitter": "packit",
            "ended_on": 1767360248.837555,
            "started_on": 1767358764.263665,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-42-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9969913-fedora-42-ppc64le",
            "build_id": 9969913,
            "package_name": "osbuild-composer",
            "package_version": "158-1.20260102125518286082.main.2.gbc95e77eb",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@osbuild/osbuild-composer/osbuild-composer",
            "git_hash": "1e61bd011ffe17dd0d928a7373bc096f5df8bce7",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@osbuild/osbuild-composer--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer/fedora-42-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@osbuild/osbuild-composer",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer",
            "result_dir": "09969913-osbuild-composer",
            "built_packages": "",
            "id": 9969913,
            "mockchain_macros": {
                "copr_username": "@osbuild",
                "copr_projectname": "osbuild-composer",
                "vendor": "Fedora Project COPR (@osbuild/osbuild-composer)"
            }
        }
    ]
}
[2026-01-02 13:24:08,918][  INFO][PID:1225498] Sending fedora-messaging bus message in build.end
[2026-01-02 13:24:08,969][  INFO][PID:1225498] Compressing /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-42-ppc64le/09969913-osbuild-composer/builder-live.log by gzip
[2026-01-02 13:24:08,970][  INFO][PID:1225498] Running command 'gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-42-ppc64le/09969913-osbuild-composer/builder-live.log' as PID 1399165
[2026-01-02 13:24:08,972][  INFO][PID:1225498] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-42-ppc64le/09969913-osbuild-composer/builder-live.log)
[2026-01-02 13:24:08,972][  INFO][PID:1225498] Compressing /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-42-ppc64le/09969913-osbuild-composer/backend.log by gzip
[2026-01-02 13:24:08,973][  INFO][PID:1225498] Running command 'gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-42-ppc64le/09969913-osbuild-composer/backend.log' as PID 1399166