[2026-02-10 20:16:57,012][  INFO][PID:3203363] Marking build as starting
[2026-02-10 20:16:57,133][  INFO][PID:3203363] Checking that /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/repodata/repomd.xml exists
[2026-02-10 20:16:57,249][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:16:57,251][  INFO][PID:3203363] VM allocation process starts
[2026-02-10 20:16:57,271][  INFO][PID:3203363] Trying to allocate VM: ResallocHost, ticket_id=1263292, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2026-02-10 20:17:00,290][  INFO][PID:3203363] Allocated host ResallocHost, ticket_id=1263292, hostname=2620:52:6:1161:dead:beef:cafe:c110, name=vmhost_p09_02_prod_02150777_20260210_201238, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2026-02-10 20:17:00,290][  INFO][PID:3203363] Allocating ssh connection to builder
[2026-02-10 20:17:00,292][  INFO][PID:3203363] Checking that builder machine is OK
[2026-02-10 20:17:01,300][  INFO][PID:3203363] Running remote command: copr-builder-ready centos-stream+epel-next-9-ppc64le
[2026-02-10 20:17:01,858][  INFO][PID:3203363] Red Hat subscription not needed for centos-stream+epel-next-9-ppc64le
Builder is ready to be used

[2026-02-10 20:17:01,858][  INFO][PID:3203363] Filling build.info file with builder info
[2026-02-10 20:17:01,859][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:01,860][  INFO][PID:3203363] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@meta",
            "project_name": "drgn",
            "project_dirname": "drgn",
            "submitter": "packit",
            "ended_on": null,
            "started_on": 1770754621.8603008,
            "submitted_on": null,
            "status": 3,
            "chroot": "centos-stream+epel-next-9-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10114070-centos-stream+epel-next-9-ppc64le",
            "build_id": 10114070,
            "package_name": "python-drgn",
            "package_version": "0.0.33+193.gf84a0beb-1.20260210201338848497.main.193.gf84a0beb",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@meta/drgn/python-drgn",
            "git_hash": "805a1678c2d4830b30cdb2d52c6a50eee3c8e3a2",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@meta/drgn--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://kojihub.stream.centos.org/kojifiles/repos/c9s-build/latest/ppc64le/",
                    "id": "https_kojihub_stream_centos_org_kojifiles_repos_c9s_build_latest_ppc64le",
                    "name": "Additional repo https_kojihub_stream_centos_org_kojifiles_repos_c9s_build_latest_ppc64le"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@meta/drgn",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@meta/drgn",
            "result_dir": "10114070-python-drgn",
            "built_packages": "",
            "id": 10114070,
            "mockchain_macros": {
                "copr_username": "@meta",
                "copr_projectname": "drgn",
                "vendor": "Fedora Project COPR (@meta/drgn)"
            }
        }
    ]
}
[2026-02-10 20:17:01,967][  INFO][PID:3203363] Sending fedora-messaging bus message in build.start
[2026-02-10 20:17:02,485][  INFO][PID:3203363] Sending fedora-messaging bus message in chroot.start
[2026-02-10 20:17:02,510][  INFO][PID:3203363] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10114070-centos-stream+epel-next-9-ppc64le --chroot centos-stream+epel-next-9-ppc64le --detached
[2026-02-10 20:17:03,540][  INFO][PID:3203363] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c110' (ED25519) to the list of known hosts.


[2026-02-10 20:17:03,687][  INFO][PID:3203363] Downloading the builder-live.log file, attempt 1
[2026-02-10 20:17:03,688][  INFO][PID:3203363] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c110 copr-rpmbuild-log
[2026-02-10 20:17:08,695][  INFO][PID:3203363] Periodic builder liveness probe: alive
[2026-02-10 20:17:08,695][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:13,696][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:18,726][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:23,739][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:28,740][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:33,743][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:38,745][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:43,778][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:48,779][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:53,817][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:17:58,922][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:03,925][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:09,175][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:14,176][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:19,185][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:24,186][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:29,187][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:34,188][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:39,190][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:44,191][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:49,192][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:54,193][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:18:59,194][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:04,208][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:09,209][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:14,210][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:19,211][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:24,212][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:29,215][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:34,216][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:39,217][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:44,218][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:49,345][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:54,346][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:19:59,348][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:04,350][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:09,350][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:14,351][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:19,352][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:24,353][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:29,354][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:34,355][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:39,356][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:44,548][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:49,552][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:54,554][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:20:59,555][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:04,556][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:09,558][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:14,559][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:19,561][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:24,570][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:29,572][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:34,573][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:39,599][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:44,601][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:49,604][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:54,605][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:21:59,606][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:04,607][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:09,608][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:14,609][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:19,610][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:24,611][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:29,612][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:34,659][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:39,660][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:44,661][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:49,662][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:54,663][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:22:59,664][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:04,664][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:09,665][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:14,666][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:20,337][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:25,357][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:30,357][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:35,358][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:38,381][ ERROR][PID:3203363] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 20:23:40,359][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:45,360][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:48,382][WARNING][PID:3203363] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-02-10 20:23:50,362][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:55,363][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:23:58,383][  INFO][PID:3203363] Downloading the builder-live.log file, attempt 2
[2026-02-10 20:23:58,384][  INFO][PID:3203363] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c110 copr-rpmbuild-log
[2026-02-10 20:24:00,370][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:05,371][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:10,372][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:15,373][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:20,374][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:25,386][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:30,395][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:35,397][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:40,398][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:45,399][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:50,402][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:55,446][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:24:58,453][ ERROR][PID:3203363] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 20:25:00,448][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:05,452][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:08,552][WARNING][PID:3203363] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-02-10 20:25:10,453][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:15,454][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:18,553][  INFO][PID:3203363] Downloading the builder-live.log file, attempt 3
[2026-02-10 20:25:18,555][  INFO][PID:3203363] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c110 copr-rpmbuild-log
[2026-02-10 20:25:20,459][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:25,462][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:30,463][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:35,470][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:40,471][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:45,472][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:50,473][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:25:55,478][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:00,585][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:05,627][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:10,635][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:15,726][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:18,615][ ERROR][PID:3203363] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 20:26:20,984][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:26,277][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:28,624][WARNING][PID:3203363] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-02-10 20:26:31,367][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:36,484][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:38,625][  INFO][PID:3203363] Downloading the builder-live.log file, attempt 4
[2026-02-10 20:26:38,627][  INFO][PID:3203363] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c110 copr-rpmbuild-log
[2026-02-10 20:26:41,486][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:46,601][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:51,603][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:26:56,607][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:01,608][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:06,611][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:11,617][  INFO][PID:3203363] Periodic builder liveness probe: alive
[2026-02-10 20:27:11,618][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:16,619][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:21,648][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:26,662][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:31,663][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:36,694][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:38,696][ ERROR][PID:3203363] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 20:27:41,701][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:46,755][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:48,712][ ERROR][PID:3203363] Giving up for unstable SSH, failures: 1770755028.3827176, 1770755108.5528173, 1770755188.624686, 1770755268.712388
[2026-02-10 20:27:48,713][ ERROR][PID:3203363] Re-try request for task on 'ResallocHost, ticket_id=1263292, hostname=2620:52:6:1161:dead:beef:cafe:c110, name=vmhost_p09_02_prod_02150777_20260210_201238, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-02-10 20:27:48,714][  INFO][PID:3203363] Releasing VM back to pool
[2026-02-10 20:27:48,725][  INFO][PID:3203363] Retry #1 (on other host)
[2026-02-10 20:27:48,726][  INFO][PID:3203363] Checking that /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/repodata/repomd.xml exists
[2026-02-10 20:27:48,835][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:27:48,838][  INFO][PID:3203363] VM allocation process starts
[2026-02-10 20:27:48,865][  INFO][PID:3203363] Trying to allocate VM: ResallocHost, ticket_id=1263498, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2026-02-10 20:27:51,880][  INFO][PID:3203363] Allocated host ResallocHost, ticket_id=1263498, hostname=2620:52:6:1161:dead:beef:cafe:c113, name=vmhost_p09_02_prod_02150753_20260210_201008, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2026-02-10 20:27:51,892][  INFO][PID:3203363] Allocating ssh connection to builder
[2026-02-10 20:27:51,894][  INFO][PID:3203363] Checking that builder machine is OK
[2026-02-10 20:28:51,968][ ERROR][PID:3203363] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c113 port 22: Connection timed out

[2026-02-10 20:29:02,171][ ERROR][PID:3203363] Re-try request for task on 'ResallocHost, ticket_id=1263498, hostname=2620:52:6:1161:dead:beef:cafe:c113, name=vmhost_p09_02_prod_02150753_20260210_201008, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts
[2026-02-10 20:29:02,176][  INFO][PID:3203363] Releasing VM back to pool
[2026-02-10 20:29:02,238][  INFO][PID:3203363] Retry #2 (on other host)
[2026-02-10 20:29:02,244][  INFO][PID:3203363] Checking that /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/repodata/repomd.xml exists
[2026-02-10 20:29:02,246][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:02,250][  INFO][PID:3203363] VM allocation process starts
[2026-02-10 20:29:02,343][  INFO][PID:3203363] Trying to allocate VM: ResallocHost, ticket_id=1263537, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2026-02-10 20:29:05,416][  INFO][PID:3203363] Allocated host ResallocHost, ticket_id=1263537, hostname=2620:52:6:1161:dead:beef:cafe:c189, name=vmhost_p09_04_prod_02150823_20260210_201803, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2026-02-10 20:29:05,416][  INFO][PID:3203363] Allocating ssh connection to builder
[2026-02-10 20:29:05,417][  INFO][PID:3203363] Checking that builder machine is OK
[2026-02-10 20:29:06,498][  INFO][PID:3203363] Running remote command: copr-builder-ready centos-stream+epel-next-9-ppc64le
[2026-02-10 20:29:07,113][  INFO][PID:3203363] Red Hat subscription not needed for centos-stream+epel-next-9-ppc64le
Builder is ready to be used

[2026-02-10 20:29:07,113][  INFO][PID:3203363] Filling build.info file with builder info
[2026-02-10 20:29:07,229][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:07,231][  INFO][PID:3203363] Not re-notifying FE and msg buses for the new host.
[2026-02-10 20:29:07,232][  INFO][PID:3203363] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10114070-centos-stream+epel-next-9-ppc64le --chroot centos-stream+epel-next-9-ppc64le --detached
[2026-02-10 20:29:08,358][  INFO][PID:3203363] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c189' (ED25519) to the list of known hosts.


[2026-02-10 20:29:08,364][  INFO][PID:3203363] Downloading the builder-live.log file, attempt 1
[2026-02-10 20:29:08,375][  INFO][PID:3203363] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c189 copr-rpmbuild-log
[2026-02-10 20:29:13,369][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:18,373][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:23,390][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:28,409][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:33,416][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:38,419][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:43,467][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:48,473][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:53,484][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:29:58,486][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:03,490][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:08,494][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:13,498][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:18,506][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:23,508][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:28,512][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:33,516][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:38,519][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:43,549][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:48,551][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:53,552][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:30:58,584][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:03,587][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:08,655][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:13,657][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:18,658][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:23,660][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:28,661][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:33,662][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:38,681][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:43,683][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:48,689][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:53,723][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:31:58,746][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:03,776][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:08,779][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:13,801][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:18,804][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:23,821][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:28,824][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:33,825][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:38,826][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:43,828][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:48,829][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:53,830][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:32:58,831][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:03,832][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:08,834][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:13,839][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:18,866][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:23,867][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:28,868][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:33,869][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:38,870][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:43,871][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:48,873][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:53,874][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:33:58,888][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:03,890][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:08,963][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:13,964][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:18,965][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:23,966][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:28,967][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:33,968][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:38,969][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:44,008][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:49,009][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:54,010][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:34:59,011][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:04,012][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:09,068][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:14,069][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:19,070][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:24,071][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:29,072][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:34,073][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:39,074][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:44,075][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:49,076][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:54,186][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:35:59,187][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:04,243][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:09,244][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:14,245][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:19,246][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:24,247][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:29,248][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:34,250][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:39,258][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:44,260][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:49,261][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:54,262][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:36:59,263][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:04,281][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:09,294][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:14,301][  INFO][PID:3203363] Periodic builder liveness probe: alive
[2026-02-10 20:37:14,302][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:19,318][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:24,318][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:29,364][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:34,381][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:39,382][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:44,383][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:49,384][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:54,385][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:37:59,386][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:04,387][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:09,388][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:14,425][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:19,426][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:24,441][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:29,442][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:34,451][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:39,452][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:44,459][  INFO][PID:3203363] Checking for cancel request
[2026-02-10 20:38:48,386][  INFO][PID:3203363] Downloading results from builder
[2026-02-10 20:38:48,387][  INFO][PID:3203363] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c189]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn started
[2026-02-10 20:38:48,387][  INFO][PID:3203363] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c189]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/ &> /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/build-10114070.rsync.log
[2026-02-10 20:39:48,587][  INFO][PID:3203363] rsyncing finished.
[2026-02-10 20:39:48,599][ ERROR][PID:3203363] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 20:39:48,599][ ERROR][PID:3203363] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 20:39:58,600][  INFO][PID:3203363] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c189]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn started
[2026-02-10 20:39:58,626][  INFO][PID:3203363] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c189]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/ &> /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/build-10114070.rsync.log
[2026-02-10 20:40:58,716][  INFO][PID:3203363] rsyncing finished.
[2026-02-10 20:40:58,717][ ERROR][PID:3203363] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 20:40:58,717][ ERROR][PID:3203363] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 20:41:08,717][  INFO][PID:3203363] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c189]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn started
[2026-02-10 20:41:08,719][  INFO][PID:3203363] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c189]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/ &> /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/build-10114070.rsync.log
[2026-02-10 20:42:08,804][  INFO][PID:3203363] rsyncing finished.
[2026-02-10 20:42:08,830][ ERROR][PID:3203363] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 20:42:08,830][ ERROR][PID:3203363] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 20:42:18,831][ ERROR][PID:3203363] Re-try request for task on 'ResallocHost, ticket_id=1263537, hostname=2620:52:6:1161:dead:beef:cafe:c189, name=vmhost_p09_04_prod_02150823_20260210_201803, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 3 SSH attempts
[2026-02-10 20:42:18,831][  INFO][PID:3203363] Releasing VM back to pool
[2026-02-10 20:42:18,840][ ERROR][PID:3203363] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c110', '2620:52:6:1161:dead:beef:cafe:c189', '2620:52:6:1161:dead:beef:cafe:c113'}
[2026-02-10 20:42:18,840][WARNING][PID:3203363] Switching not-finished job state to 'failed'
[2026-02-10 20:42:18,841][  INFO][PID:3203363] Worker failed build, took 791.6093716621399
[2026-02-10 20:42:18,841][  INFO][PID:3203363] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@meta",
            "project_name": "drgn",
            "project_dirname": "drgn",
            "submitter": "packit",
            "ended_on": 1770756138.8407748,
            "started_on": 1770755347.231403,
            "submitted_on": null,
            "status": 0,
            "chroot": "centos-stream+epel-next-9-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10114070-centos-stream+epel-next-9-ppc64le",
            "build_id": 10114070,
            "package_name": "python-drgn",
            "package_version": "0.0.33+193.gf84a0beb-1.20260210201338848497.main.193.gf84a0beb",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@meta/drgn/python-drgn",
            "git_hash": "805a1678c2d4830b30cdb2d52c6a50eee3c8e3a2",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@meta/drgn--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://kojihub.stream.centos.org/kojifiles/repos/c9s-build/latest/ppc64le/",
                    "id": "https_kojihub_stream_centos_org_kojifiles_repos_c9s_build_latest_ppc64le",
                    "name": "Additional repo https_kojihub_stream_centos_org_kojifiles_repos_c9s_build_latest_ppc64le"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@meta/drgn",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@meta/drgn",
            "result_dir": "10114070-python-drgn",
            "built_packages": "",
            "id": 10114070,
            "mockchain_macros": {
                "copr_username": "@meta",
                "copr_projectname": "drgn",
                "vendor": "Fedora Project COPR (@meta/drgn)"
            }
        }
    ]
}
[2026-02-10 20:42:18,921][  INFO][PID:3203363] Sending fedora-messaging bus message in build.end
[2026-02-10 20:42:18,975][  INFO][PID:3203363] Compressing /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/builder-live.log by gzip
[2026-02-10 20:42:18,976][  INFO][PID:3203363] Running command 'gzip /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/builder-live.log' as PID 3348882
[2026-02-10 20:42:19,274][  INFO][PID:3203363] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/builder-live.log)
[2026-02-10 20:42:19,275][  INFO][PID:3203363] Compressing /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/backend.log by gzip
[2026-02-10 20:42:19,276][  INFO][PID:3203363] Running command 'gzip /var/lib/copr/public_html/results/@meta/drgn/centos-stream+epel-next-9-ppc64le/10114070-python-drgn/backend.log' as PID 3348934