[2025-12-29 02:56:41,018][  INFO][PID:1211389] Marking build as starting
[2025-12-29 02:56:41,143][  INFO][PID:1211389] Checking that /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/repodata/repomd.xml exists
[2025-12-29 02:56:41,169][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:56:41,172][  INFO][PID:1211389] VM allocation process starts
[2025-12-29 02:56:41,206][  INFO][PID:1211389] Trying to allocate VM: ResallocHost, ticket_id=551598, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-29 02:56:46,212][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:56:51,213][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:56:56,214][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:01,216][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:06,218][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:11,219][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:16,222][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:21,549][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:26,550][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:31,551][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:36,552][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:41,553][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:46,556][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:51,558][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:57:56,559][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:01,560][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:06,561][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:11,562][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:16,563][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:21,564][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:26,565][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:31,567][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:36,568][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:41,570][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:46,571][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:51,572][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:58:56,574][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:01,576][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:06,577][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:11,578][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:16,580][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:21,581][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:26,582][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:31,583][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:36,584][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:41,585][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:46,586][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:51,779][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 02:59:56,780][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:01,781][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:06,782][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:11,783][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:16,784][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:21,785][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:26,786][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:31,787][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:36,788][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:41,789][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:46,790][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:51,791][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:00:56,792][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:01,793][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:06,794][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:11,795][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:16,796][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:21,797][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:26,798][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:31,799][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:36,800][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:41,801][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:46,802][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:51,803][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:56,804][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:01:59,352][  INFO][PID:1211389] Allocated host ResallocHost, ticket_id=551598, hostname=2620:52:6:1161:dead:beef:cafe:c196, name=vmhost_p09_04_prod_00804063_20251229_025647, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-29 03:01:59,353][  INFO][PID:1211389] Allocating ssh connection to builder
[2025-12-29 03:01:59,353][  INFO][PID:1211389] Checking that builder machine is OK
[2025-12-29 03:02:00,419][  INFO][PID:1211389] Running remote command: copr-builder-ready rhel+epel-10-ppc64le
[2025-12-29 03:02:00,988][  INFO][PID:1211389] Red Hat subscription not needed for rhel+epel-10-ppc64le
Builder is ready to be used

[2025-12-29 03:02:00,988][  INFO][PID:1211389] Filling build.info file with builder info
[2025-12-29 03:02:00,989][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:00,990][  INFO][PID:1211389] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "ligenix",
            "project_name": "enterprise-cosmic-testing",
            "project_dirname": "enterprise-cosmic-testing",
            "submitter": "ligenix",
            "ended_on": null,
            "started_on": 1766977320.9900212,
            "submitted_on": null,
            "status": 3,
            "chroot": "rhel+epel-10-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9957098-rhel+epel-10-ppc64le",
            "build_id": 9957098,
            "package_name": "cosmic-comp",
            "package_version": "1.0.0-1.20251223",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ligenix/enterprise-cosmic-testing/cosmic-comp",
            "git_hash": "12b291ecc2044c70fd0c5bfe82084107be52b708",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "ligenix/enterprise-cosmic-testing--ligenix",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic/rhel+epel-10-ppc64le/",
                    "id": "copr_ligenix_enterprise_cosmic_rhel_epel_releasever_basearch",
                    "name": "Additional repo copr_ligenix_enterprise_cosmic_rhel_epel_releasever_basearch"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic-testing",
            "result_dir": "09957098-cosmic-comp",
            "built_packages": "",
            "id": 9957098,
            "mockchain_macros": {
                "copr_username": "ligenix",
                "copr_projectname": "enterprise-cosmic-testing",
                "vendor": "Fedora Project COPR (ligenix/enterprise-cosmic-testing)"
            }
        }
    ]
}
[2025-12-29 03:02:01,208][  INFO][PID:1211389] Sending fedora-messaging bus message in build.start
[2025-12-29 03:02:01,849][  INFO][PID:1211389] Sending fedora-messaging bus message in chroot.start
[2025-12-29 03:02:01,879][  INFO][PID:1211389] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9957098-rhel+epel-10-ppc64le --chroot rhel+epel-10-ppc64le --detached
[2025-12-29 03:02:02,914][  INFO][PID:1211389] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c196' (ED25519) to the list of known hosts.


[2025-12-29 03:02:02,914][  INFO][PID:1211389] Downloading the builder-live.log file, attempt 1
[2025-12-29 03:02:02,916][  INFO][PID:1211389] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c196 copr-rpmbuild-log
[2025-12-29 03:02:07,919][  INFO][PID:1211389] Periodic builder liveness probe: alive
[2025-12-29 03:02:07,920][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:12,921][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:17,922][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:22,923][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:27,924][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:32,925][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:37,926][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:42,927][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:47,928][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:52,929][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:02:57,931][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:02,932][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:07,933][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:12,934][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:17,935][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:22,936][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:27,937][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:32,938][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:37,939][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:42,940][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:47,941][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:52,942][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:03:57,943][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:02,944][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:07,945][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:12,946][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:17,947][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:22,948][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:27,949][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:32,950][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:37,951][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:42,952][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:47,953][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:52,955][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:04:57,956][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:02,958][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:07,959][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:12,960][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:17,961][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:22,962][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:27,963][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:32,981][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:37,983][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:42,984][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:47,985][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:52,986][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:05:57,987][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:02,988][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:07,989][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:12,990][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:17,991][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:23,024][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:28,025][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:33,027][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:38,028][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:43,029][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:48,031][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:53,032][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:06:58,034][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:03,041][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:08,042][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:13,043][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:18,044][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:23,045][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:28,046][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:33,047][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:38,048][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:43,182][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:48,183][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:53,184][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:07:58,185][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:03,186][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:08,187][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:13,188][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:18,189][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:23,190][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:28,195][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:33,197][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:38,198][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:43,199][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:48,200][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:53,201][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:08:58,202][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:03,204][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:08,207][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:13,210][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:18,211][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:23,213][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:28,214][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:33,216][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:38,218][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:43,223][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:48,224][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:53,225][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:09:58,226][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:03,227][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:08,228][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:13,229][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:18,232][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:23,233][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:28,236][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:33,237][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:38,240][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:43,243][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:48,243][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:53,245][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:10:58,247][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:03,249][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:08,251][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:13,253][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:18,254][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:23,258][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:28,259][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:33,260][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:38,284][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:43,287][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:48,289][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:53,291][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:11:58,297][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:03,299][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:08,315][  INFO][PID:1211389] Periodic builder liveness probe: alive
[2025-12-29 03:12:08,316][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:13,317][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:18,321][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:23,323][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:28,324][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:33,326][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:38,326][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:43,327][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:48,329][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:53,330][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:12:58,364][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:03,365][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:08,372][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:13,376][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:18,377][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:23,378][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:28,380][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:33,415][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:38,417][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:43,424][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:48,426][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:53,428][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:13:58,430][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:03,431][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:08,532][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:13,533][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:18,535][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:23,538][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:28,542][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:33,544][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:38,545][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:43,547][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:48,548][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:53,552][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:14:58,553][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:03,602][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:08,604][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:13,606][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:18,607][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:23,608][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:28,676][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:33,681][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:38,682][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:43,683][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:48,698][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:53,700][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:15:58,701][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:03,841][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:08,878][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:13,879][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:18,881][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:23,883][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:28,884][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:33,885][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:38,888][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:43,889][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:48,890][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:53,891][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:16:59,002][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:04,666][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:09,667][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:14,668][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:19,669][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:24,670][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:29,671][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:34,672][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:39,674][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:44,688][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:49,689][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:54,703][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:17:59,704][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:04,705][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:09,706][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:14,707][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:19,708][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:24,709][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:29,710][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:34,711][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:39,712][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:44,713][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:49,714][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:54,715][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:18:59,716][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:04,717][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:09,718][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:14,719][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:19,720][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:24,721][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:29,722][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:34,723][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:39,723][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:44,724][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:49,725][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:54,726][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:19:59,727][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:04,728][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:09,729][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:14,730][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:19,731][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:24,731][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:29,843][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:34,844][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:39,845][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:43,374][ ERROR][PID:1211389] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-29 03:20:44,846][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:50,048][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:20:53,375][WARNING][PID:1211389] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-29 03:20:55,049][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:00,050][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:03,378][  INFO][PID:1211389] Downloading the builder-live.log file, attempt 2
[2025-12-29 03:21:03,380][  INFO][PID:1211389] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c196 copr-rpmbuild-log
[2025-12-29 03:21:05,052][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:10,053][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:15,054][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:20,055][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:25,056][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:30,057][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:35,059][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:40,060][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:45,061][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:50,063][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:21:55,069][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:22:00,082][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:22:03,446][ ERROR][PID:1211389] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-29 03:22:05,083][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:22:10,088][  INFO][PID:1211389] Periodic builder liveness probe: dead
[2025-12-29 03:22:10,089][  INFO][PID:1211389] calling cancel callback, and waiting
[2025-12-29 03:22:13,456][WARNING][PID:1211389] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-29 03:22:23,457][  INFO][PID:1211389] Downloading the builder-live.log file, attempt 3
[2025-12-29 03:22:23,458][  INFO][PID:1211389] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c196 copr-rpmbuild-log
[2025-12-29 03:22:46,880][WARNING][PID:1211389] Can't cancel build
out:

err:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c196' (ED25519) to the list of known hosts.
Traceback (most recent call last):
  File "/usr/bin/copr-rpmbuild-cancel", line 85, in <module>
    _main()
    ~~~~~^^
  File "/usr/bin/copr-rpmbuild-cancel", line 69, in _main
    with open(pidfile, "r") as pidfd:
         ~~~~^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/copr-rpmbuild/pid'

[2025-12-29 03:24:36,298][ ERROR][PID:1211389] Re-try request for task on 'ResallocHost, ticket_id=551598, hostname=2620:52:6:1161:dead:beef:cafe:c196, name=vmhost_p09_04_prod_00804063_20251229_025647, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-12-29 03:24:36,298][  INFO][PID:1211389] Releasing VM back to pool
[2025-12-29 03:24:36,308][  INFO][PID:1211389] Retry #1 (on other host)
[2025-12-29 03:24:36,308][  INFO][PID:1211389] Checking that /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/repodata/repomd.xml exists
[2025-12-29 03:24:36,309][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:24:36,309][  INFO][PID:1211389] VM allocation process starts
[2025-12-29 03:24:36,318][  INFO][PID:1211389] Trying to allocate VM: ResallocHost, ticket_id=552015, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-29 03:24:39,331][  INFO][PID:1211389] Allocated host ResallocHost, ticket_id=552015, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_00804281_20251229_031553, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-29 03:24:39,331][  INFO][PID:1211389] Allocating ssh connection to builder
[2025-12-29 03:24:39,332][  INFO][PID:1211389] Checking that builder machine is OK
[2025-12-29 03:25:39,400][ ERROR][PID:1211389] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c14d port 22: Connection timed out

[2025-12-29 03:25:49,401][ ERROR][PID:1211389] Re-try request for task on 'ResallocHost, ticket_id=552015, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_00804281_20251229_031553, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts
[2025-12-29 03:25:49,401][  INFO][PID:1211389] Releasing VM back to pool
[2025-12-29 03:25:49,413][  INFO][PID:1211389] Retry #2 (on other host)
[2025-12-29 03:25:49,413][  INFO][PID:1211389] Checking that /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/repodata/repomd.xml exists
[2025-12-29 03:25:49,414][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:25:49,414][  INFO][PID:1211389] VM allocation process starts
[2025-12-29 03:25:49,426][  INFO][PID:1211389] Trying to allocate VM: ResallocHost, ticket_id=552028, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-29 03:25:52,473][  INFO][PID:1211389] Allocated host ResallocHost, ticket_id=552028, hostname=2620:52:6:1161:dead:beef:cafe:c18c, name=vmhost_p09_04_prod_00804293_20251229_031718, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-29 03:25:52,474][  INFO][PID:1211389] Allocating ssh connection to builder
[2025-12-29 03:25:52,474][  INFO][PID:1211389] Checking that builder machine is OK
[2025-12-29 03:25:53,548][  INFO][PID:1211389] Running remote command: copr-builder-ready rhel+epel-10-ppc64le
[2025-12-29 03:25:54,063][  INFO][PID:1211389] Red Hat subscription not needed for rhel+epel-10-ppc64le
Builder is ready to be used

[2025-12-29 03:25:54,063][  INFO][PID:1211389] Filling build.info file with builder info
[2025-12-29 03:25:54,123][  INFO][PID:1211389] Checking for cancel request
[2025-12-29 03:25:54,124][  INFO][PID:1211389] Not re-notifying FE and msg buses for the new host.
[2025-12-29 03:25:54,125][  INFO][PID:1211389] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9957098-rhel+epel-10-ppc64le --chroot rhel+epel-10-ppc64le --detached
[2025-12-29 03:25:55,064][  INFO][PID:1211389] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18c' (ED25519) to the list of known hosts.


[2025-12-29 03:25:55,065][  INFO][PID:1211389] Downloading the builder-live.log file, attempt 1
[2025-12-29 03:25:55,067][  INFO][PID:1211389] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log
[2025-12-29 03:26:00,066][  INFO][PID:1211389] calling cancel callback, and waiting
[2025-12-29 03:26:03,739][  INFO][PID:1211389] Cancel request succeeded
out:
success
err:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18c' (ED25519) to the list of known hosts.

[2025-12-29 03:26:03,740][  INFO][PID:1211389] Downloading results from builder
[2025-12-29 03:26:03,740][  INFO][PID:1211389] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18c]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/09957098-cosmic-comp started
[2025-12-29 03:26:03,742][  INFO][PID:1211389] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18c]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/09957098-cosmic-comp/ &> /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/09957098-cosmic-comp/build-09957098.rsync.log
[2025-12-29 03:26:04,319][  INFO][PID:1211389] rsyncing finished.
[2025-12-29 03:26:04,320][  INFO][PID:1211389] Releasing VM back to pool
[2025-12-29 03:26:04,330][  INFO][PID:1211389] Searching for 'success' file in resultdir
[2025-12-29 03:26:04,330][ ERROR][PID:1211389] Build failed: Backend process error: No success file => build failure
[2025-12-29 03:26:04,331][  INFO][PID:1211389] Finished build: id=9957098 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing chroot=rhel+epel-10-ppc64le 
[2025-12-29 03:26:04,333][ ERROR][PID:1211389] Backend process error: No success file => build failure
[2025-12-29 03:26:04,333][  INFO][PID:1211389] Worker failed build, took 10.208987712860107
[2025-12-29 03:26:04,334][  INFO][PID:1211389] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "ligenix",
            "project_name": "enterprise-cosmic-testing",
            "project_dirname": "enterprise-cosmic-testing",
            "submitter": "ligenix",
            "ended_on": 1766978764.333744,
            "started_on": 1766978754.1247563,
            "submitted_on": null,
            "status": 0,
            "chroot": "rhel+epel-10-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9957098-rhel+epel-10-ppc64le",
            "build_id": 9957098,
            "package_name": "cosmic-comp",
            "package_version": "1.0.0-1.20251223",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/ligenix/enterprise-cosmic-testing/cosmic-comp",
            "git_hash": "12b291ecc2044c70fd0c5bfe82084107be52b708",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "ligenix/enterprise-cosmic-testing--ligenix",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic/rhel+epel-10-ppc64le/",
                    "id": "copr_ligenix_enterprise_cosmic_rhel_epel_releasever_basearch",
                    "name": "Additional repo copr_ligenix_enterprise_cosmic_rhel_epel_releasever_basearch"
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/ligenix/enterprise-cosmic-testing",
            "result_dir": "09957098-cosmic-comp",
            "built_packages": "",
            "id": 9957098,
            "mockchain_macros": {
                "copr_username": "ligenix",
                "copr_projectname": "enterprise-cosmic-testing",
                "vendor": "Fedora Project COPR (ligenix/enterprise-cosmic-testing)"
            }
        }
    ]
}
[2025-12-29 03:26:04,407][  INFO][PID:1211389] Sending fedora-messaging bus message in build.end
[2025-12-29 03:26:04,437][  INFO][PID:1211389] Compressing /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/09957098-cosmic-comp/builder-live.log by gzip
[2025-12-29 03:26:04,438][  INFO][PID:1211389] Running command 'gzip /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/09957098-cosmic-comp/builder-live.log' as PID 1365223
[2025-12-29 03:26:04,448][  INFO][PID:1211389] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/09957098-cosmic-comp/builder-live.log)
[2025-12-29 03:26:04,449][  INFO][PID:1211389] Compressing /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/09957098-cosmic-comp/backend.log by gzip
[2025-12-29 03:26:04,451][  INFO][PID:1211389] Running command 'gzip /var/lib/copr/public_html/results/ligenix/enterprise-cosmic-testing/rhel+epel-10-ppc64le/09957098-cosmic-comp/backend.log' as PID 1365227