[2025-12-09 23:44:13,001][  INFO][PID:1735704] Marking build as starting
[2025-12-09 23:44:13,055][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:13,057][  INFO][PID:1735704] VM allocation process starts
[2025-12-09 23:44:13,130][  INFO][PID:1735704] Trying to allocate VM: ResallocHost, ticket_id=248661, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-09 23:44:16,158][  INFO][PID:1735704] Allocated host ResallocHost, ticket_id=248661, hostname=2620:52:6:1161:dead:beef:cafe:c14b, name=vmhost_p09_03_prod_00292316_20251209_233732, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-09 23:44:16,159][  INFO][PID:1735704] Allocating ssh connection to builder
[2025-12-09 23:44:16,159][  INFO][PID:1735704] Checking that builder machine is OK
[2025-12-09 23:44:17,332][  INFO][PID:1735704] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-12-09 23:44:17,834][  INFO][PID:1735704] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-12-09 23:44:17,835][  INFO][PID:1735704] Filling build.info file with builder info
[2025-12-09 23:44:17,836][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:17,837][  INFO][PID:1735704] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "scipy-1.10.0",
            "project_dirname": "scipy-1.10.0",
            "submitter": "https://src.fedoraproject.org/user/ellert",
            "ended_on": null,
            "started_on": 1765323857.8369925,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9892103-fedora-rawhide-ppc64le",
            "build_id": 9892103,
            "package_name": "root",
            "package_version": "6.38.00-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/scipy-1.10.0/root",
            "git_hash": "5b86e5a7ebe237f7df5c405bbbc58e6683aef9d2",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/scipy-1.10.0--https://src.fedoraproject.org/user/ellert",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/scipy-1.10.0",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.10.0",
            "result_dir": "09892103-root",
            "built_packages": "",
            "id": 9892103,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "scipy-1.10.0",
                "vendor": "Fedora Project COPR (psimovec/scipy-1.10.0)"
            }
        }
    ]
}
[2025-12-09 23:44:17,891][  INFO][PID:1735704] Sending fedora-messaging bus message in build.start
[2025-12-09 23:44:18,434][  INFO][PID:1735704] Sending fedora-messaging bus message in chroot.start
[2025-12-09 23:44:18,462][  INFO][PID:1735704] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9892103-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-12-09 23:44:19,380][  INFO][PID:1735704] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14b' (ED25519) to the list of known hosts.


[2025-12-09 23:44:19,381][  INFO][PID:1735704] Downloading the builder-live.log file, attempt 1
[2025-12-09 23:44:19,383][  INFO][PID:1735704] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14b copr-rpmbuild-log
[2025-12-09 23:44:24,389][  INFO][PID:1735704] Periodic builder liveness probe: alive
[2025-12-09 23:44:24,390][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:29,391][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:34,392][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:39,395][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:44,396][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:49,397][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:54,398][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:44:59,399][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:04,400][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:09,401][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:14,402][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:19,403][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:24,404][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:29,405][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:34,406][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:39,407][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:44,407][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:49,408][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:54,409][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:45:59,410][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:04,411][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:09,412][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:14,413][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:19,414][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:24,415][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:29,417][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:34,418][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:39,419][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:44,420][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:49,421][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:54,422][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:46:59,423][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:04,425][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:09,426][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:14,427][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:19,428][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:24,429][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:29,430][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:34,431][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:39,432][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:44,433][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:49,435][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:54,436][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:47:59,438][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:04,439][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:09,440][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:14,441][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:19,442][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:24,443][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:29,444][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:34,445][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:39,445][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:44,446][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:49,447][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:54,448][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:48:59,449][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:04,450][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:09,451][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:14,452][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:19,453][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:24,454][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:29,455][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:34,456][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:39,457][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:44,458][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:49,459][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:54,460][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:49:59,460][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:04,461][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:09,462][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:14,464][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:19,465][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:24,465][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:29,466][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:34,467][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:39,468][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:44,469][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:49,470][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:54,471][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:50:59,472][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:04,473][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:09,474][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:14,475][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:19,476][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:24,477][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:29,478][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:34,479][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:39,480][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:44,481][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:49,482][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:54,483][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:51:59,484][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:04,485][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:09,486][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:14,487][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:19,488][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:24,489][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:29,490][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:34,491][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:39,492][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:44,493][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:49,494][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:54,495][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:52:59,496][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:04,497][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:09,497][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:14,498][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:19,499][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:24,500][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:29,501][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:34,502][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:39,503][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:44,504][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:49,505][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:54,506][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:53:59,507][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:04,508][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:09,509][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:14,510][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:19,511][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:24,517][  INFO][PID:1735704] Periodic builder liveness probe: alive
[2025-12-09 23:54:24,518][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:29,518][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:34,519][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:39,520][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:44,521][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:49,522][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:54,523][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:54:59,524][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:04,525][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:09,526][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:14,527][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:19,528][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:24,529][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:29,530][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:34,531][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:39,532][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:44,533][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:49,533][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:54,534][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:55:59,535][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:04,536][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:09,537][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:14,538][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:19,539][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:24,540][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:29,541][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:34,542][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:39,543][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:44,543][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:49,544][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:54,545][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:56:59,546][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:04,547][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:09,548][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:14,549][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:19,550][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:24,551][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:29,552][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:34,553][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:39,554][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:44,555][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:49,556][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:54,556][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:57:59,558][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:04,558][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:09,559][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:14,560][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:19,561][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:24,562][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:29,563][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:34,564][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:39,565][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:44,566][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:49,567][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:54,568][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:58:59,569][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:04,570][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:09,571][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:14,572][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:19,573][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:24,574][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:29,575][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:34,576][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:39,576][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:44,577][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:49,578][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:54,579][  INFO][PID:1735704] Checking for cancel request
[2025-12-09 23:59:59,580][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:04,581][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:09,582][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:14,583][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:19,584][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:24,585][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:29,586][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:34,587][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:39,588][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:44,589][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:49,590][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:54,590][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:00:59,591][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:04,592][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:09,593][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:14,594][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:19,595][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:24,596][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:29,597][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:34,598][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:39,599][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:44,600][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:49,601][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:54,602][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:01:59,603][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:04,604][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:09,605][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:14,606][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:19,607][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:24,607][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:29,608][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:34,609][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:39,610][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:44,611][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:49,612][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:54,613][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:02:59,614][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:04,615][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:09,616][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:14,617][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:19,618][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:24,619][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:29,620][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:34,621][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:39,621][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:44,622][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:49,623][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:54,624][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:03:59,625][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:04,626][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:09,627][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:14,628][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:19,629][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:24,635][  INFO][PID:1735704] Periodic builder liveness probe: alive
[2025-12-10 00:04:24,635][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:29,636][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:34,637][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:39,638][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:44,638][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:49,639][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:54,640][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:04:59,641][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:04,642][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:09,643][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:14,644][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:19,645][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:24,646][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:29,647][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:34,648][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:39,649][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:44,650][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:49,651][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:54,654][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:05:59,655][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:04,656][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:09,657][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:14,658][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:19,659][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:24,660][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:29,661][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:34,662][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:39,663][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:44,664][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:49,665][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:54,666][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:06:59,671][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:04,674][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:09,675][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:14,676][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:19,677][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:24,678][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:29,679][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:34,680][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:39,681][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:44,682][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:49,683][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:54,684][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:07:59,685][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:04,686][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:09,687][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:14,688][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:19,689][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:24,691][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:29,692][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:34,693][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:39,693][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:44,696][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:49,697][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:54,699][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:08:59,699][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:04,700][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:09,702][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:14,703][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:19,704][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:24,706][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:29,707][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:34,708][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:39,709][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:44,710][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:49,712][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:54,713][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:09:59,714][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:04,715][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:09,715][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:14,717][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:19,717][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:24,718][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:29,720][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:34,721][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:39,722][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:44,723][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:49,724][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:54,725][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:10:59,726][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:04,726][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:09,727][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:14,728][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:19,729][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:24,730][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:29,732][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:34,732][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:39,733][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:44,734][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:49,735][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:54,736][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:11:59,737][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:04,738][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:09,739][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:14,740][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:19,741][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:24,742][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:29,743][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:34,744][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:39,745][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:44,746][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:49,747][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:54,748][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:12:59,749][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:04,750][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:09,751][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:14,752][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:19,753][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:24,754][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:29,754][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:34,755][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:39,756][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:44,757][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:49,758][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:54,759][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:13:59,760][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:14:04,761][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:14:09,762][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:14:13,503][ ERROR][PID:1735704] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-10 00:14:14,763][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:14:19,764][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:14:23,504][WARNING][PID:1735704] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-10 00:14:24,773][  INFO][PID:1735704] Periodic builder liveness probe: dead
[2025-12-10 00:14:24,774][  INFO][PID:1735704] calling cancel callback, and waiting
[2025-12-10 00:14:26,109][WARNING][PID:1735704] Can't cancel build
out:

err:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14b' (ED25519) to the list of known hosts.
Traceback (most recent call last):
  File "/usr/bin/copr-rpmbuild-cancel", line 85, in <module>
    _main()
    ~~~~~^^
  File "/usr/bin/copr-rpmbuild-cancel", line 69, in _main
    with open(pidfile, "r") as pidfd:
         ~~~~^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/copr-rpmbuild/pid'

[2025-12-10 00:14:33,505][  INFO][PID:1735704] Downloading the builder-live.log file, attempt 2
[2025-12-10 00:14:33,518][  INFO][PID:1735704] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14b copr-rpmbuild-log
[2025-12-10 00:16:34,268][ ERROR][PID:1735704] Re-try request for task on 'ResallocHost, ticket_id=248661, hostname=2620:52:6:1161:dead:beef:cafe:c14b, name=vmhost_p09_03_prod_00292316_20251209_233732, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-12-10 00:16:34,274][  INFO][PID:1735704] Releasing VM back to pool
[2025-12-10 00:16:34,346][  INFO][PID:1735704] Retry #1 (on other host)
[2025-12-10 00:16:34,357][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:16:34,359][  INFO][PID:1735704] VM allocation process starts
[2025-12-10 00:16:34,393][  INFO][PID:1735704] Trying to allocate VM: ResallocHost, ticket_id=248895, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-10 00:16:37,492][  INFO][PID:1735704] Allocated host ResallocHost, ticket_id=248895, hostname=2620:52:6:1161:dead:beef:cafe:c10d, name=vmhost_p09_02_prod_00292542_20251209_235154, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-10 00:16:37,493][  INFO][PID:1735704] Allocating ssh connection to builder
[2025-12-10 00:16:37,497][  INFO][PID:1735704] Checking that builder machine is OK
[2025-12-10 00:16:38,604][  INFO][PID:1735704] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-12-10 00:16:39,111][  INFO][PID:1735704] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-12-10 00:16:39,113][  INFO][PID:1735704] Filling build.info file with builder info
[2025-12-10 00:16:39,122][  INFO][PID:1735704] Checking for cancel request
[2025-12-10 00:16:39,127][  INFO][PID:1735704] Not re-notifying FE and msg buses for the new host.
[2025-12-10 00:16:39,133][  INFO][PID:1735704] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9892103-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-12-10 00:16:40,185][  INFO][PID:1735704] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c10d' (ED25519) to the list of known hosts.


[2025-12-10 00:16:40,189][  INFO][PID:1735704] Downloading the builder-live.log file, attempt 1
[2025-12-10 00:16:40,191][  INFO][PID:1735704] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10d copr-rpmbuild-log
[2025-12-10 00:16:45,190][  INFO][PID:1735704] calling cancel callback, and waiting
[2025-12-10 00:16:48,863][  INFO][PID:1735704] Cancel request succeeded
out:
success
err:
Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c10d' (ED25519) to the list of known hosts.

[2025-12-10 00:16:48,865][  INFO][PID:1735704] Downloading results from builder
[2025-12-10 00:16:48,874][  INFO][PID:1735704] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10d]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/09892103-root started
[2025-12-10 00:16:48,879][  INFO][PID:1735704] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10d]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/09892103-root/ &> /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/09892103-root/build-09892103.rsync.log
[2025-12-10 00:16:49,376][  INFO][PID:1735704] rsyncing finished.
[2025-12-10 00:16:49,378][  INFO][PID:1735704] Releasing VM back to pool
[2025-12-10 00:16:49,388][  INFO][PID:1735704] Searching for 'success' file in resultdir
[2025-12-10 00:16:49,388][ ERROR][PID:1735704] Build failed: Backend process error: No success file => build failure
[2025-12-10 00:16:49,463][  INFO][PID:1735704] Finished build: id=9892103 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/scipy-1.10.0 chroot=fedora-rawhide-ppc64le 
[2025-12-10 00:16:49,521][ ERROR][PID:1735704] Backend process error: No success file => build failure
[2025-12-10 00:16:49,523][  INFO][PID:1735704] Worker failed build, took 10.399438858032227
[2025-12-10 00:16:49,530][  INFO][PID:1735704] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "scipy-1.10.0",
            "project_dirname": "scipy-1.10.0",
            "submitter": "https://src.fedoraproject.org/user/ellert",
            "ended_on": 1765325809.523096,
            "started_on": 1765325799.1236572,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9892103-fedora-rawhide-ppc64le",
            "build_id": 9892103,
            "package_name": "root",
            "package_version": "6.38.00-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/scipy-1.10.0/root",
            "git_hash": "5b86e5a7ebe237f7df5c405bbbc58e6683aef9d2",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/scipy-1.10.0--https://src.fedoraproject.org/user/ellert",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/scipy-1.10.0",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.10.0",
            "result_dir": "09892103-root",
            "built_packages": "",
            "id": 9892103,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "scipy-1.10.0",
                "vendor": "Fedora Project COPR (psimovec/scipy-1.10.0)"
            }
        }
    ]
}
[2025-12-10 00:16:50,219][  INFO][PID:1735704] Sending fedora-messaging bus message in build.end
[2025-12-10 00:16:50,553][  INFO][PID:1735704] Compressing /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/09892103-root/builder-live.log by gzip
[2025-12-10 00:16:50,556][  INFO][PID:1735704] Running command 'gzip /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/09892103-root/builder-live.log' as PID 1871915
[2025-12-10 00:16:50,570][  INFO][PID:1735704] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/09892103-root/builder-live.log)
[2025-12-10 00:16:50,575][  INFO][PID:1735704] Compressing /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/09892103-root/backend.log by gzip
[2025-12-10 00:16:50,576][  INFO][PID:1735704] Running command 'gzip /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-rawhide-ppc64le/09892103-root/backend.log' as PID 1871918