[2025-09-09 14:44:13,279][  INFO][PID:3392862] Marking build as starting
[2025-09-09 14:44:13,381][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:13,383][  INFO][PID:3392862] VM allocation process starts
[2025-09-09 14:44:13,406][  INFO][PID:3392862] Trying to allocate VM: ResallocHost, ticket_id=4983954, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-09-09 14:44:18,415][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:19,448][  INFO][PID:3392862] Allocated host ResallocHost, ticket_id=4983954, hostname=140.211.168.153, name=copr_osuosl_p09_prod_06682018_20250909_140254, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-09-09 14:44:19,449][  INFO][PID:3392862] Allocating ssh connection to builder
[2025-09-09 14:44:19,449][  INFO][PID:3392862] Checking that builder machine is OK
[2025-09-09 14:44:20,985][  INFO][PID:3392862] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-09-09 14:44:21,972][  INFO][PID:3392862] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-09-09 14:44:21,973][  INFO][PID:3392862] Filling build.info file with builder info
[2025-09-09 14:44:21,974][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:21,975][  INFO][PID:3392862] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "python3-scipy-1.10.1",
            "project_dirname": "python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17",
            "submitter": "https://src.fedoraproject.org/user/packit",
            "ended_on": null,
            "started_on": 1757429061.9748442,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9540079-fedora-rawhide-ppc64le",
            "build_id": 9540079,
            "package_name": "python-fsleyes",
            "package_version": "1.16.0-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/python-fsleyes",
            "git_hash": "f6c3f6647626773fd3dc3ad6b613e978b9a5724c",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/python3-scipy-1.10.1--https://src.fedoraproject.org/user/packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.10.1/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/",
                    "id": "copr_coprdir",
                    "name": "Coprdir repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17",
            "result_dir": "09540079-python-fsleyes",
            "built_packages": "",
            "id": 9540079,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "python3-scipy-1.10.1",
                "vendor": "Fedora Project COPR (psimovec/python3-scipy-1.10.1)"
            }
        }
    ]
}
[2025-09-09 14:44:22,020][  INFO][PID:3392862] Sending fedora-messaging bus message in build.start
[2025-09-09 14:44:22,868][  INFO][PID:3392862] Sending fedora-messaging bus message in chroot.start
[2025-09-09 14:44:22,898][  INFO][PID:3392862] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9540079-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-09-09 14:44:24,400][  INFO][PID:3392862] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '140.211.168.153' (ED25519) to the list of known hosts.


[2025-09-09 14:44:24,401][  INFO][PID:3392862] Downloading the builder-live.log file, attempt 1
[2025-09-09 14:44:24,402][  INFO][PID:3392862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.153 copr-rpmbuild-log
[2025-09-09 14:44:29,412][  INFO][PID:3392862] Periodic builder liveness probe: alive
[2025-09-09 14:44:29,413][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:34,414][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:39,415][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:44,416][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:49,418][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:54,419][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:44:59,420][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:04,421][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:09,423][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:14,424][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:19,426][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:24,427][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:29,430][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:34,437][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:39,439][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:44,441][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:49,445][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:54,446][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:45:59,447][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:04,448][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:09,449][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:14,451][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:19,452][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:24,453][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:29,454][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:34,455][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:39,497][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:44,556][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:49,647][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:54,716][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:46:59,777][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:05,047][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:10,126][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:15,185][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:20,263][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:25,390][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:30,392][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:38,095][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:43,097][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:48,098][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:53,099][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:47:58,101][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:03,102][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:08,103][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:13,104][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:18,105][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:23,106][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:28,108][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:33,109][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:38,111][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:43,112][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:48,115][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:53,117][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:48:58,118][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:03,119][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:08,121][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:13,123][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:18,123][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:23,124][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:28,126][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:33,128][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:38,129][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:43,130][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:48,132][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:53,135][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:49:58,136][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:03,137][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:08,139][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:13,140][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:18,141][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:23,142][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:28,143][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:33,144][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:38,145][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:43,146][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:48,147][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:53,148][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:50:58,149][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:03,151][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:08,152][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:13,153][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:18,154][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:23,155][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:28,156][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:33,157][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:38,161][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:43,162][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:48,574][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:53,575][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:51:58,577][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:03,578][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:08,580][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:13,581][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:18,582][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:23,583][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:28,690][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:33,745][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:38,781][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:43,896][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:48,922][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:53,981][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:52:58,995][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:04,041][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:09,054][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:14,174][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:19,233][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:24,252][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:29,332][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:34,403][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:39,437][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:44,440][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:49,475][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:54,560][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:53:59,603][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:04,650][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:09,685][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:14,694][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:19,809][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:24,927][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:30,019][  INFO][PID:3392862] Periodic builder liveness probe: alive
[2025-09-09 14:54:30,114][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:35,133][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:40,182][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:45,185][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:50,221][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:54:55,264][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:00,418][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:05,501][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:10,570][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:15,656][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:20,701][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:25,787][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:30,830][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:35,876][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:40,965][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:46,022][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:51,030][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:55:56,036][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:01,082][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:06,092][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:11,137][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:16,217][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:21,227][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:26,340][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:31,376][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:36,394][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:41,448][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:46,533][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:51,536][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:56:56,604][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:01,632][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:06,742][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:11,757][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:16,759][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:21,792][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:26,807][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:31,948][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:37,015][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:42,032][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:47,037][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:52,111][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:57:57,175][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:02,180][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:07,215][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:12,280][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:17,284][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:22,293][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:27,360][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:32,425][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:37,443][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:42,477][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:47,545][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:52,570][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:58:57,571][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:02,572][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:07,574][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:12,575][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:17,576][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:22,577][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:27,579][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:32,580][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:37,581][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:42,582][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:47,583][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:52,584][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 14:59:57,585][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:02,586][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:07,588][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:12,589][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:17,590][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:22,591][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:27,592][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:32,593][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:37,595][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:42,596][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:47,597][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:52,598][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:00:57,599][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:02,600][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:07,602][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:12,603][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:17,604][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:22,606][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:27,607][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:32,608][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:37,609][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:42,610][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:47,611][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:52,612][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:01:57,613][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:02,614][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:07,615][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:12,616][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:17,618][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:22,619][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:30,318][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:35,319][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:40,320][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:45,321][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:50,322][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:02:55,323][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:00,324][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:05,325][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:10,326][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:15,327][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:20,328][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:25,345][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:30,413][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:35,486][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:40,487][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:45,506][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:50,517][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:03:55,586][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:04:00,633][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:04:05,669][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:04:10,726][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:04:15,753][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:04:20,780][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:04:25,782][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:04:30,845][  INFO][PID:3392862] Periodic builder liveness probe: dead
[2025-09-09 15:04:30,945][  INFO][PID:3392862] calling cancel callback, and waiting
[2025-09-09 15:05:31,016][ ERROR][PID:3392862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.153 port 22: Connection timed out

[2025-09-09 15:06:13,139][ ERROR][PID:3392862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 15:06:23,140][WARNING][PID:3392862] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 15:06:33,141][  INFO][PID:3392862] Downloading the builder-live.log file, attempt 2
[2025-09-09 15:06:33,350][  INFO][PID:3392862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.153 copr-rpmbuild-log
[2025-09-09 15:06:41,085][ ERROR][PID:3392862] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.153 port 22: Connection timed out

[2025-09-09 15:07:33,418][ ERROR][PID:3392862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 15:07:43,422][WARNING][PID:3392862] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 15:07:51,157][ ERROR][PID:3392862] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.153 port 22: Connection timed out

[2025-09-09 15:07:53,422][  INFO][PID:3392862] Downloading the builder-live.log file, attempt 3
[2025-09-09 15:07:53,424][  INFO][PID:3392862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.153 copr-rpmbuild-log
[2025-09-09 15:08:53,493][ ERROR][PID:3392862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 15:09:01,229][ ERROR][PID:3392862] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.153 port 22: Connection timed out

[2025-09-09 15:09:03,498][WARNING][PID:3392862] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 15:09:11,231][ ERROR][PID:3392862] Can't ssh to cancel build.
[2025-09-09 15:09:13,501][  INFO][PID:3392862] Downloading the builder-live.log file, attempt 4
[2025-09-09 15:09:13,503][  INFO][PID:3392862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.153 copr-rpmbuild-log
[2025-09-09 15:10:13,573][ ERROR][PID:3392862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 15:10:23,578][ ERROR][PID:3392862] Giving up for unstable SSH, failures: 1757430383.1405528, 1757430463.422114, 1757430543.4983678, 1757430623.5780346
[2025-09-09 15:10:23,579][ ERROR][PID:3392862] Re-try request for task on 'ResallocHost, ticket_id=4983954, hostname=140.211.168.153, name=copr_osuosl_p09_prod_06682018_20250909_140254, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-09-09 15:10:23,579][  INFO][PID:3392862] Releasing VM back to pool
[2025-09-09 15:10:23,593][  INFO][PID:3392862] Retry #1 (on other host)
[2025-09-09 15:10:23,594][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:10:23,594][  INFO][PID:3392862] VM allocation process starts
[2025-09-09 15:10:23,608][  INFO][PID:3392862] Trying to allocate VM: ResallocHost, ticket_id=4984301, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-09-09 15:10:26,625][  INFO][PID:3392862] Allocated host ResallocHost, ticket_id=4984301, hostname=2620:52:3:1:dead:beef:cafe:c290, name=copr_p09_01_prod_06683051_20250909_150506, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-09-09 15:10:26,625][  INFO][PID:3392862] Allocating ssh connection to builder
[2025-09-09 15:10:26,626][  INFO][PID:3392862] Checking that builder machine is OK
[2025-09-09 15:10:27,417][  INFO][PID:3392862] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-09-09 15:10:27,854][  INFO][PID:3392862] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-09-09 15:10:27,854][  INFO][PID:3392862] Filling build.info file with builder info
[2025-09-09 15:10:27,857][  INFO][PID:3392862] Checking for cancel request
[2025-09-09 15:10:27,858][  INFO][PID:3392862] Not re-notifying FE and msg buses for the new host.
[2025-09-09 15:10:27,858][  INFO][PID:3392862] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9540079-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-09-09 15:10:28,564][  INFO][PID:3392862] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c290' (ED25519) to the list of known hosts.


[2025-09-09 15:10:28,565][  INFO][PID:3392862] Downloading the builder-live.log file, attempt 1
[2025-09-09 15:10:28,566][  INFO][PID:3392862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c290 copr-rpmbuild-log
[2025-09-09 15:10:33,565][  INFO][PID:3392862] calling cancel callback, and waiting
[2025-09-09 15:10:37,106][  INFO][PID:3392862] Cancel request succeeded
out:
success
err:
Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c290' (ED25519) to the list of known hosts.

[2025-09-09 15:10:37,107][  INFO][PID:3392862] Downloading results from builder
[2025-09-09 15:10:37,107][  INFO][PID:3392862] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c290]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/09540079-python-fsleyes started
[2025-09-09 15:10:37,108][  INFO][PID:3392862] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c290]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/09540079-python-fsleyes/ &> /var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/09540079-python-fsleyes/build-09540079.rsync.log
[2025-09-09 15:10:37,565][  INFO][PID:3392862] rsyncing finished.
[2025-09-09 15:10:37,566][  INFO][PID:3392862] Releasing VM back to pool
[2025-09-09 15:10:37,581][  INFO][PID:3392862] Searching for 'success' file in resultdir
[2025-09-09 15:10:37,582][ ERROR][PID:3392862] Build failed: Backend process error: No success file => build failure
[2025-09-09 15:10:37,583][  INFO][PID:3392862] Finished build: id=9540079 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17 chroot=fedora-rawhide-ppc64le 
[2025-09-09 15:10:37,585][ ERROR][PID:3392862] Backend process error: No success file => build failure
[2025-09-09 15:10:37,585][  INFO][PID:3392862] Worker failed build, took 9.727304697036743
[2025-09-09 15:10:37,586][  INFO][PID:3392862] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "python3-scipy-1.10.1",
            "project_dirname": "python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17",
            "submitter": "https://src.fedoraproject.org/user/packit",
            "ended_on": 1757430637.585678,
            "started_on": 1757430627.8583734,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9540079-fedora-rawhide-ppc64le",
            "build_id": 9540079,
            "package_name": "python-fsleyes",
            "package_version": "1.16.0-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/python-fsleyes",
            "git_hash": "f6c3f6647626773fd3dc3ad6b613e978b9a5724c",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/python3-scipy-1.10.1--https://src.fedoraproject.org/user/packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.10.1/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/",
                    "id": "copr_coprdir",
                    "name": "Coprdir repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17",
            "result_dir": "09540079-python-fsleyes",
            "built_packages": "",
            "id": 9540079,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "python3-scipy-1.10.1",
                "vendor": "Fedora Project COPR (psimovec/python3-scipy-1.10.1)"
            }
        }
    ]
}
[2025-09-09 15:10:37,708][  INFO][PID:3392862] Sending fedora-messaging bus message in build.end
[2025-09-09 15:10:37,735][  INFO][PID:3392862] Compressing /var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/09540079-python-fsleyes/builder-live.log by gzip
[2025-09-09 15:10:37,737][  INFO][PID:3392862] Running command 'gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/09540079-python-fsleyes/builder-live.log' as PID 3564930
[2025-09-09 15:10:37,739][  INFO][PID:3392862] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/09540079-python-fsleyes/builder-live.log)
[2025-09-09 15:10:37,739][  INFO][PID:3392862] Compressing /var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/09540079-python-fsleyes/backend.log by gzip
[2025-09-09 15:10:37,741][  INFO][PID:3392862] Running command 'gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.10.1:pr:fork-packit-rpms-python-fsleyes-17/fedora-rawhide-ppc64le/09540079-python-fsleyes/backend.log' as PID 3564932