[2025-12-13 01:35:41,871][  INFO][PID:176318] Marking build as starting
[2025-12-13 01:35:42,112][  INFO][PID:176318] Checking that /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/repodata/repomd.xml exists
[2025-12-13 01:35:42,186][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:35:42,203][  INFO][PID:176318] VM allocation process starts
[2025-12-13 01:35:42,264][  INFO][PID:176318] Trying to allocate VM: ResallocHost, ticket_id=313986, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-13 01:35:47,287][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:35:52,289][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:35:57,290][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:00,413][  INFO][PID:176318] Allocated host ResallocHost, ticket_id=313986, hostname=2620:52:6:1161:dead:beef:cafe:c10b, name=vmhost_p09_02_prod_00513700_20251213_003434, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-13 01:36:00,415][  INFO][PID:176318] Allocating ssh connection to builder
[2025-12-13 01:36:00,415][  INFO][PID:176318] Checking that builder machine is OK
[2025-12-13 01:36:01,638][  INFO][PID:176318] Running remote command: copr-builder-ready fedora-43-ppc64le
[2025-12-13 01:36:02,211][  INFO][PID:176318] Red Hat subscription not needed for fedora-43-ppc64le
Builder is ready to be used

[2025-12-13 01:36:02,215][  INFO][PID:176318] Filling build.info file with builder info
[2025-12-13 01:36:02,217][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:02,223][  INFO][PID:176318] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "scipy-1.10.0",
            "project_dirname": "scipy-1.10.0",
            "submitter": "https://src.fedoraproject.org/user/qulogic",
            "ended_on": null,
            "started_on": 1765589762.223142,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-43-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9904544-fedora-43-ppc64le",
            "build_id": 9904544,
            "package_name": "python-rasterio",
            "package_version": "1.4.4-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/scipy-1.10.0/python-rasterio",
            "git_hash": "a2bb74c3553748895a05b1b6115b24931a9cc78e",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/scipy-1.10.0--https://src.fedoraproject.org/user/qulogic",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/scipy-1.10.0",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.10.0",
            "result_dir": "09904544-python-rasterio",
            "built_packages": "",
            "id": 9904544,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "scipy-1.10.0",
                "vendor": "Fedora Project COPR (psimovec/scipy-1.10.0)"
            }
        }
    ]
}
[2025-12-13 01:36:02,311][  INFO][PID:176318] Sending fedora-messaging bus message in build.start
[2025-12-13 01:36:03,598][  INFO][PID:176318] Sending fedora-messaging bus message in chroot.start
[2025-12-13 01:36:03,641][  INFO][PID:176318] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9904544-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached
[2025-12-13 01:36:04,651][  INFO][PID:176318] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c10b' (ED25519) to the list of known hosts.


[2025-12-13 01:36:04,653][  INFO][PID:176318] Downloading the builder-live.log file, attempt 1
[2025-12-13 01:36:04,658][  INFO][PID:176318] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10b copr-rpmbuild-log
[2025-12-13 01:36:09,666][  INFO][PID:176318] Periodic builder liveness probe: alive
[2025-12-13 01:36:09,667][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:14,671][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:19,676][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:24,708][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:29,710][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:34,711][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:39,713][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:44,719][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:49,721][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:54,746][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:36:59,779][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:04,781][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:09,788][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:14,791][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:19,792][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:24,801][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:29,804][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:34,807][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:39,810][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:44,813][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:49,820][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:54,831][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:37:59,833][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:04,837][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:09,839][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:14,840][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:19,844][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:24,860][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:29,864][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:34,865][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:39,867][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:44,868][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:49,869][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:54,871][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:38:59,873][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:04,876][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:09,877][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:14,878][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:19,879][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:24,880][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:29,882][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:34,895][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:39,896][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:44,897][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:49,899][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:54,901][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:39:59,903][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:04,905][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:09,906][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:14,908][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:19,910][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:24,913][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:29,917][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:34,919][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:39,920][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:44,921][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:49,923][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:54,935][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:40:59,936][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:04,937][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:09,939][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:14,940][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:19,942][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:24,943][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:29,944][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:34,946][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:39,949][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:44,950][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:49,951][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:54,952][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:41:59,953][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:04,954][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:09,955][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:14,956][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:19,957][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:24,958][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:29,959][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:34,960][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:39,962][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:44,963][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:49,967][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:54,968][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:42:59,972][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:04,974][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:09,975][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:14,977][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:19,978][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:24,979][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:29,981][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:34,982][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:39,984][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:44,992][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:49,993][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:54,995][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:43:59,997][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:04,998][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:09,999][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:15,001][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:20,003][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:25,005][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:30,006][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:35,007][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:40,008][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:45,009][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:50,011][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:44:55,016][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:00,017][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:05,018][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:10,019][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:15,020][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:20,023][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:25,027][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:30,028][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:35,029][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:40,030][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:45,031][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:50,032][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:45:55,034][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:00,034][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:05,036][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:10,057][  INFO][PID:176318] Periodic builder liveness probe: alive
[2025-12-13 01:46:10,065][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:15,066][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:20,067][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:25,068][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:30,070][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:35,071][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:40,072][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:45,073][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:50,074][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:46:55,075][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:00,077][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:05,079][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:10,080][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:15,081][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:20,082][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:25,083][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:30,084][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:35,085][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:40,086][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:45,087][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:50,088][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:47:55,089][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:00,090][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:05,091][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:10,092][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:15,093][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:20,094][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:25,094][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:30,095][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:35,097][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:40,098][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:45,099][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:50,100][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:48:55,101][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:00,102][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:05,103][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:10,104][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:15,105][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:20,106][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:25,107][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:30,108][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:35,109][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:40,110][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:45,111][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:49:48,577][  INFO][PID:176318] Downloading results from builder
[2025-12-13 01:49:48,579][  INFO][PID:176318] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10b]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio started
[2025-12-13 01:49:48,584][  INFO][PID:176318] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10b]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/ &> /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/build-09904544.rsync.log
[2025-12-13 01:50:48,676][  INFO][PID:176318] rsyncing finished.
[2025-12-13 01:50:48,677][ ERROR][PID:176318] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-12-13 01:50:48,677][ ERROR][PID:176318] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-12-13 01:50:58,679][  INFO][PID:176318] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10b]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio started
[2025-12-13 01:50:58,680][  INFO][PID:176318] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10b]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/ &> /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/build-09904544.rsync.log
[2025-12-13 01:51:58,763][  INFO][PID:176318] rsyncing finished.
[2025-12-13 01:51:58,764][ ERROR][PID:176318] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-12-13 01:51:58,764][ ERROR][PID:176318] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-12-13 01:52:08,765][  INFO][PID:176318] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10b]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio started
[2025-12-13 01:52:08,766][  INFO][PID:176318] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10b]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/ &> /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/build-09904544.rsync.log
[2025-12-13 01:53:08,843][  INFO][PID:176318] rsyncing finished.
[2025-12-13 01:53:08,844][ ERROR][PID:176318] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-12-13 01:53:08,844][ ERROR][PID:176318] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-12-13 01:53:18,845][ ERROR][PID:176318] Re-try request for task on 'ResallocHost, ticket_id=313986, hostname=2620:52:6:1161:dead:beef:cafe:c10b, name=vmhost_p09_02_prod_00513700_20251213_003434, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 3 SSH attempts
[2025-12-13 01:53:18,846][  INFO][PID:176318] Releasing VM back to pool
[2025-12-13 01:53:18,871][  INFO][PID:176318] Retry #1 (on other host)
[2025-12-13 01:53:18,873][  INFO][PID:176318] Checking that /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/repodata/repomd.xml exists
[2025-12-13 01:53:18,876][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:53:18,877][  INFO][PID:176318] VM allocation process starts
[2025-12-13 01:53:18,889][  INFO][PID:176318] Trying to allocate VM: ResallocHost, ticket_id=314291, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-13 01:53:21,930][  INFO][PID:176318] Allocated host ResallocHost, ticket_id=314291, hostname=2620:52:6:1161:dead:beef:cafe:c154, name=vmhost_p09_03_prod_00514235_20251213_012240, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-13 01:53:21,932][  INFO][PID:176318] Allocating ssh connection to builder
[2025-12-13 01:53:21,933][  INFO][PID:176318] Checking that builder machine is OK
[2025-12-13 01:54:22,013][ ERROR][PID:176318] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c154 port 22: Connection timed out

[2025-12-13 01:54:32,027][ ERROR][PID:176318] Re-try request for task on 'ResallocHost, ticket_id=314291, hostname=2620:52:6:1161:dead:beef:cafe:c154, name=vmhost_p09_03_prod_00514235_20251213_012240, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2025-12-13 01:54:32,028][  INFO][PID:176318] Releasing VM back to pool
[2025-12-13 01:54:32,042][  INFO][PID:176318] Retry #2 (on other host)
[2025-12-13 01:54:32,043][  INFO][PID:176318] Checking that /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/repodata/repomd.xml exists
[2025-12-13 01:54:32,044][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:54:32,049][  INFO][PID:176318] VM allocation process starts
[2025-12-13 01:54:32,064][  INFO][PID:176318] Trying to allocate VM: ResallocHost, ticket_id=314332, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-13 01:54:37,065][  INFO][PID:176318] Checking for cancel request
[2025-12-13 01:54:38,103][  INFO][PID:176318] Allocated host ResallocHost, ticket_id=314332, hostname=2620:52:6:1161:dead:beef:cafe:c196, name=vmhost_p09_04_prod_00514561_20251213_013915, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-13 01:54:38,104][  INFO][PID:176318] Allocating ssh connection to builder
[2025-12-13 01:54:38,104][  INFO][PID:176318] Checking that builder machine is OK
[2025-12-13 01:55:38,174][ ERROR][PID:176318] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c196 port 22: Connection timed out

[2025-12-13 01:55:48,194][ ERROR][PID:176318] Re-try request for task on 'ResallocHost, ticket_id=314332, hostname=2620:52:6:1161:dead:beef:cafe:c196, name=vmhost_p09_04_prod_00514561_20251213_013915, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2025-12-13 01:55:48,195][  INFO][PID:176318] Releasing VM back to pool
[2025-12-13 01:55:48,211][ ERROR][PID:176318] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c10b', '2620:52:6:1161:dead:beef:cafe:c196', '2620:52:6:1161:dead:beef:cafe:c154'}
[2025-12-13 01:55:48,213][WARNING][PID:176318] Switching not-finished job state to 'failed'
[2025-12-13 01:55:48,214][  INFO][PID:176318] Worker failed build, took 1185.9905078411102
[2025-12-13 01:55:48,216][  INFO][PID:176318] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "scipy-1.10.0",
            "project_dirname": "scipy-1.10.0",
            "submitter": "https://src.fedoraproject.org/user/qulogic",
            "ended_on": 1765590948.2136497,
            "started_on": 1765589762.223142,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-43-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9904544-fedora-43-ppc64le",
            "build_id": 9904544,
            "package_name": "python-rasterio",
            "package_version": "1.4.4-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/scipy-1.10.0/python-rasterio",
            "git_hash": "a2bb74c3553748895a05b1b6115b24931a9cc78e",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/scipy-1.10.0--https://src.fedoraproject.org/user/qulogic",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/scipy-1.10.0",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.10.0",
            "result_dir": "09904544-python-rasterio",
            "built_packages": "",
            "id": 9904544,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "scipy-1.10.0",
                "vendor": "Fedora Project COPR (psimovec/scipy-1.10.0)"
            }
        }
    ]
}
[2025-12-13 01:55:48,364][  INFO][PID:176318] Sending fedora-messaging bus message in build.end
[2025-12-13 01:55:48,408][  INFO][PID:176318] Compressing /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/builder-live.log by gzip
[2025-12-13 01:55:48,411][  INFO][PID:176318] Running command 'gzip /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/builder-live.log' as PID 306668
[2025-12-13 01:55:48,503][  INFO][PID:176318] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/builder-live.log)
[2025-12-13 01:55:48,507][  INFO][PID:176318] Compressing /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/backend.log by gzip
[2025-12-13 01:55:48,511][  INFO][PID:176318] Running command 'gzip /var/lib/copr/public_html/results/psimovec/scipy-1.10.0/fedora-43-ppc64le/09904544-python-rasterio/backend.log' as PID 306678