[2025-07-27 11:38:38,077][  INFO][PID:2514169] Marking build as starting
[2025-07-27 11:38:38,113][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:38:38,114][  INFO][PID:2514169] VM allocation process starts
[2025-07-27 11:38:38,124][  INFO][PID:2514169] Trying to allocate VM: ResallocHost, ticket_id=4184742, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-27 11:38:43,125][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:38:48,127][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:38:53,128][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:38:58,129][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:03,130][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:08,131][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:13,132][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:18,133][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:23,134][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:28,135][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:33,136][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:38,136][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:43,138][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:48,139][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:53,140][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:39:58,141][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:03,142][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:08,143][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:13,144][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:18,145][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:23,146][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:28,147][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:33,148][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:38,149][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:43,150][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:48,151][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:53,152][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:40:58,154][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:03,155][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:08,156][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:13,157][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:18,158][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:23,159][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:28,161][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:33,161][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:38,163][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:43,164][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:48,165][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:53,166][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:41:58,167][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:03,168][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:08,169][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:13,170][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:18,171][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:23,172][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:28,173][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:33,174][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:38,175][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:43,176][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:48,177][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:53,178][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:42:58,179][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:03,180][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:08,181][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:13,182][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:18,183][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:23,184][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:28,185][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:33,186][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:38,187][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:43,188][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:48,189][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:53,190][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:43:58,192][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:03,193][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:08,194][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:13,195][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:18,196][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:23,197][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:28,198][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:33,199][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:38,200][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:43,201][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:48,202][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:53,203][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:44:58,204][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:03,205][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:08,206][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:13,207][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:18,208][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:23,209][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:28,210][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:33,211][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:38,212][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:43,214][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:48,215][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:53,216][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:56,251][  INFO][PID:2514169] Allocated host ResallocHost, ticket_id=4184742, hostname=140.211.168.61, name=copr_osuosl_p09_prod_05906125_20250727_114147, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-27 11:45:56,252][  INFO][PID:2514169] Allocating ssh connection to builder
[2025-07-27 11:45:56,253][  INFO][PID:2514169] Checking that builder machine is OK
[2025-07-27 11:45:57,793][  INFO][PID:2514169] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-07-27 11:45:58,824][  INFO][PID:2514169] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-07-27 11:45:58,824][  INFO][PID:2514169] Filling build.info file with builder info
[2025-07-27 11:45:58,825][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:45:58,826][  INFO][PID:2514169] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "openblas-0.3.23",
            "project_dirname": "openblas-0.3.23",
            "submitter": "https://src.fedoraproject.org/user/releng",
            "ended_on": null,
            "started_on": 1753616758.8258443,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9321062-fedora-rawhide-ppc64le",
            "build_id": 9321062,
            "package_name": "luabind",
            "package_version": "0.9.1-49",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/luabind",
            "git_hash": "758babc1974b4400fd9945cf8a8de6c057fa6a85",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/releng",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23",
            "result_dir": "09321062-luabind",
            "built_packages": "",
            "id": 9321062,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "openblas-0.3.23",
                "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)"
            }
        }
    ]
}
[2025-07-27 11:45:58,861][  INFO][PID:2514169] Sending fedora-messaging bus message in build.start
[2025-07-27 11:45:59,503][  INFO][PID:2514169] Sending fedora-messaging bus message in chroot.start
[2025-07-27 11:45:59,527][  INFO][PID:2514169] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9321062-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-07-27 11:46:01,010][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 1
[2025-07-27 11:46:01,012][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-27 11:46:06,016][  INFO][PID:2514169] Periodic builder liveness probe: alive
[2025-07-27 11:46:06,017][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:11,018][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:16,019][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:21,020][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:26,021][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:31,022][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:36,024][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:41,025][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:46,026][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:51,027][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:46:56,028][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:01,029][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:06,030][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:11,031][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:16,033][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:21,034][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:26,035][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:31,036][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:36,037][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:41,038][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:46,039][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:51,040][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:47:56,041][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:01,042][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:06,043][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:11,044][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:16,045][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:21,047][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:26,047][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:31,048][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:36,049][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:41,050][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:46,051][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:51,053][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:48:56,054][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:01,055][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:06,056][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:11,057][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:16,058][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:21,059][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:26,060][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:31,061][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:31,139][ ERROR][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-27 11:49:36,062][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:41,063][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:41,140][WARNING][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-27 11:49:46,064][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:51,065][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:49:51,141][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 2
[2025-07-27 11:49:51,142][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-27 11:49:56,066][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:01,067][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:06,068][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:11,069][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:16,070][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:21,071][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:26,072][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:31,073][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:36,074][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:41,075][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:46,077][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:51,078][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:50:51,277][ ERROR][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-27 11:50:56,079][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:01,080][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:01,280][WARNING][PID:2514169] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-27 11:51:06,081][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:11,082][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:11,281][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 3
[2025-07-27 11:51:11,282][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-27 11:51:16,083][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:21,085][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:26,086][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:31,087][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:36,089][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:41,090][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:46,091][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:51,092][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:51:56,093][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:01,094][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:06,095][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:11,096][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:11,419][ ERROR][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-27 11:52:16,097][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:21,098][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:21,420][WARNING][PID:2514169] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-27 11:52:26,099][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:31,100][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:31,421][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 4
[2025-07-27 11:52:31,423][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-07-27 11:52:36,101][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:41,102][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:46,103][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:51,105][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:52:56,106][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:01,107][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:06,108][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:11,109][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:16,110][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:21,111][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:26,112][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:31,113][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:31,555][ ERROR][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-27 11:53:36,114][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:41,115][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:41,556][ ERROR][PID:2514169] Giving up for unstable SSH, failures: 1753616981.140668, 1753617061.2806802, 1753617141.4206662, 1753617221.5565262
[2025-07-27 11:53:41,557][ ERROR][PID:2514169] Re-try request for task on 'ResallocHost, ticket_id=4184742, hostname=140.211.168.61, name=copr_osuosl_p09_prod_05906125_20250727_114147, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-07-27 11:53:41,558][  INFO][PID:2514169] Releasing VM back to pool
[2025-07-27 11:53:41,567][  INFO][PID:2514169] Retry #1 (on other host)
[2025-07-27 11:53:41,567][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:41,568][  INFO][PID:2514169] VM allocation process starts
[2025-07-27 11:53:41,603][  INFO][PID:2514169] Trying to allocate VM: ResallocHost, ticket_id=4184785, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-27 11:53:46,604][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:51,605][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:53:56,606][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:01,607][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:06,608][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:11,609][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:16,610][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:21,611][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:26,612][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:31,614][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:36,615][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:41,616][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:46,617][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:51,618][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:54:56,619][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:01,620][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:06,621][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:11,622][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:16,623][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:21,624][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:26,625][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:31,626][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:36,627][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:41,629][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:46,630][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:51,631][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:55:56,632][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:01,633][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:06,634][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:11,635][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:16,636][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:21,637][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:26,638][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:31,639][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:36,640][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:41,641][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:46,642][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:51,643][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:56:56,644][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:01,645][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:06,646][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:11,647][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:16,648][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:21,649][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:26,650][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:31,652][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:36,653][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:41,654][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:46,655][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:51,656][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:56,657][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:57:59,691][  INFO][PID:2514169] Allocated host ResallocHost, ticket_id=4184785, hostname=140.211.168.171, name=copr_osuosl_p09_prod_05906194_20250727_115403, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-27 11:57:59,692][  INFO][PID:2514169] Allocating ssh connection to builder
[2025-07-27 11:57:59,692][  INFO][PID:2514169] Checking that builder machine is OK
[2025-07-27 11:58:01,162][  INFO][PID:2514169] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-07-27 11:58:02,178][  INFO][PID:2514169] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-07-27 11:58:02,178][  INFO][PID:2514169] Filling build.info file with builder info
[2025-07-27 11:58:02,179][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:02,180][  INFO][PID:2514169] Not re-notifying FE and msg buses for the new host.
[2025-07-27 11:58:02,180][  INFO][PID:2514169] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9321062-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-07-27 11:58:03,507][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 1
[2025-07-27 11:58:03,509][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.171 copr-rpmbuild-log
[2025-07-27 11:58:08,512][  INFO][PID:2514169] Periodic builder liveness probe: alive
[2025-07-27 11:58:08,513][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:13,514][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:18,515][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:23,516][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:28,517][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:33,518][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:38,519][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:43,520][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:48,521][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:53,522][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:58:58,523][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:03,524][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:08,525][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:13,527][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:18,534][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:23,535][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:28,536][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:33,537][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:38,538][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:43,540][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:48,541][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:53,542][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 11:59:58,543][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:03,544][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:08,545][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:13,546][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:18,547][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:23,548][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:28,549][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:33,550][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:38,551][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:43,553][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:48,554][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:53,555][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:00:58,556][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:03,557][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:08,558][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:13,559][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:18,560][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:23,562][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:28,563][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:33,564][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:35,145][ ERROR][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-27 12:01:38,565][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:43,575][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:45,145][WARNING][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-27 12:01:48,579][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:53,580][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:01:55,146][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 2
[2025-07-27 12:01:55,148][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.171 copr-rpmbuild-log
[2025-07-27 12:01:58,581][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:03,582][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:08,583][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:13,584][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:18,585][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:23,587][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:28,588][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:33,590][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:38,597][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:43,599][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:48,602][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:53,604][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:02:55,285][ ERROR][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-27 12:02:58,605][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:03,606][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:05,288][WARNING][PID:2514169] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-27 12:03:08,607][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:13,608][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:15,289][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 3
[2025-07-27 12:03:15,290][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.171 copr-rpmbuild-log
[2025-07-27 12:03:18,610][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:23,612][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:28,615][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:33,616][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:38,618][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:43,619][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:48,620][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:53,622][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:03:58,623][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:03,624][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:08,626][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:13,627][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:15,429][ ERROR][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-27 12:04:18,628][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:23,629][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:25,430][WARNING][PID:2514169] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-27 12:04:28,630][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:33,632][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:35,431][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 4
[2025-07-27 12:04:35,433][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.171 copr-rpmbuild-log
[2025-07-27 12:04:38,633][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:43,634][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:48,635][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:53,636][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:04:58,637][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:03,638][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:08,639][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:13,640][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:18,641][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:23,642][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:28,643][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:33,644][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:35,570][ ERROR][PID:2514169] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-27 12:05:38,645][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:43,646][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:45,571][ ERROR][PID:2514169] Giving up for unstable SSH, failures: 1753617705.1459036, 1753617785.28881, 1753617865.4306297, 1753617945.5710523
[2025-07-27 12:05:45,572][ ERROR][PID:2514169] Re-try request for task on 'ResallocHost, ticket_id=4184785, hostname=140.211.168.171, name=copr_osuosl_p09_prod_05906194_20250727_115403, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-07-27 12:05:45,572][  INFO][PID:2514169] Releasing VM back to pool
[2025-07-27 12:05:45,581][  INFO][PID:2514169] Retry #2 (on other host)
[2025-07-27 12:05:45,582][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:45,582][  INFO][PID:2514169] VM allocation process starts
[2025-07-27 12:05:45,593][  INFO][PID:2514169] Trying to allocate VM: ResallocHost, ticket_id=4184847, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-27 12:05:50,594][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:05:55,595][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:00,596][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:05,597][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:10,598][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:15,599][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:20,600][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:25,601][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:30,602][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:35,603][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:40,604][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:45,605][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:50,606][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:06:55,607][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:00,608][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:05,609][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:10,610][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:15,611][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:20,612][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:25,613][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:30,614][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:35,616][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:40,617][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:45,618][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:50,619][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:07:55,620][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:00,621][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:05,622][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:10,623][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:15,624][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:20,625][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:25,630][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:30,631][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:35,632][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:40,633][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:45,634][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:50,638][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:08:55,639][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:00,640][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:05,641][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:10,642][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:15,643][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:20,645][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:25,646][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:30,647][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:35,649][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:40,650][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:45,651][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:50,652][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:09:55,653][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:00,655][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:05,656][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:10,657][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:15,658][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:20,659][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:25,660][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:30,661][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:35,662][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:40,664][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:45,665][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:50,666][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:10:55,667][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:00,668][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:05,669][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:10,670][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:15,672][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:20,673][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:25,674][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:30,675][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:35,676][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:40,677][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:45,678][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:50,679][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:11:55,681][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:00,682][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:05,683][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:10,684][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:15,685][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:20,686][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:25,687][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:30,688][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:35,689][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:40,690][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:45,691][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:50,692][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:12:55,693][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:00,694][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:03,731][  INFO][PID:2514169] Allocated host ResallocHost, ticket_id=4184847, hostname=2620:52:3:1:dead:beef:cafe:c24a, name=copr_hv_ppc64le_02_prod_05906174_20250727_115045, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-27 12:13:03,732][  INFO][PID:2514169] Allocating ssh connection to builder
[2025-07-27 12:13:03,732][  INFO][PID:2514169] Checking that builder machine is OK
[2025-07-27 12:13:05,403][  INFO][PID:2514169] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-07-27 12:13:06,207][  INFO][PID:2514169] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-07-27 12:13:06,207][  INFO][PID:2514169] Filling build.info file with builder info
[2025-07-27 12:13:06,212][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:06,213][  INFO][PID:2514169] Not re-notifying FE and msg buses for the new host.
[2025-07-27 12:13:06,214][  INFO][PID:2514169] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9321062-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-07-27 12:13:08,184][  INFO][PID:2514169] Downloading the builder-live.log file, attempt 1
[2025-07-27 12:13:08,185][  INFO][PID:2514169] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c24a copr-rpmbuild-log
[2025-07-27 12:13:13,191][  INFO][PID:2514169] Periodic builder liveness probe: alive
[2025-07-27 12:13:13,191][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:18,192][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:23,193][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:28,194][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:33,195][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:38,196][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:43,197][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:48,198][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:53,199][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:13:58,200][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:03,201][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:08,202][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:13,203][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:18,204][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:23,206][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:28,206][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:33,207][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:38,209][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:43,210][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:48,211][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:53,212][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:14:58,213][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:03,214][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:08,215][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:13,216][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:18,217][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:23,218][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:28,219][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:33,220][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:38,221][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:43,222][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:48,223][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:53,224][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:15:58,226][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:03,227][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:08,228][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:13,229][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:18,230][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:23,231][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:28,232][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:33,233][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:38,234][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:43,235][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:48,236][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:53,237][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:16:58,238][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:03,239][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:08,240][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:13,241][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:18,242][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:23,243][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:28,245][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:33,246][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:38,246][  INFO][PID:2514169] Checking for cancel request
[2025-07-27 12:17:42,423][  INFO][PID:2514169] Downloading results from builder
[2025-07-27 12:17:42,423][  INFO][PID:2514169] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c24a]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind started
[2025-07-27 12:17:42,424][  INFO][PID:2514169] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c24a]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/ &> /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/build-09321062.rsync.log
[2025-07-27 12:17:44,141][  INFO][PID:2514169] rsyncing finished.
[2025-07-27 12:17:44,142][  INFO][PID:2514169] Releasing VM back to pool
[2025-07-27 12:17:44,152][  INFO][PID:2514169] Searching for 'success' file in resultdir
[2025-07-27 12:17:44,153][  INFO][PID:2514169] Going to sign pkgs from source: 9321062-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le
[2025-07-27 12:17:44,153][  INFO][PID:2514169] Calling '/bin/sign -u psimovec#openblas-0.3.23@copr.fedorahosted.org -p' (attempt #1)
[2025-07-27 12:17:44,325][  INFO][PID:2514169] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-devel-0.9.1-49.fc43.ppc64le.rpm' (attempt #1)
[2025-07-27 12:17:45,096][  INFO][PID:2514169] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-devel-0.9.1-49.fc43.ppc64le.rpm
[2025-07-27 12:17:45,097][  INFO][PID:2514169] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-debugsource-0.9.1-49.fc43.ppc64le.rpm' (attempt #1)
[2025-07-27 12:17:45,710][  INFO][PID:2514169] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-debugsource-0.9.1-49.fc43.ppc64le.rpm
[2025-07-27 12:17:45,711][  INFO][PID:2514169] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-0.9.1-49.fc43.src.rpm' (attempt #1)
[2025-07-27 12:17:46,497][  INFO][PID:2514169] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-0.9.1-49.fc43.src.rpm
[2025-07-27 12:17:46,497][  INFO][PID:2514169] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-debuginfo-0.9.1-49.fc43.ppc64le.rpm' (attempt #1)
[2025-07-27 12:17:47,425][  INFO][PID:2514169] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-debuginfo-0.9.1-49.fc43.ppc64le.rpm
[2025-07-27 12:17:47,426][  INFO][PID:2514169] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-0.9.1-49.fc43.ppc64le.rpm' (attempt #1)
[2025-07-27 12:17:48,363][  INFO][PID:2514169] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/luabind-0.9.1-49.fc43.ppc64le.rpm
[2025-07-27 12:17:48,364][  INFO][PID:2514169] Sign done
[2025-07-27 12:17:48,364][  INFO][PID:2514169] Incremental createrepo run, adding 09321062-luabind into https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le, (auto-create-repo=True)
[2025-07-27 12:17:48,365][  INFO][PID:2514169] Running command 'copr-repo --batched /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le --add 09321062-luabind --no-appstream-metadata' as PID 2632040
[2025-07-27 12:17:59,706][  INFO][PID:2514169] Finished after 11 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le --add 09321062-luabind --no-appstream-metadata)
[2025-07-27 12:17:59,707][  INFO][PID:2514169] Getting build details
[2025-07-27 12:17:59,708][  INFO][PID:2514169] Listing built binary packages in /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind
[2025-07-27 12:17:59,708][  INFO][PID:2514169] Built packages:
['luabind-debugsource 0.9.1', 'luabind 0.9.1', 'luabind-debuginfo 0.9.1', 'luabind-devel 0.9.1']
[2025-07-27 12:17:59,709][  INFO][PID:2514169] build details: {'built_packages': 'luabind-debugsource 0.9.1\nluabind 0.9.1\nluabind-debuginfo 0.9.1\nluabind-devel 0.9.1'}
[2025-07-27 12:17:59,709][  INFO][PID:2514169] Retrieving pubkey
[2025-07-27 12:17:59,710][  INFO][PID:2514169] Calling '/bin/sign -u psimovec#openblas-0.3.23@copr.fedorahosted.org -p' (attempt #1)
[2025-07-27 12:18:00,034][  INFO][PID:2514169] Added pubkey for user psimovec project openblas-0.3.23 into: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/pubkey.gpg
[2025-07-27 12:18:00,035][  INFO][PID:2514169] Finished build: id=9321062 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/openblas-0.3.23 chroot=fedora-rawhide-ppc64le 
[2025-07-27 12:18:00,036][  INFO][PID:2514169] Worker succeeded build, took 293.82262802124023
[2025-07-27 12:18:00,037][  INFO][PID:2514169] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "openblas-0.3.23",
            "project_dirname": "openblas-0.3.23",
            "submitter": "https://src.fedoraproject.org/user/releng",
            "ended_on": 1753618680.036513,
            "started_on": 1753618386.213885,
            "submitted_on": null,
            "status": 1,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9321062-fedora-rawhide-ppc64le",
            "build_id": 9321062,
            "package_name": "luabind",
            "package_version": "0.9.1-49",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/luabind",
            "git_hash": "758babc1974b4400fd9945cf8a8de6c057fa6a85",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/releng",
            "results": {
                "packages": [
                    {
                        "name": "luabind-debugsource",
                        "epoch": null,
                        "version": "0.9.1",
                        "release": "49.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "luabind",
                        "epoch": null,
                        "version": "0.9.1",
                        "release": "49.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "luabind-debuginfo",
                        "epoch": null,
                        "version": "0.9.1",
                        "release": "49.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "luabind-devel",
                        "epoch": null,
                        "version": "0.9.1",
                        "release": "49.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "luabind",
                        "epoch": null,
                        "version": "0.9.1",
                        "release": "49.fc43",
                        "arch": "src"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23",
            "result_dir": "09321062-luabind",
            "built_packages": "luabind-debugsource 0.9.1\nluabind 0.9.1\nluabind-debuginfo 0.9.1\nluabind-devel 0.9.1",
            "id": 9321062,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "openblas-0.3.23",
                "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)"
            }
        }
    ]
}
[2025-07-27 12:18:00,087][  INFO][PID:2514169] Sending fedora-messaging bus message in build.end
[2025-07-27 12:18:00,112][  INFO][PID:2514169] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/builder-live.log by gzip
[2025-07-27 12:18:00,114][  INFO][PID:2514169] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/builder-live.log' as PID 2632404
[2025-07-27 12:18:00,121][  INFO][PID:2514169] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/builder-live.log)
[2025-07-27 12:18:00,122][  INFO][PID:2514169] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/backend.log by gzip
[2025-07-27 12:18:00,123][  INFO][PID:2514169] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09321062-luabind/backend.log' as PID 2632405