[2025-07-28 02:02:46,587][  INFO][PID:1708938] Marking build as starting
[2025-07-28 02:02:46,625][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:02:46,626][  INFO][PID:1708938] VM allocation process starts
[2025-07-28 02:02:46,640][  INFO][PID:1708938] Trying to allocate VM: ResallocHost, ticket_id=4190984, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-28 02:02:51,642][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:02:56,643][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:01,644][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:06,645][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:11,646][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:16,647][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:21,649][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:26,650][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:31,651][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:36,652][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:41,653][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:46,654][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:51,655][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:03:56,656][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:01,658][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:06,659][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:11,660][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:16,661][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:21,662][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:26,664][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:31,667][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:36,668][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:41,669][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:46,670][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:51,671][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:04:56,673][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:01,674][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:06,675][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:11,676][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:16,677][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:21,678][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:26,680][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:31,681][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:36,682][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:41,683][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:46,684][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:51,685][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:05:56,686][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:01,688][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:06,689][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:11,690][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:16,691][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:21,692][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:26,693][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:31,694][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:36,696][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:41,697][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:46,698][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:51,699][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:06:56,700][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:01,701][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:06,707][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:11,708][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:16,709][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:21,710][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:26,711][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:31,712][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:36,713][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:41,714][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:46,715][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:51,716][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:07:56,718][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:01,719][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:06,720][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:11,721][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:16,722][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:21,725][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:26,726][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:31,728][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:36,729][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:41,731][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:46,732][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:51,734][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:08:56,735][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:01,737][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:06,739][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:11,740][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:16,741][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:21,742][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:26,743][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:31,744][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:36,745][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:41,746][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:46,747][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:51,748][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:09:56,749][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:01,750][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:06,751][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:11,752][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:16,753][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:21,754][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:26,755][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:31,757][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:36,758][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:41,759][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:46,760][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:51,761][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:10:56,762][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:01,763][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:04,818][  INFO][PID:1708938] Allocated host ResallocHost, ticket_id=4190984, hostname=140.211.168.240, name=copr_osuosl_p09_prod_05919976_20250728_020707, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-28 02:11:04,819][  INFO][PID:1708938] Allocating ssh connection to builder
[2025-07-28 02:11:04,819][  INFO][PID:1708938] Checking that builder machine is OK
[2025-07-28 02:11:06,052][  INFO][PID:1708938] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-07-28 02:11:07,043][  INFO][PID:1708938] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-07-28 02:11:07,044][  INFO][PID:1708938] Filling build.info file with builder info
[2025-07-28 02:11:07,045][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:07,046][  INFO][PID:1708938] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "openblas-0.3.23",
            "project_dirname": "openblas-0.3.23",
            "submitter": "https://src.fedoraproject.org/user/releng",
            "ended_on": null,
            "started_on": 1753668667.046256,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9323533-fedora-rawhide-ppc64le",
            "build_id": 9323533,
            "package_name": "plasma-welcome",
            "package_version": "6.4.3-2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/plasma-welcome",
            "git_hash": "cf8574aff128bb40a36331e423784af92fdbb11d",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/releng",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23",
            "result_dir": "09323533-plasma-welcome",
            "built_packages": "",
            "id": 9323533,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "openblas-0.3.23",
                "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)"
            }
        }
    ]
}
[2025-07-28 02:11:07,105][  INFO][PID:1708938] Sending fedora-messaging bus message in build.start
[2025-07-28 02:11:07,808][  INFO][PID:1708938] Sending fedora-messaging bus message in chroot.start
[2025-07-28 02:11:07,834][  INFO][PID:1708938] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9323533-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-07-28 02:11:09,134][  INFO][PID:1708938] Downloading the builder-live.log file, attempt 1
[2025-07-28 02:11:09,135][  INFO][PID:1708938] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.240 copr-rpmbuild-log
[2025-07-28 02:11:14,143][  INFO][PID:1708938] Periodic builder liveness probe: alive
[2025-07-28 02:11:14,144][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:19,145][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:24,146][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:29,147][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:34,148][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:39,149][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:44,150][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:49,151][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:54,153][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:11:59,154][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:04,156][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:09,156][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:14,158][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:19,159][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:24,160][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:29,161][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:34,162][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:39,163][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:44,165][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:49,166][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:54,167][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:12:59,168][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:04,170][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:09,171][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:14,172][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:19,173][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:24,174][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:29,176][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:34,177][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:39,178][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:44,179][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:49,181][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:54,182][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:13:59,189][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:04,194][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:09,195][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:14,196][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:19,198][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:24,208][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:29,214][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:34,216][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:39,217][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:44,218][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:49,219][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:54,221][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:14:59,222][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:04,223][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:09,227][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:14,230][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:19,232][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:24,233][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:29,236][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:34,239][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:39,249][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:44,253][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:49,255][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:54,257][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:15:59,258][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:04,259][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:09,262][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:14,264][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:19,266][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:24,270][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:29,273][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:34,276][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:39,279][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:44,280][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:49,284][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:54,294][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:16:59,296][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:04,299][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:09,303][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:14,306][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:19,307][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:24,308][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:29,312][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:34,313][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:39,314][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:44,317][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:49,320][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:54,322][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:17:59,325][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:04,326][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:09,328][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:14,336][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:19,338][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:24,339][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:29,340][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:34,139][ ERROR][PID:1708938] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-28 02:18:34,341][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:39,342][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:44,140][WARNING][PID:1708938] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-28 02:18:44,344][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:49,345][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:54,141][  INFO][PID:1708938] Downloading the builder-live.log file, attempt 2
[2025-07-28 02:18:54,144][  INFO][PID:1708938] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.240 copr-rpmbuild-log
[2025-07-28 02:18:54,347][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:18:59,353][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:04,363][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:09,365][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:14,366][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:19,368][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:24,370][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:29,372][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:34,373][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:39,374][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:44,375][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:49,376][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:54,281][ ERROR][PID:1708938] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-28 02:19:54,378][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:19:59,379][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:04,284][WARNING][PID:1708938] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-28 02:20:04,381][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:09,386][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:14,286][  INFO][PID:1708938] Downloading the builder-live.log file, attempt 3
[2025-07-28 02:20:14,290][  INFO][PID:1708938] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.240 copr-rpmbuild-log
[2025-07-28 02:20:14,388][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:19,389][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:24,393][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:29,396][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:34,398][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:39,399][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:44,402][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:49,403][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:54,405][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:20:59,407][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:04,408][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:09,410][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:14,435][ ERROR][PID:1708938] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-28 02:21:14,455][  INFO][PID:1708938] Periodic builder liveness probe: alive
[2025-07-28 02:21:14,459][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:19,467][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:24,437][WARNING][PID:1708938] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-07-28 02:21:24,470][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:29,474][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:34,438][  INFO][PID:1708938] Downloading the builder-live.log file, attempt 4
[2025-07-28 02:21:34,449][  INFO][PID:1708938] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.240 copr-rpmbuild-log
[2025-07-28 02:21:34,477][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:39,479][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:44,481][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:49,487][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:54,489][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:21:59,491][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:04,492][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:09,496][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:14,499][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:19,504][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:24,505][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:29,506][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:34,507][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:34,591][ ERROR][PID:1708938] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-07-28 02:22:39,509][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:44,510][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:44,595][ ERROR][PID:1708938] Giving up for unstable SSH, failures: 1753669124.14033, 1753669204.2844915, 1753669284.4377656, 1753669364.595607
[2025-07-28 02:22:44,596][ ERROR][PID:1708938] Re-try request for task on 'ResallocHost, ticket_id=4190984, hostname=140.211.168.240, name=copr_osuosl_p09_prod_05919976_20250728_020707, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-07-28 02:22:44,597][  INFO][PID:1708938] Releasing VM back to pool
[2025-07-28 02:22:44,609][  INFO][PID:1708938] Retry #1 (on other host)
[2025-07-28 02:22:44,609][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:44,610][  INFO][PID:1708938] VM allocation process starts
[2025-07-28 02:22:44,623][  INFO][PID:1708938] Trying to allocate VM: ResallocHost, ticket_id=4191277, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-28 02:22:49,624][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:54,625][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:22:59,627][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:04,628][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:09,629][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:14,630][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:19,631][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:24,632][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:29,634][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:34,635][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:39,636][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:44,637][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:49,638][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:54,639][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:23:59,640][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:04,642][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:09,643][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:14,644][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:19,645][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:24,646][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:29,647][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:34,648][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:39,649][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:44,650][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:49,651][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:54,651][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:24:59,653][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:04,654][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:09,654][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:14,655][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:19,657][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:24,658][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:29,659][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:34,660][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:39,661][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:44,662][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:49,663][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:54,664][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:25:59,665][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:04,666][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:09,669][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:14,670][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:19,671][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:24,672][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:29,673][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:34,674][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:39,675][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:44,676][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:49,677][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:54,678][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:26:59,679][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:04,680][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:09,681][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:14,682][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:19,683][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:24,684][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:29,685][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:34,686][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:39,687][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:44,688][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:49,689][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:54,690][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:27:59,691][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:04,692][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:09,693][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:14,694][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:19,695][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:24,696][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:29,697][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:34,698][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:39,699][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:44,701][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:49,701][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:54,702][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:28:59,703][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:04,704][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:09,705][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:14,706][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:19,708][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:24,709][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:29,710][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:34,711][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:39,712][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:44,713][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:49,714][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:54,715][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:29:59,716][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:04,717][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:09,718][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:14,719][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:19,720][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:24,721][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:29,723][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:34,723][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:39,724][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:44,725][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:49,726][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:54,727][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:30:59,729][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:04,729][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:09,731][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:14,732][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:19,732][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:24,733][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:29,735][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:34,736][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:39,737][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:44,738][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:49,738][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:54,740][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:31:59,741][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:04,741][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:09,742][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:14,743][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:19,745][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:24,746][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:29,747][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:34,748][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:39,749][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:44,750][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:49,751][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:54,752][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:32:59,753][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:04,754][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:09,755][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:14,756][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:19,757][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:24,758][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:29,759][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:34,760][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:39,761][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:44,762][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:49,763][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:54,765][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:33:59,766][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:04,767][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:09,768][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:14,769][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:19,770][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:24,771][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:29,772][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:34,773][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:39,774][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:44,775][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:49,776][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:54,777][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:34:59,778][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:04,779][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:09,780][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:14,782][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:19,783][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:24,784][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:29,785][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:34,786][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:39,787][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:44,788][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:49,789][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:54,790][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:35:59,791][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:02,796][  INFO][PID:1708938] Allocated host ResallocHost, ticket_id=4191277, hostname=2620:52:3:1:dead:beef:cafe:c216, name=copr_hv_ppc64le_01_prod_05919979_20250728_020725, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-07-28 02:36:02,797][  INFO][PID:1708938] Allocating ssh connection to builder
[2025-07-28 02:36:02,797][  INFO][PID:1708938] Checking that builder machine is OK
[2025-07-28 02:36:05,386][  INFO][PID:1708938] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-07-28 02:36:06,518][  INFO][PID:1708938] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-07-28 02:36:06,519][  INFO][PID:1708938] Filling build.info file with builder info
[2025-07-28 02:36:06,523][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:06,523][  INFO][PID:1708938] Not re-notifying FE and msg buses for the new host.
[2025-07-28 02:36:06,524][  INFO][PID:1708938] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9323533-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-07-28 02:36:08,237][  INFO][PID:1708938] Downloading the builder-live.log file, attempt 1
[2025-07-28 02:36:08,238][  INFO][PID:1708938] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c216 copr-rpmbuild-log
[2025-07-28 02:36:13,242][  INFO][PID:1708938] Periodic builder liveness probe: alive
[2025-07-28 02:36:13,243][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:18,243][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:23,245][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:28,246][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:33,247][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:38,248][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:43,250][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:48,251][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:53,252][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:36:58,253][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:03,254][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:08,255][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:13,256][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:18,257][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:23,258][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:28,259][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:33,260][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:38,261][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:43,262][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:48,263][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:53,264][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:37:58,265][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:03,266][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:08,267][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:13,268][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:18,269][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:23,271][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:28,272][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:33,273][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:38,274][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:43,275][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:48,276][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:53,277][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:38:58,278][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:03,279][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:08,280][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:13,281][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:18,283][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:23,284][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:28,285][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:33,286][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:38,287][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:43,288][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:48,289][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:53,290][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:39:58,291][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:03,292][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:08,293][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:13,294][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:18,295][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:23,296][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:28,297][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:33,298][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:38,299][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:43,300][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:48,301][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:53,302][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:40:58,303][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:03,304][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:08,305][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:13,709][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:18,710][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:23,711][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:28,712][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:33,714][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:38,715][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:43,716][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:48,717][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:53,718][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:41:58,720][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:03,721][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:08,722][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:13,722][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:18,723][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:23,724][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:28,726][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:33,727][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:38,728][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:43,729][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:48,730][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:53,731][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:42:58,732][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:03,733][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:08,735][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:13,736][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:18,737][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:23,738][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:28,739][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:33,741][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:38,741][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:43,743][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:48,743][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:53,745][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:43:58,746][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:03,747][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:08,748][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:13,749][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:18,750][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:23,751][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:28,752][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:33,753][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:38,754][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:43,755][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:48,756][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:53,757][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:44:58,758][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:03,759][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:08,760][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:13,761][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:18,764][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:23,765][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:28,766][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:33,767][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:38,768][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:43,769][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:48,770][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:53,771][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:45:58,772][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:03,773][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:08,774][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:13,784][  INFO][PID:1708938] Periodic builder liveness probe: alive
[2025-07-28 02:46:13,785][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:18,786][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:23,787][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:28,788][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:33,789][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:38,790][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:43,791][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:48,792][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:53,793][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:46:58,794][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:03,795][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:08,796][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:13,797][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:18,798][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:23,800][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:28,801][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:33,802][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:38,803][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:43,804][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:48,805][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:53,806][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:47:58,808][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:48:03,809][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:48:08,810][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:48:13,811][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:48:18,812][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:48:23,813][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:48:28,813][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:48:33,815][  INFO][PID:1708938] Checking for cancel request
[2025-07-28 02:48:34,150][  INFO][PID:1708938] Downloading results from builder
[2025-07-28 02:48:34,150][  INFO][PID:1708938] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c216]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome started
[2025-07-28 02:48:34,151][  INFO][PID:1708938] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c216]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/ &> /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/build-09323533.rsync.log
[2025-07-28 02:48:36,599][  INFO][PID:1708938] rsyncing finished.
[2025-07-28 02:48:36,600][  INFO][PID:1708938] Releasing VM back to pool
[2025-07-28 02:48:36,608][  INFO][PID:1708938] Searching for 'success' file in resultdir
[2025-07-28 02:48:36,609][  INFO][PID:1708938] Going to sign pkgs from source: 9323533-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le
[2025-07-28 02:48:36,609][  INFO][PID:1708938] Calling '/bin/sign -u psimovec#openblas-0.3.23@copr.fedorahosted.org -p' (attempt #1)
[2025-07-28 02:48:36,934][  INFO][PID:1708938] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/plasma-welcome-debugsource-6.4.3-2.fc43.ppc64le.rpm' (attempt #1)
[2025-07-28 02:48:37,882][  INFO][PID:1708938] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/plasma-welcome-debugsource-6.4.3-2.fc43.ppc64le.rpm
[2025-07-28 02:48:37,882][  INFO][PID:1708938] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/plasma-welcome-6.4.3-2.fc43.src.rpm' (attempt #1)
[2025-07-28 02:48:38,822][  INFO][PID:1708938] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/plasma-welcome-6.4.3-2.fc43.src.rpm
[2025-07-28 02:48:38,823][  INFO][PID:1708938] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/plasma-welcome-6.4.3-2.fc43.ppc64le.rpm' (attempt #1)
[2025-07-28 02:48:39,775][  INFO][PID:1708938] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/plasma-welcome-6.4.3-2.fc43.ppc64le.rpm
[2025-07-28 02:48:39,776][  INFO][PID:1708938] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/plasma-welcome-debuginfo-6.4.3-2.fc43.ppc64le.rpm' (attempt #1)
[2025-07-28 02:48:40,749][  INFO][PID:1708938] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/plasma-welcome-debuginfo-6.4.3-2.fc43.ppc64le.rpm
[2025-07-28 02:48:40,749][  INFO][PID:1708938] Sign done
[2025-07-28 02:48:40,750][  INFO][PID:1708938] Incremental createrepo run, adding 09323533-plasma-welcome into https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le, (auto-create-repo=True)
[2025-07-28 02:48:40,751][  INFO][PID:1708938] Running command 'copr-repo --batched /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le --add 09323533-plasma-welcome --no-appstream-metadata' as PID 1931789
[2025-07-28 02:48:51,549][  INFO][PID:1708938] Finished after 10 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le --add 09323533-plasma-welcome --no-appstream-metadata)
[2025-07-28 02:48:51,549][  INFO][PID:1708938] Getting build details
[2025-07-28 02:48:51,550][  INFO][PID:1708938] Listing built binary packages in /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome
[2025-07-28 02:48:51,550][  INFO][PID:1708938] Built packages:
['plasma-welcome-debuginfo 6.4.3', 'plasma-welcome-debugsource 6.4.3', 'plasma-welcome 6.4.3']
[2025-07-28 02:48:51,551][  INFO][PID:1708938] build details: {'built_packages': 'plasma-welcome-debuginfo 6.4.3\nplasma-welcome-debugsource 6.4.3\nplasma-welcome 6.4.3'}
[2025-07-28 02:48:51,551][  INFO][PID:1708938] Retrieving pubkey
[2025-07-28 02:48:51,552][  INFO][PID:1708938] Calling '/bin/sign -u psimovec#openblas-0.3.23@copr.fedorahosted.org -p' (attempt #1)
[2025-07-28 02:48:51,881][  INFO][PID:1708938] Added pubkey for user psimovec project openblas-0.3.23 into: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/pubkey.gpg
[2025-07-28 02:48:51,882][  INFO][PID:1708938] Finished build: id=9323533 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/openblas-0.3.23 chroot=fedora-rawhide-ppc64le 
[2025-07-28 02:48:51,883][  INFO][PID:1708938] Worker succeeded build, took 765.3597555160522
[2025-07-28 02:48:51,884][  INFO][PID:1708938] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "openblas-0.3.23",
            "project_dirname": "openblas-0.3.23",
            "submitter": "https://src.fedoraproject.org/user/releng",
            "ended_on": 1753670931.8836813,
            "started_on": 1753670166.5239258,
            "submitted_on": null,
            "status": 1,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9323533-fedora-rawhide-ppc64le",
            "build_id": 9323533,
            "package_name": "plasma-welcome",
            "package_version": "6.4.3-2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/plasma-welcome",
            "git_hash": "cf8574aff128bb40a36331e423784af92fdbb11d",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/releng",
            "results": {
                "packages": [
                    {
                        "name": "plasma-welcome-debuginfo",
                        "epoch": null,
                        "version": "6.4.3",
                        "release": "2.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "plasma-welcome-debugsource",
                        "epoch": null,
                        "version": "6.4.3",
                        "release": "2.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "plasma-welcome",
                        "epoch": null,
                        "version": "6.4.3",
                        "release": "2.fc43",
                        "arch": "src"
                    },
                    {
                        "name": "plasma-welcome",
                        "epoch": null,
                        "version": "6.4.3",
                        "release": "2.fc43",
                        "arch": "ppc64le"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23",
            "result_dir": "09323533-plasma-welcome",
            "built_packages": "plasma-welcome-debuginfo 6.4.3\nplasma-welcome-debugsource 6.4.3\nplasma-welcome 6.4.3",
            "id": 9323533,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "openblas-0.3.23",
                "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)"
            }
        }
    ]
}
[2025-07-28 02:48:51,934][  INFO][PID:1708938] Sending fedora-messaging bus message in build.end
[2025-07-28 02:48:51,957][  INFO][PID:1708938] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/builder-live.log by gzip
[2025-07-28 02:48:51,958][  INFO][PID:1708938] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/builder-live.log' as PID 1932244
[2025-07-28 02:48:51,974][  INFO][PID:1708938] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/builder-live.log)
[2025-07-28 02:48:51,974][  INFO][PID:1708938] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/backend.log by gzip
[2025-07-28 02:48:51,975][  INFO][PID:1708938] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-rawhide-ppc64le/09323533-plasma-welcome/backend.log' as PID 1932245