[2025-09-02 22:23:56,060][ INFO][PID:2320534] Marking build as starting [2025-09-02 22:23:56,273][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:23:56,274][ INFO][PID:2320534] VM allocation process starts [2025-09-02 22:23:56,288][ INFO][PID:2320534] Trying to allocate VM: ResallocHost, ticket_id=4882068, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-09-02 22:24:01,292][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:06,293][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:08,348][ INFO][PID:2320534] Allocated host ResallocHost, ticket_id=4882068, hostname=140.211.168.35, name=copr_osuosl_p09_prod_06542233_20250902_143718, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-09-02 22:24:08,350][ INFO][PID:2320534] Allocating ssh connection to builder [2025-09-02 22:24:08,350][ INFO][PID:2320534] Checking that builder machine is OK [2025-09-02 22:24:10,212][ INFO][PID:2320534] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2025-09-02 22:24:11,244][ INFO][PID:2320534] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2025-09-02 22:24:11,245][ INFO][PID:2320534] Filling build.info file with builder info [2025-09-02 22:24:11,246][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:11,248][ INFO][PID:2320534] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "scipy-1.8.3", "project_dirname": "scipy-1.8.3", "submitter": "https://src.fedoraproject.org/user/music", "ended_on": null, "started_on": 1756851851.248075, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9516867-fedora-rawhide-ppc64le", "build_id": 9516867, "package_name": "python-trimesh", "package_version": "4.8.1-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/scipy-1.8.3/python-trimesh", "git_hash": "85a82126f70063d271482cf145630987156f0129", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/scipy-1.8.3--https://src.fedoraproject.org/user/music", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/scipy-1.8.3", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.8.3", "result_dir": "09516867-python-trimesh", "built_packages": "", "id": 9516867, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "scipy-1.8.3", "vendor": "Fedora Project COPR (psimovec/scipy-1.8.3)" } } ] } [2025-09-02 22:24:11,342][ INFO][PID:2320534] Sending fedora-messaging bus message in build.start [2025-09-02 22:24:12,187][ INFO][PID:2320534] Sending fedora-messaging bus message in chroot.start [2025-09-02 22:24:12,212][ INFO][PID:2320534] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9516867-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2025-09-02 22:24:13,507][ INFO][PID:2320534] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '140.211.168.35' (ED25519) to the list of known hosts. [2025-09-02 22:24:13,507][ INFO][PID:2320534] Downloading the builder-live.log file, attempt 1 [2025-09-02 22:24:13,509][ INFO][PID:2320534] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.35 copr-rpmbuild-log [2025-09-02 22:24:18,517][ INFO][PID:2320534] Periodic builder liveness probe: alive [2025-09-02 22:24:18,518][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:23,519][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:28,520][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:33,521][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:38,522][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:43,523][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:48,525][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:53,531][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:24:58,532][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:03,534][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:08,535][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:13,536][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:18,536][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:23,538][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:28,540][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:33,542][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:38,543][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:43,545][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:48,546][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:53,547][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:25:58,549][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:03,550][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:08,552][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:13,553][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:18,556][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:23,558][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:28,560][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:33,561][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:38,562][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:43,563][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:48,564][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:53,565][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:26:58,566][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:03,567][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:08,568][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:13,569][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:18,570][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:23,571][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:28,572][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:33,573][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:38,574][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:43,575][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:48,576][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:53,577][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:27:58,578][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:03,579][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:08,580][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:13,581][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:18,582][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:23,583][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:28,584][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:33,585][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:38,586][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:43,587][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:48,588][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:53,589][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:28:58,590][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:03,591][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:08,592][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:13,593][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:18,594][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:23,595][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:28,596][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:33,598][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:38,599][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:43,600][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:48,601][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:53,602][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:29:58,603][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:03,604][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:08,605][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:13,606][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:18,608][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:23,609][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:28,610][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:33,612][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:38,615][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:43,616][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:48,618][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:53,619][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:30:58,620][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:03,621][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:08,623][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:13,624][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:18,625][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:23,626][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:28,627][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:33,628][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:38,629][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:43,630][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:48,631][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:53,633][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:31:58,634][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:03,635][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:08,636][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:13,637][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:18,638][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:23,645][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:28,646][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:33,647][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:38,648][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:43,650][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:48,650][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:53,652][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:32:58,652][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:03,653][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:08,656][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:13,657][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:18,658][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:23,659][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:28,660][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:33,661][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:38,663][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:43,668][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:48,678][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:53,679][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:33:58,680][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:03,682][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:08,686][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:13,688][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:18,734][ INFO][PID:2320534] Periodic builder liveness probe: alive [2025-09-02 22:34:18,735][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:23,759][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:28,761][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:33,764][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:38,771][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:43,779][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:48,783][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:53,788][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:34:58,790][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:03,791][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:08,796][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:13,799][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:18,800][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:23,805][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:28,808][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:33,815][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:38,819][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:43,834][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:48,841][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:53,847][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:35:58,848][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:03,849][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:08,852][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:13,854][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:18,855][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:23,856][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:28,863][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:33,864][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:38,865][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:43,868][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:48,870][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:53,874][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:36:58,879][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:03,880][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:08,884][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:13,889][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:18,891][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:23,894][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:28,896][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:33,897][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:38,899][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:43,901][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:48,903][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:53,905][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:37:58,906][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:03,908][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:08,910][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:13,912][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:18,913][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:23,914][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:28,917][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:33,918][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:38,919][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:43,920][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:48,922][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:53,923][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:38:58,924][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:03,925][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:08,926][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:13,927][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:18,928][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:23,929][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:28,930][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:33,931][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:38,932][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:43,934][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:48,935][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:53,936][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:39:58,937][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:03,938][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:08,939][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:13,940][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:18,941][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:23,942][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:28,943][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:33,945][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:38,946][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:43,947][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:48,949][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:53,950][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:40:58,951][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:03,953][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:08,954][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:13,955][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:18,956][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:23,957][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:28,958][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:33,962][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:38,964][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:43,965][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:48,966][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:53,967][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:41:58,967][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:03,968][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:08,970][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:13,970][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:18,971][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:23,972][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:28,973][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:33,974][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:38,975][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:43,976][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:48,977][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:53,978][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:42:58,979][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:03,980][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:08,981][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:13,982][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:18,983][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:23,984][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:28,985][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:33,986][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:38,987][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:43,988][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:48,989][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:53,990][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:43:58,990][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:03,991][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:08,992][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:13,993][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:18,999][ INFO][PID:2320534] Periodic builder liveness probe: alive [2025-09-02 22:44:19,000][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:24,001][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:29,002][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:34,003][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:39,004][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:44,005][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:49,006][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:54,007][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:44:59,008][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:04,009][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:09,010][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:14,011][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:19,012][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:24,013][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:29,014][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:34,014][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:39,015][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:44,016][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:49,017][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:54,018][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:45:59,019][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:04,020][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:09,021][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:14,022][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:19,023][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:24,024][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:29,025][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:34,026][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:39,027][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:44,028][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:49,029][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:54,030][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:46:59,032][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:04,033][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:09,034][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:14,035][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:19,036][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:24,037][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:29,038][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:34,039][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:39,040][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:44,041][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:49,042][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:54,043][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:47:59,044][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:04,045][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:09,046][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:14,047][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:19,048][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:24,049][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:29,050][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:34,051][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:39,052][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:44,053][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:49,054][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:54,054][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:48:59,055][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:04,057][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:09,058][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:14,059][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:19,060][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:24,060][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:29,061][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:34,062][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:39,063][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:44,064][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:49,065][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:54,066][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:49:59,067][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:04,068][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:09,069][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:14,070][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:19,071][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:24,072][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:29,073][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:34,074][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:39,075][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:44,076][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:49,077][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:54,078][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:50:59,079][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:04,080][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:09,081][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:14,082][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:19,083][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:24,084][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:29,085][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:34,086][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:39,088][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:44,088][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:49,089][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:54,090][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:51:59,092][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:04,093][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:09,093][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:14,094][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:19,095][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:24,096][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:29,097][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:34,099][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:39,100][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:44,101][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:49,102][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:54,103][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:52:59,104][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:04,105][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:09,106][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:14,108][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:19,110][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:24,112][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:29,113][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:34,114][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:39,115][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:44,116][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:49,117][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:54,118][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:53:59,120][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:54:04,121][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:54:09,124][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:54:14,126][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:54:19,139][ INFO][PID:2320534] Periodic builder liveness probe: dead [2025-09-02 22:54:19,140][ INFO][PID:2320534] calling cancel callback, and waiting [2025-09-02 22:55:19,209][ ERROR][PID:2320534] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 140.211.168.35 port 22: Connection timed out [2025-09-02 22:55:35,140][ ERROR][PID:2320534] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-09-02 22:55:45,141][WARNING][PID:2320534] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-09-02 22:55:55,141][ INFO][PID:2320534] Downloading the builder-live.log file, attempt 2 [2025-09-02 22:55:55,143][ INFO][PID:2320534] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.35 copr-rpmbuild-log [2025-09-02 22:56:29,279][ ERROR][PID:2320534] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 140.211.168.35 port 22: Connection timed out [2025-09-02 22:56:55,211][ ERROR][PID:2320534] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-09-02 22:57:05,214][WARNING][PID:2320534] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-09-02 22:57:15,214][ INFO][PID:2320534] Downloading the builder-live.log file, attempt 3 [2025-09-02 22:57:15,215][ INFO][PID:2320534] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.35 copr-rpmbuild-log [2025-09-02 22:57:39,348][ ERROR][PID:2320534] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 140.211.168.35 port 22: Connection timed out [2025-09-02 22:58:15,283][ ERROR][PID:2320534] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-09-02 22:58:25,284][WARNING][PID:2320534] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-09-02 22:58:35,285][ INFO][PID:2320534] Downloading the builder-live.log file, attempt 4 [2025-09-02 22:58:35,286][ INFO][PID:2320534] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.35 copr-rpmbuild-log [2025-09-02 22:58:49,417][ ERROR][PID:2320534] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 140.211.168.35 port 22: Connection timed out [2025-09-02 22:58:59,417][ ERROR][PID:2320534] Can't ssh to cancel build. [2025-09-02 22:59:35,354][ ERROR][PID:2320534] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-09-02 22:59:45,360][ ERROR][PID:2320534] Giving up for unstable SSH, failures: 1756853745.1409414, 1756853825.2139838, 1756853905.284774, 1756853985.3602786 [2025-09-02 22:59:45,361][ ERROR][PID:2320534] Re-try request for task on 'ResallocHost, ticket_id=4882068, hostname=140.211.168.35, name=copr_osuosl_p09_prod_06542233_20250902_143718, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED [2025-09-02 22:59:45,362][ INFO][PID:2320534] Releasing VM back to pool [2025-09-02 22:59:45,374][ INFO][PID:2320534] Retry #1 (on other host) [2025-09-02 22:59:45,375][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:59:45,376][ INFO][PID:2320534] VM allocation process starts [2025-09-02 22:59:45,405][ INFO][PID:2320534] Trying to allocate VM: ResallocHost, ticket_id=4882402, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-09-02 22:59:48,442][ INFO][PID:2320534] Allocated host ResallocHost, ticket_id=4882402, hostname=2620:52:3:1:dead:beef:cafe:c20a, name=copr_hv_ppc64le_01_prod_06547918_20250902_223854, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-09-02 22:59:48,443][ INFO][PID:2320534] Allocating ssh connection to builder [2025-09-02 22:59:48,443][ INFO][PID:2320534] Checking that builder machine is OK [2025-09-02 22:59:49,144][ INFO][PID:2320534] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2025-09-02 22:59:49,510][ INFO][PID:2320534] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2025-09-02 22:59:49,511][ INFO][PID:2320534] Filling build.info file with builder info [2025-09-02 22:59:49,513][ INFO][PID:2320534] Checking for cancel request [2025-09-02 22:59:49,513][ INFO][PID:2320534] Not re-notifying FE and msg buses for the new host. [2025-09-02 22:59:49,514][ INFO][PID:2320534] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9516867-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2025-09-02 22:59:50,094][ INFO][PID:2320534] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c20a' (ED25519) to the list of known hosts. [2025-09-02 22:59:50,094][ INFO][PID:2320534] Downloading the builder-live.log file, attempt 1 [2025-09-02 22:59:50,095][ INFO][PID:2320534] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c20a copr-rpmbuild-log [2025-09-02 22:59:55,095][ INFO][PID:2320534] calling cancel callback, and waiting [2025-09-02 22:59:58,734][ INFO][PID:2320534] Cancel request succeeded out: success err: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c20a' (ED25519) to the list of known hosts. [2025-09-02 22:59:58,735][ INFO][PID:2320534] Downloading results from builder [2025-09-02 22:59:58,735][ INFO][PID:2320534] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c20a]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/09516867-python-trimesh started [2025-09-02 22:59:58,736][ INFO][PID:2320534] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c20a]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/09516867-python-trimesh/ &> /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/09516867-python-trimesh/build-09516867.rsync.log [2025-09-02 22:59:59,164][ INFO][PID:2320534] rsyncing finished. [2025-09-02 22:59:59,165][ INFO][PID:2320534] Releasing VM back to pool [2025-09-02 22:59:59,174][ INFO][PID:2320534] Searching for 'success' file in resultdir [2025-09-02 22:59:59,175][ ERROR][PID:2320534] Build failed: Backend process error: No success file => build failure [2025-09-02 22:59:59,176][ INFO][PID:2320534] Finished build: id=9516867 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/scipy-1.8.3 chroot=fedora-rawhide-ppc64le [2025-09-02 22:59:59,177][ ERROR][PID:2320534] Backend process error: No success file => build failure [2025-09-02 22:59:59,178][ INFO][PID:2320534] Worker failed build, took 9.664251565933228 [2025-09-02 22:59:59,178][ INFO][PID:2320534] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "scipy-1.8.3", "project_dirname": "scipy-1.8.3", "submitter": "https://src.fedoraproject.org/user/music", "ended_on": 1756853999.177977, "started_on": 1756853989.5137255, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9516867-fedora-rawhide-ppc64le", "build_id": 9516867, "package_name": "python-trimesh", "package_version": "4.8.1-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/scipy-1.8.3/python-trimesh", "git_hash": "85a82126f70063d271482cf145630987156f0129", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/scipy-1.8.3--https://src.fedoraproject.org/user/music", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/scipy-1.8.3", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.8.3", "result_dir": "09516867-python-trimesh", "built_packages": "", "id": 9516867, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "scipy-1.8.3", "vendor": "Fedora Project COPR (psimovec/scipy-1.8.3)" } } ] } [2025-09-02 22:59:59,215][ INFO][PID:2320534] Sending fedora-messaging bus message in build.end [2025-09-02 22:59:59,239][ INFO][PID:2320534] Compressing /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/09516867-python-trimesh/builder-live.log by gzip [2025-09-02 22:59:59,240][ INFO][PID:2320534] Running command 'gzip /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/09516867-python-trimesh/builder-live.log' as PID 2545764 [2025-09-02 22:59:59,242][ INFO][PID:2320534] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/09516867-python-trimesh/builder-live.log) [2025-09-02 22:59:59,243][ INFO][PID:2320534] Compressing /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/09516867-python-trimesh/backend.log by gzip [2025-09-02 22:59:59,244][ INFO][PID:2320534] Running command 'gzip /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-rawhide-ppc64le/09516867-python-trimesh/backend.log' as PID 2545765