[2025-10-30 20:21:58,676][ INFO][PID:3777470] Marking build as starting [2025-10-30 20:21:58,735][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:21:58,736][ INFO][PID:3777470] VM allocation process starts [2025-10-30 20:21:58,753][ INFO][PID:3777470] Trying to allocate VM: ResallocHost, ticket_id=5939372, requested_tags=['copr_builder', 'arch_s390x'] [2025-10-30 20:22:01,772][ INFO][PID:3777470] Allocated host ResallocHost, ticket_id=5939372, hostname=163.107.84.59, name=copr_ic_s390x_br_sao_2_prod_07741836_20251030_195309, requested_tags=['copr_builder', 'arch_s390x'] [2025-10-30 20:22:01,772][ INFO][PID:3777470] Allocating ssh connection to builder [2025-10-30 20:22:01,773][ INFO][PID:3777470] Checking that builder machine is OK [2025-10-30 20:23:04,072][ ERROR][PID:3777470] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: Connection timed out during banner exchange Connection to 163.107.84.59 port 22 timed out [2025-10-30 20:23:14,072][ ERROR][PID:3777470] Re-try request for task on 'ResallocHost, ticket_id=5939372, hostname=163.107.84.59, name=copr_ic_s390x_br_sao_2_prod_07741836_20251030_195309, requested_tags=['copr_builder', 'arch_s390x']': Unable to finish after 1 SSH attempts [2025-10-30 20:23:14,073][ INFO][PID:3777470] Releasing VM back to pool [2025-10-30 20:23:14,083][ INFO][PID:3777470] Retry #1 (on other host) [2025-10-30 20:23:14,084][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:23:14,084][ INFO][PID:3777470] VM allocation process starts [2025-10-30 20:23:14,096][ INFO][PID:3777470] Trying to allocate VM: ResallocHost, ticket_id=5939377, requested_tags=['copr_builder', 'arch_s390x'] [2025-10-30 20:23:17,110][ INFO][PID:3777470] Allocated host ResallocHost, ticket_id=5939377, hostname=13.120.92.135, name=copr_ic_s390x_eu_es_1_prod_07742179_20251030_201938, requested_tags=['copr_builder', 'arch_s390x'] [2025-10-30 20:23:17,110][ INFO][PID:3777470] Allocating ssh connection to builder [2025-10-30 20:23:17,111][ INFO][PID:3777470] Checking that builder machine is OK [2025-10-30 20:23:18,537][ INFO][PID:3777470] Running remote command: copr-builder-ready fedora-41-s390x [2025-10-30 20:23:19,844][ INFO][PID:3777470] Red Hat subscription not needed for fedora-41-s390x Builder is ready to be used [2025-10-30 20:23:19,844][ INFO][PID:3777470] Filling build.info file with builder info [2025-10-30 20:23:19,845][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:23:19,845][ INFO][PID:3777470] Not re-notifying FE and msg buses for the new host. [2025-10-30 20:23:19,846][ INFO][PID:3777470] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9747797-fedora-41-s390x --chroot fedora-41-s390x --detached [2025-10-30 20:23:21,310][ INFO][PID:3777470] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '13.120.92.135' (ED25519) to the list of known hosts. [2025-10-30 20:23:21,311][ INFO][PID:3777470] Downloading the builder-live.log file, attempt 1 [2025-10-30 20:23:21,313][ INFO][PID:3777470] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@13.120.92.135 copr-rpmbuild-log [2025-10-30 20:23:26,319][ INFO][PID:3777470] Periodic builder liveness probe: alive [2025-10-30 20:23:26,320][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:23:31,321][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:23:36,322][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:23:41,323][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:23:46,324][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:23:51,325][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:23:56,326][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:01,326][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:06,328][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:11,331][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:16,333][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:21,334][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:26,335][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:31,336][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:36,336][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:41,338][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:46,339][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:51,339][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:24:56,340][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:01,342][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:06,343][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:11,344][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:16,345][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:21,346][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:26,348][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:31,349][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:36,350][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:41,352][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:46,353][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:51,354][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:25:56,355][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:01,356][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:06,358][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:11,359][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:16,360][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:21,361][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:26,363][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:31,364][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:36,365][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:41,366][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:46,367][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:51,368][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:26:56,369][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:01,370][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:06,372][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:11,373][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:16,374][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:21,375][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:26,376][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:31,377][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:36,377][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:41,379][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:46,380][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:51,381][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:27:56,382][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:01,384][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:06,386][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:11,387][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:16,388][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:21,389][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:26,390][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:31,392][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:36,393][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:41,394][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:46,394][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:51,396][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:28:56,397][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:01,398][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:06,398][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:11,399][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:16,400][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:21,401][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:26,402][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:31,403][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:36,404][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:41,405][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:46,406][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:51,408][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:29:56,409][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:01,410][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:06,411][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:11,412][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:16,413][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:21,413][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:26,414][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:31,415][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:36,416][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:41,417][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:46,418][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:51,419][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:30:56,420][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:01,421][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:06,422][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:11,423][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:16,424][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:21,425][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:26,426][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:31,427][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:36,428][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:41,429][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:46,430][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:51,431][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:31:56,432][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:32:01,433][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:32:06,434][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:32:11,435][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:32:16,435][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:32:21,436][ INFO][PID:3777470] Checking for cancel request [2025-10-30 20:32:23,977][ INFO][PID:3777470] Downloading results from builder [2025-10-30 20:32:23,978][ INFO][PID:3777470] rsyncing of mockbuilder@13.120.92.135:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11 started [2025-10-30 20:32:23,979][ INFO][PID:3777470] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@13.120.92.135:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/ &> /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/build-09747797.rsync.log [2025-10-30 20:32:26,357][ INFO][PID:3777470] rsyncing finished. [2025-10-30 20:32:26,358][ INFO][PID:3777470] Releasing VM back to pool [2025-10-30 20:32:26,366][ INFO][PID:3777470] Searching for 'success' file in resultdir [2025-10-30 20:32:26,366][ INFO][PID:3777470] Going to sign pkgs from source: 9747797-fedora-41-s390x in chroot: /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x [2025-10-30 20:32:26,367][ INFO][PID:3777470] Calling '/bin/sign -u psimovec#scipy-1.8.3@copr.fedorahosted.org -p' (attempt #1) [2025-10-30 20:32:26,551][ INFO][PID:3777470] Calling '/bin/sign -4 -h sha256 -u psimovec#scipy-1.8.3@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/python3-pybind11-3.0-2.fc44.s390x.rpm' (attempt #1) [2025-10-30 20:32:27,357][ INFO][PID:3777470] signed rpm: /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/python3-pybind11-3.0-2.fc44.s390x.rpm [2025-10-30 20:32:27,358][ INFO][PID:3777470] Calling '/bin/sign -4 -h sha256 -u psimovec#scipy-1.8.3@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/pybind11-devel-3.0-2.fc44.s390x.rpm' (attempt #1) [2025-10-30 20:32:28,187][ INFO][PID:3777470] signed rpm: /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/pybind11-devel-3.0-2.fc44.s390x.rpm [2025-10-30 20:32:28,188][ INFO][PID:3777470] Calling '/bin/sign -4 -h sha256 -u psimovec#scipy-1.8.3@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/pybind11-3.0-2.fc44.src.rpm' (attempt #1) [2025-10-30 20:32:28,848][ INFO][PID:3777470] signed rpm: /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/pybind11-3.0-2.fc44.src.rpm [2025-10-30 20:32:28,849][ INFO][PID:3777470] Sign done [2025-10-30 20:32:28,849][ INFO][PID:3777470] Incremental createrepo run, adding 09747797-pybind11 into https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.8.3/fedora-41-s390x, (auto-create-repo=True) [2025-10-30 20:32:28,850][ INFO][PID:3777470] Running command 'copr-repo --batched /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x --add 09747797-pybind11 --no-appstream-metadata' as PID 3838174 [2025-10-30 20:32:30,600][ INFO][PID:3777470] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x --add 09747797-pybind11 --no-appstream-metadata) [2025-10-30 20:32:30,601][ INFO][PID:3777470] Getting build details [2025-10-30 20:32:30,601][ INFO][PID:3777470] Listing built binary packages in /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11 [2025-10-30 20:32:30,602][ INFO][PID:3777470] Built packages: ['python3-pybind11 3.0', 'pybind11-devel 3.0'] [2025-10-30 20:32:30,602][ INFO][PID:3777470] build details: {'built_packages': 'python3-pybind11 3.0\npybind11-devel 3.0'} [2025-10-30 20:32:30,602][ INFO][PID:3777470] Retrieving pubkey [2025-10-30 20:32:30,603][ INFO][PID:3777470] Calling '/bin/sign -u psimovec#scipy-1.8.3@copr.fedorahosted.org -p' (attempt #1) [2025-10-30 20:32:30,788][ INFO][PID:3777470] Added pubkey for user psimovec project scipy-1.8.3 into: /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/pubkey.gpg [2025-10-30 20:32:30,789][ INFO][PID:3777470] Finished build: id=9747797 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/scipy-1.8.3 chroot=fedora-41-s390x [2025-10-30 20:32:30,790][ INFO][PID:3777470] Worker succeeded build, took 550.9448852539062 [2025-10-30 20:32:30,791][ INFO][PID:3777470] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "scipy-1.8.3", "project_dirname": "scipy-1.8.3", "submitter": "https://src.fedoraproject.org/user/jussilehtola", "ended_on": 1761856350.7908022, "started_on": 1761855799.845917, "submitted_on": null, "status": 1, "chroot": "fedora-41-s390x", "arch": "s390x", "buildroot_pkgs": [], "task_id": "9747797-fedora-41-s390x", "build_id": 9747797, "package_name": "pybind11", "package_version": "3.0-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/scipy-1.8.3/pybind11", "git_hash": "26b98a04e240d1de8770ed27a5ec842b429ff19d", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/scipy-1.8.3--https://src.fedoraproject.org/user/jussilehtola", "results": { "packages": [ { "name": "python3-pybind11", "epoch": null, "version": "3.0", "release": "2.fc44", "arch": "s390x" }, { "name": "pybind11-devel", "epoch": null, "version": "3.0", "release": "2.fc44", "arch": "s390x" }, { "name": "pybind11", "epoch": null, "version": "3.0", "release": "2.fc44", "arch": "src" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.8.3/fedora-41-s390x/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_s390x" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/scipy-1.8.3", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/scipy-1.8.3", "result_dir": "09747797-pybind11", "built_packages": "python3-pybind11 3.0\npybind11-devel 3.0", "id": 9747797, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "scipy-1.8.3", "vendor": "Fedora Project COPR (psimovec/scipy-1.8.3)" } } ] } [2025-10-30 20:32:30,837][ INFO][PID:3777470] Sending fedora-messaging bus message in build.end [2025-10-30 20:32:31,448][ INFO][PID:3777470] Compressing /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/builder-live.log by gzip [2025-10-30 20:32:31,450][ INFO][PID:3777470] Running command 'gzip /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/builder-live.log' as PID 3838282 [2025-10-30 20:32:31,460][ INFO][PID:3777470] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/builder-live.log) [2025-10-30 20:32:31,461][ INFO][PID:3777470] Compressing /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/backend.log by gzip [2025-10-30 20:32:31,461][ INFO][PID:3777470] Running command 'gzip /var/lib/copr/public_html/results/psimovec/scipy-1.8.3/fedora-41-s390x/09747797-pybind11/backend.log' as PID 3838283