[2025-10-31 02:16:42,055][ INFO][PID:1329725] Marking build as starting [2025-10-31 02:16:42,156][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:16:42,162][ INFO][PID:1329725] VM allocation process starts [2025-10-31 02:16:42,226][ INFO][PID:1329725] Trying to allocate VM: ResallocHost, ticket_id=5944205, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-10-31 02:16:47,238][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:16:48,323][ INFO][PID:1329725] Allocated host ResallocHost, ticket_id=5944205, hostname=2620:52:6:1161:dead:beef:cafe:c191, name=vmhost_p09_04_prod_07745454_20251031_020236, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-10-31 02:16:48,326][ INFO][PID:1329725] Allocating ssh connection to builder [2025-10-31 02:16:48,327][ INFO][PID:1329725] Checking that builder machine is OK [2025-10-31 02:17:48,430][ ERROR][PID:1329725] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c191 port 22: Connection timed out [2025-10-31 02:17:58,431][ ERROR][PID:1329725] Re-try request for task on 'ResallocHost, ticket_id=5944205, hostname=2620:52:6:1161:dead:beef:cafe:c191, name=vmhost_p09_04_prod_07745454_20251031_020236, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts [2025-10-31 02:17:58,432][ INFO][PID:1329725] Releasing VM back to pool [2025-10-31 02:17:58,455][ INFO][PID:1329725] Retry #1 (on other host) [2025-10-31 02:17:58,457][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:17:58,458][ INFO][PID:1329725] VM allocation process starts [2025-10-31 02:17:58,478][ INFO][PID:1329725] Trying to allocate VM: ResallocHost, ticket_id=5944275, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-10-31 02:18:03,479][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:18:04,523][ INFO][PID:1329725] Allocated host ResallocHost, ticket_id=5944275, hostname=2620:52:6:1161:dead:beef:cafe:c18f, name=vmhost_p09_04_prod_07745515_20251031_020628, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-10-31 02:18:04,524][ INFO][PID:1329725] Allocating ssh connection to builder [2025-10-31 02:18:04,526][ INFO][PID:1329725] Checking that builder machine is OK [2025-10-31 02:18:05,498][ INFO][PID:1329725] Running remote command: copr-builder-ready rhel+epel-10-ppc64le [2025-10-31 02:19:05,589][ ERROR][PID:1329725] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18f port 22: Connection timed out [2025-10-31 02:19:15,591][ ERROR][PID:1329725] Re-try request for task on 'ResallocHost, ticket_id=5944275, hostname=2620:52:6:1161:dead:beef:cafe:c18f, name=vmhost_p09_04_prod_07745515_20251031_020628, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts [2025-10-31 02:19:15,592][ INFO][PID:1329725] Releasing VM back to pool [2025-10-31 02:19:15,613][ INFO][PID:1329725] Retry #2 (on other host) [2025-10-31 02:19:15,614][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:15,615][ INFO][PID:1329725] VM allocation process starts [2025-10-31 02:19:15,634][ INFO][PID:1329725] Trying to allocate VM: ResallocHost, ticket_id=5944297, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-10-31 02:19:20,637][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:21,665][ INFO][PID:1329725] Allocated host ResallocHost, ticket_id=5944297, hostname=2620:52:6:1161:dead:beef:cafe:c149, name=vmhost_p09_03_prod_07745519_20251031_020634, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-10-31 02:19:21,666][ INFO][PID:1329725] Allocating ssh connection to builder [2025-10-31 02:19:21,667][ INFO][PID:1329725] Checking that builder machine is OK [2025-10-31 02:19:22,705][ INFO][PID:1329725] Running remote command: copr-builder-ready rhel+epel-10-ppc64le [2025-10-31 02:19:23,247][ INFO][PID:1329725] Red Hat subscription not needed for rhel+epel-10-ppc64le Builder is ready to be used [2025-10-31 02:19:23,248][ INFO][PID:1329725] Filling build.info file with builder info [2025-10-31 02:19:23,249][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:23,250][ INFO][PID:1329725] Not re-notifying FE and msg buses for the new host. [2025-10-31 02:19:23,251][ INFO][PID:1329725] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9751214-rhel+epel-10-ppc64le --chroot rhel+epel-10-ppc64le --detached [2025-10-31 02:19:24,176][ INFO][PID:1329725] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c149' (ED25519) to the list of known hosts. [2025-10-31 02:19:24,177][ INFO][PID:1329725] Downloading the builder-live.log file, attempt 1 [2025-10-31 02:19:24,179][ INFO][PID:1329725] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c149 copr-rpmbuild-log [2025-10-31 02:19:29,187][ INFO][PID:1329725] Periodic builder liveness probe: alive [2025-10-31 02:19:29,188][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:34,189][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:39,190][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:44,194][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:49,195][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:54,200][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:19:59,205][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:04,207][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:09,210][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:14,213][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:19,214][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:24,219][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:29,220][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:34,224][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:39,226][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:44,228][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:49,235][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:54,237][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:20:59,240][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:04,241][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:09,242][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:14,246][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:19,248][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:24,257][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:29,259][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:34,262][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:39,265][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:44,272][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:49,275][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:54,278][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:21:59,280][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:04,282][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:09,284][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:14,285][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:19,288][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:24,290][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:29,302][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:34,303][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:39,306][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:44,312][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:49,324][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:54,326][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:22:59,331][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:04,333][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:09,337][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:14,346][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:19,351][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:24,353][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:29,359][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:34,363][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:39,365][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:44,366][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:49,370][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:54,375][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:23:59,379][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:24:04,380][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:24:09,390][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:24:14,394][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:24:19,395][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:24:24,397][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:24:29,399][ INFO][PID:1329725] Checking for cancel request [2025-10-31 02:24:30,990][ INFO][PID:1329725] Downloading results from builder [2025-10-31 02:24:30,991][ INFO][PID:1329725] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c149]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn started [2025-10-31 02:24:30,995][ INFO][PID:1329725] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c149]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/ &> /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/build-09751214.rsync.log [2025-10-31 02:24:32,092][ INFO][PID:1329725] rsyncing finished. [2025-10-31 02:24:32,099][ INFO][PID:1329725] Releasing VM back to pool [2025-10-31 02:24:32,167][ INFO][PID:1329725] Searching for 'success' file in resultdir [2025-10-31 02:24:32,169][ INFO][PID:1329725] Going to sign pkgs from source: 9751214-rhel+epel-10-ppc64le in chroot: /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le [2025-10-31 02:24:32,172][ INFO][PID:1329725] Calling '/bin/sign -u @OpenVPN#openvpn-git@copr.fedorahosted.org -p' (attempt #1) [2025-10-31 02:24:32,524][ INFO][PID:1329725] Calling '/bin/sign -4 -h sha256 -u @OpenVPN#openvpn-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-2.7-20251030gitfdd4072.el10.src.rpm' (attempt #1) [2025-10-31 02:24:33,292][ INFO][PID:1329725] signed rpm: /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-2.7-20251030gitfdd4072.el10.src.rpm [2025-10-31 02:24:33,294][ INFO][PID:1329725] Calling '/bin/sign -4 -h sha256 -u @OpenVPN#openvpn-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-2.7-20251030gitfdd4072.el10.ppc64le.rpm' (attempt #1) [2025-10-31 02:24:34,376][ INFO][PID:1329725] signed rpm: /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-2.7-20251030gitfdd4072.el10.ppc64le.rpm [2025-10-31 02:24:34,378][ INFO][PID:1329725] Calling '/bin/sign -4 -h sha256 -u @OpenVPN#openvpn-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-debugsource-2.7-20251030gitfdd4072.el10.ppc64le.rpm' (attempt #1) [2025-10-31 02:24:35,447][ INFO][PID:1329725] signed rpm: /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-debugsource-2.7-20251030gitfdd4072.el10.ppc64le.rpm [2025-10-31 02:24:35,448][ INFO][PID:1329725] Calling '/bin/sign -4 -h sha256 -u @OpenVPN#openvpn-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-devel-2.7-20251030gitfdd4072.el10.ppc64le.rpm' (attempt #1) [2025-10-31 02:24:36,737][ INFO][PID:1329725] signed rpm: /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-devel-2.7-20251030gitfdd4072.el10.ppc64le.rpm [2025-10-31 02:24:36,737][ INFO][PID:1329725] Calling '/bin/sign -4 -h sha256 -u @OpenVPN#openvpn-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-debuginfo-2.7-20251030gitfdd4072.el10.ppc64le.rpm' (attempt #1) [2025-10-31 02:24:37,517][ INFO][PID:1329725] signed rpm: /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/openvpn-debuginfo-2.7-20251030gitfdd4072.el10.ppc64le.rpm [2025-10-31 02:24:37,520][ INFO][PID:1329725] Sign done [2025-10-31 02:24:37,522][ INFO][PID:1329725] Incremental createrepo run, adding 09751214-openvpn into https://download.copr.fedorainfracloud.org/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le, (auto-create-repo=True) [2025-10-31 02:24:37,524][ INFO][PID:1329725] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le --add 09751214-openvpn --no-appstream-metadata' as PID 1396245 [2025-10-31 02:24:39,083][ INFO][PID:1329725] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le --add 09751214-openvpn --no-appstream-metadata) [2025-10-31 02:24:39,084][ INFO][PID:1329725] Getting build details [2025-10-31 02:24:39,084][ INFO][PID:1329725] Listing built binary packages in /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn [2025-10-31 02:24:39,086][ INFO][PID:1329725] Built packages: ['openvpn-debuginfo 2.7', 'openvpn-devel 2.7', 'openvpn 2.7', 'openvpn-debugsource 2.7'] [2025-10-31 02:24:39,086][ INFO][PID:1329725] build details: {'built_packages': 'openvpn-debuginfo 2.7\nopenvpn-devel 2.7\nopenvpn 2.7\nopenvpn-debugsource 2.7'} [2025-10-31 02:24:39,087][ INFO][PID:1329725] Retrieving pubkey [2025-10-31 02:24:39,087][ INFO][PID:1329725] Calling '/bin/sign -u @OpenVPN#openvpn-git@copr.fedorahosted.org -p' (attempt #1) [2025-10-31 02:24:39,277][ INFO][PID:1329725] Added pubkey for user @OpenVPN project openvpn-git into: /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/pubkey.gpg [2025-10-31 02:24:39,279][ INFO][PID:1329725] Finished build: id=9751214 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@OpenVPN/openvpn-git chroot=rhel+epel-10-ppc64le [2025-10-31 02:24:39,287][ INFO][PID:1329725] Worker succeeded build, took 316.0364475250244 [2025-10-31 02:24:39,290][ INFO][PID:1329725] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@OpenVPN", "project_name": "openvpn-git", "project_dirname": "openvpn-git", "submitter": "gitlab.com:flichtenheld", "ended_on": 1761877479.2870386, "started_on": 1761877163.250591, "submitted_on": null, "status": 1, "chroot": "rhel+epel-10-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9751214-rhel+epel-10-ppc64le", "build_id": 9751214, "package_name": "openvpn", "package_version": "2.7-20251030gitfdd4072", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@OpenVPN/openvpn-git/openvpn", "git_hash": "187a9f8634c13353625f2b1263387b506580ffc9", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@OpenVPN/openvpn-git--gitlab.com:flichtenheld", "results": { "packages": [ { "name": "openvpn-debuginfo", "epoch": null, "version": "2.7", "release": "20251030gitfdd4072.el10", "arch": "ppc64le" }, { "name": "openvpn", "epoch": null, "version": "2.7", "release": "20251030gitfdd4072.el10", "arch": "src" }, { "name": "openvpn-devel", "epoch": null, "version": "2.7", "release": "20251030gitfdd4072.el10", "arch": "ppc64le" }, { "name": "openvpn", "epoch": null, "version": "2.7", "release": "20251030gitfdd4072.el10", "arch": "ppc64le" }, { "name": "openvpn-debugsource", "epoch": null, "version": "2.7", "release": "20251030gitfdd4072.el10", "arch": "ppc64le" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le", "arch_power9" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@OpenVPN/openvpn-git", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@OpenVPN/openvpn-git", "result_dir": "09751214-openvpn", "built_packages": "openvpn-debuginfo 2.7\nopenvpn-devel 2.7\nopenvpn 2.7\nopenvpn-debugsource 2.7", "id": 9751214, "mockchain_macros": { "copr_username": "@OpenVPN", "copr_projectname": "openvpn-git", "vendor": "Fedora Project COPR (@OpenVPN/openvpn-git)" } } ] } [2025-10-31 02:24:39,374][ INFO][PID:1329725] Sending fedora-messaging bus message in build.end [2025-10-31 02:24:40,760][ INFO][PID:1329725] Compressing /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/builder-live.log by gzip [2025-10-31 02:24:40,770][ INFO][PID:1329725] Running command 'gzip /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/builder-live.log' as PID 1396767 [2025-10-31 02:24:40,803][ INFO][PID:1329725] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/builder-live.log) [2025-10-31 02:24:40,806][ INFO][PID:1329725] Compressing /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/backend.log by gzip [2025-10-31 02:24:40,810][ INFO][PID:1329725] Running command 'gzip /var/lib/copr/public_html/results/@OpenVPN/openvpn-git/rhel+epel-10-ppc64le/09751214-openvpn/backend.log' as PID 1396779