[2025-09-27 10:13:10,819][ INFO][PID:2051600] Marking build as starting [2025-09-27 10:13:10,922][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:10,924][ INFO][PID:2051600] VM allocation process starts [2025-09-27 10:13:11,011][ INFO][PID:2051600] Trying to allocate VM: ResallocHost, ticket_id=5312065, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-09-27 10:13:14,125][ INFO][PID:2051600] Allocated host ResallocHost, ticket_id=5312065, hostname=2620:52:3:1:dead:beef:cafe:c291, name=copr_p09_01_prod_06967817_20250927_091739, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-09-27 10:13:14,126][ INFO][PID:2051600] Allocating ssh connection to builder [2025-09-27 10:13:14,126][ INFO][PID:2051600] Checking that builder machine is OK [2025-09-27 10:13:15,037][ INFO][PID:2051600] Running remote command: copr-builder-ready epel-10-ppc64le [2025-09-27 10:13:15,462][ INFO][PID:2051600] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-09-27 10:13:15,463][ INFO][PID:2051600] Filling build.info file with builder info [2025-09-27 10:13:15,463][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:15,464][ INFO][PID:2051600] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "nucleo", "project_name": "freerouter", "project_dirname": "freerouter", "submitter": "https://api.github.com/users/nuqleo", "ended_on": null, "started_on": 1758967995.464442, "submitted_on": null, "status": 3, "chroot": "epel-10-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9609991-epel-10-ppc64le", "build_id": 9609991, "package_name": "freerouter", "package_version": "25.9.26-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/nucleo/freerouter/freerouter", "git_hash": "87e9eb9f862a92f847b490b1603bf94fe313427f", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "nucleo/freerouter--https://api.github.com/users/nuqleo", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/nucleo/freerouter/epel-10-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le", "arch_power9" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/nucleo/freerouter", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/nucleo/freerouter", "result_dir": "09609991-freerouter", "built_packages": "", "id": 9609991, "mockchain_macros": { "copr_username": "nucleo", "copr_projectname": "freerouter", "vendor": "Fedora Project COPR (nucleo/freerouter)" } } ] } [2025-09-27 10:13:15,516][ INFO][PID:2051600] Sending fedora-messaging bus message in build.start [2025-09-27 10:13:16,274][ INFO][PID:2051600] Sending fedora-messaging bus message in chroot.start [2025-09-27 10:13:16,299][ INFO][PID:2051600] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9609991-epel-10-ppc64le --chroot epel-10-ppc64le --detached [2025-09-27 10:13:17,200][ INFO][PID:2051600] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c291' (ED25519) to the list of known hosts. [2025-09-27 10:13:17,201][ INFO][PID:2051600] Downloading the builder-live.log file, attempt 1 [2025-09-27 10:13:17,202][ INFO][PID:2051600] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c291 copr-rpmbuild-log [2025-09-27 10:13:22,206][ INFO][PID:2051600] Periodic builder liveness probe: alive [2025-09-27 10:13:22,206][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:27,207][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:32,208][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:37,209][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:42,210][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:47,211][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:52,216][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:13:57,224][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:02,226][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:07,228][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:12,229][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:17,230][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:22,231][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:27,232][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:32,233][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:37,273][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:42,274][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:47,275][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:52,276][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:14:57,277][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:02,278][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:07,279][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:12,280][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:17,281][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:17,820][ ERROR][PID:2051600] Re-try request for task on 'ResallocHost, ticket_id=5312065, hostname=2620:52:3:1:dead:beef:cafe:c291, name=copr_p09_01_prod_06967817_20250927_091739, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': SSH problems when downloading live log: copr-rpmbuild-log shouldn't exit != 0 [2025-09-27 10:15:17,821][ INFO][PID:2051600] Releasing VM back to pool [2025-09-27 10:15:17,831][ INFO][PID:2051600] Retry #1 (on other host) [2025-09-27 10:15:17,831][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:17,832][ INFO][PID:2051600] VM allocation process starts [2025-09-27 10:15:17,846][ INFO][PID:2051600] Trying to allocate VM: ResallocHost, ticket_id=5312152, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-09-27 10:15:20,863][ INFO][PID:2051600] Allocated host ResallocHost, ticket_id=5312152, hostname=2620:52:3:1:dead:beef:cafe:c29b, name=copr_p09_01_prod_06967984_20250927_093443, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9'] [2025-09-27 10:15:20,864][ INFO][PID:2051600] Allocating ssh connection to builder [2025-09-27 10:15:20,865][ INFO][PID:2051600] Checking that builder machine is OK [2025-09-27 10:15:21,663][ INFO][PID:2051600] Running remote command: copr-builder-ready epel-10-ppc64le [2025-09-27 10:15:22,084][ INFO][PID:2051600] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2025-09-27 10:15:22,085][ INFO][PID:2051600] Filling build.info file with builder info [2025-09-27 10:15:22,088][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:22,088][ INFO][PID:2051600] Not re-notifying FE and msg buses for the new host. [2025-09-27 10:15:22,089][ INFO][PID:2051600] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9609991-epel-10-ppc64le --chroot epel-10-ppc64le --detached [2025-09-27 10:15:22,902][ INFO][PID:2051600] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c29b' (ED25519) to the list of known hosts. [2025-09-27 10:15:22,902][ INFO][PID:2051600] Downloading the builder-live.log file, attempt 1 [2025-09-27 10:15:22,903][ INFO][PID:2051600] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29b copr-rpmbuild-log [2025-09-27 10:15:27,903][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:32,903][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:37,904][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:42,905][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:47,907][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:52,908][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:15:57,909][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:02,910][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:07,911][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:12,912][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:17,913][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:22,917][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:27,918][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:32,919][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:37,920][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:42,921][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:47,922][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:52,923][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:16:57,924][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:02,926][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:07,927][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:12,928][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:17,929][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:22,930][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:27,931][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:32,932][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:37,933][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:42,934][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:47,935][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:52,936][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:17:57,938][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:18:02,939][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:18:07,940][ INFO][PID:2051600] Checking for cancel request [2025-09-27 10:18:11,002][ INFO][PID:2051600] Downloading results from builder [2025-09-27 10:18:11,003][ INFO][PID:2051600] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c29b]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter started [2025-09-27 10:18:11,004][ INFO][PID:2051600] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c29b]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/ &> /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/build-09609991.rsync.log [2025-09-27 10:18:11,560][ INFO][PID:2051600] rsyncing finished. [2025-09-27 10:18:11,560][ INFO][PID:2051600] Releasing VM back to pool [2025-09-27 10:18:11,569][ INFO][PID:2051600] Searching for 'success' file in resultdir [2025-09-27 10:18:11,570][ INFO][PID:2051600] Going to sign pkgs from source: 9609991-epel-10-ppc64le in chroot: /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le [2025-09-27 10:18:11,571][ INFO][PID:2051600] Calling '/bin/sign -u nucleo#freerouter@copr.fedorahosted.org -p' (attempt #1) [2025-09-27 10:18:11,911][ INFO][PID:2051600] Calling '/bin/sign -4 -h sha256 -u nucleo#freerouter@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-doc-25.9.26-1.el10.noarch.rpm' (attempt #1) [2025-09-27 10:18:12,853][ INFO][PID:2051600] signed rpm: /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-doc-25.9.26-1.el10.noarch.rpm [2025-09-27 10:18:12,854][ INFO][PID:2051600] Calling '/bin/sign -4 -h sha256 -u nucleo#freerouter@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-25.9.26-1.el10.ppc64le.rpm' (attempt #1) [2025-09-27 10:18:13,849][ INFO][PID:2051600] signed rpm: /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-25.9.26-1.el10.ppc64le.rpm [2025-09-27 10:18:13,850][ INFO][PID:2051600] Calling '/bin/sign -4 -h sha256 -u nucleo#freerouter@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-native-25.9.26-1.el10.ppc64le.rpm' (attempt #1) [2025-09-27 10:18:14,773][ INFO][PID:2051600] signed rpm: /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-native-25.9.26-1.el10.ppc64le.rpm [2025-09-27 10:18:14,774][ INFO][PID:2051600] Calling '/bin/sign -4 -h sha256 -u nucleo#freerouter@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-native-debuginfo-25.9.26-1.el10.ppc64le.rpm' (attempt #1) [2025-09-27 10:18:15,696][ INFO][PID:2051600] signed rpm: /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-native-debuginfo-25.9.26-1.el10.ppc64le.rpm [2025-09-27 10:18:15,697][ INFO][PID:2051600] Calling '/bin/sign -4 -h sha256 -u nucleo#freerouter@copr.fedorahosted.org -r /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-25.9.26-1.el10.src.rpm' (attempt #1) [2025-09-27 10:18:16,667][ INFO][PID:2051600] signed rpm: /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/freerouter-25.9.26-1.el10.src.rpm [2025-09-27 10:18:16,668][ INFO][PID:2051600] Sign done [2025-09-27 10:18:16,669][ INFO][PID:2051600] Incremental createrepo run, adding 09609991-freerouter into https://download.copr.fedorainfracloud.org/results/nucleo/freerouter/epel-10-ppc64le, (auto-create-repo=True) [2025-09-27 10:18:16,670][ INFO][PID:2051600] Running command 'copr-repo --batched /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le --add 09609991-freerouter --no-appstream-metadata' as PID 2079987 [2025-09-27 10:18:17,797][ INFO][PID:2051600] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le --add 09609991-freerouter --no-appstream-metadata) [2025-09-27 10:18:17,799][ INFO][PID:2051600] Getting build details [2025-09-27 10:18:17,800][ INFO][PID:2051600] Listing built binary packages in /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter [2025-09-27 10:18:17,801][ INFO][PID:2051600] Built packages: ['freerouter-native-debuginfo 25.9.26', 'freerouter-native 25.9.26', 'freerouter 25.9.26', 'freerouter-doc 25.9.26'] [2025-09-27 10:18:17,803][ INFO][PID:2051600] build details: {'built_packages': 'freerouter-native-debuginfo 25.9.26\nfreerouter-native 25.9.26\nfreerouter 25.9.26\nfreerouter-doc 25.9.26'} [2025-09-27 10:18:17,803][ INFO][PID:2051600] Retrieving pubkey [2025-09-27 10:18:17,804][ INFO][PID:2051600] Calling '/bin/sign -u nucleo#freerouter@copr.fedorahosted.org -p' (attempt #1) [2025-09-27 10:18:18,147][ INFO][PID:2051600] Added pubkey for user nucleo project freerouter into: /var/lib/copr/public_html/results/nucleo/freerouter/pubkey.gpg [2025-09-27 10:18:18,148][ INFO][PID:2051600] Finished build: id=9609991 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/nucleo/freerouter chroot=epel-10-ppc64le [2025-09-27 10:18:18,150][ INFO][PID:2051600] Worker succeeded build, took 176.06203818321228 [2025-09-27 10:18:18,151][ INFO][PID:2051600] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "nucleo", "project_name": "freerouter", "project_dirname": "freerouter", "submitter": "https://api.github.com/users/nuqleo", "ended_on": 1758968298.150835, "started_on": 1758968122.0887969, "submitted_on": null, "status": 1, "chroot": "epel-10-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9609991-epel-10-ppc64le", "build_id": 9609991, "package_name": "freerouter", "package_version": "25.9.26-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/nucleo/freerouter/freerouter", "git_hash": "87e9eb9f862a92f847b490b1603bf94fe313427f", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "nucleo/freerouter--https://api.github.com/users/nuqleo", "results": { "packages": [ { "name": "freerouter-native-debuginfo", "epoch": null, "version": "25.9.26", "release": "1.el10", "arch": "ppc64le" }, { "name": "freerouter-native", "epoch": null, "version": "25.9.26", "release": "1.el10", "arch": "ppc64le" }, { "name": "freerouter", "epoch": null, "version": "25.9.26", "release": "1.el10", "arch": "ppc64le" }, { "name": "freerouter-doc", "epoch": null, "version": "25.9.26", "release": "1.el10", "arch": "noarch" }, { "name": "freerouter", "epoch": null, "version": "25.9.26", "release": "1.el10", "arch": "src" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/nucleo/freerouter/epel-10-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le", "arch_power9" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/nucleo/freerouter", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/nucleo/freerouter", "result_dir": "09609991-freerouter", "built_packages": "freerouter-native-debuginfo 25.9.26\nfreerouter-native 25.9.26\nfreerouter 25.9.26\nfreerouter-doc 25.9.26", "id": 9609991, "mockchain_macros": { "copr_username": "nucleo", "copr_projectname": "freerouter", "vendor": "Fedora Project COPR (nucleo/freerouter)" } } ] } [2025-09-27 10:18:18,205][ INFO][PID:2051600] Sending fedora-messaging bus message in build.end [2025-09-27 10:18:18,232][ INFO][PID:2051600] Compressing /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/builder-live.log by gzip [2025-09-27 10:18:18,233][ INFO][PID:2051600] Running command 'gzip /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/builder-live.log' as PID 2080311 [2025-09-27 10:18:18,241][ INFO][PID:2051600] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/builder-live.log) [2025-09-27 10:18:18,242][ INFO][PID:2051600] Compressing /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/backend.log by gzip [2025-09-27 10:18:18,243][ INFO][PID:2051600] Running command 'gzip /var/lib/copr/public_html/results/nucleo/freerouter/epel-10-ppc64le/09609991-freerouter/backend.log' as PID 2080315