[2026-01-07 13:53:34,834][ INFO][PID:2917596] Marking build as starting [2026-01-07 13:53:34,932][ INFO][PID:2917596] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-07 13:53:35,810][ INFO][PID:2917596] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-07 13:53:36,917][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:53:36,920][ INFO][PID:2917596] VM allocation process starts [2026-01-07 13:53:36,938][ INFO][PID:2917596] Trying to allocate VM: ResallocHost, ticket_id=691266, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-07 13:53:41,941][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:53:43,019][ INFO][PID:2917596] Allocated host ResallocHost, ticket_id=691266, hostname=2620:52:6:1161:dead:beef:cafe:c18b, name=vmhost_p09_04_prod_00986981_20260107_134548, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-07 13:53:43,023][ INFO][PID:2917596] Allocating ssh connection to builder [2026-01-07 13:53:43,024][ INFO][PID:2917596] Checking that builder machine is OK [2026-01-07 13:53:44,011][ INFO][PID:2917596] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2026-01-07 13:53:44,568][ INFO][PID:2917596] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2026-01-07 13:53:44,568][ INFO][PID:2917596] Filling build.info file with builder info [2026-01-07 13:53:44,569][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:53:44,573][ INFO][PID:2917596] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@copr", "project_name": "copr-pull-requests", "project_dirname": "copr-pull-requests:pr:4106", "submitter": null, "ended_on": null, "started_on": 1767794024.571498, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9981752-fedora-rawhide-ppc64le", "build_id": 9981752, "package_name": "copr-rpmbuild", "package_version": "1.6.post1-1.git.89.822ccd0", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/copr-pull-requests:pr:4106/copr-rpmbuild", "git_hash": "35233e32d7b7848287b37d434584d13dcd8032b3", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@copr/copr-pull-requests--7a23753f-fe66-4ef5-b278-b863b7087d84", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/", "id": "copr_coprdir", "name": "Coprdir repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-dev/fedora-rawhide-ppc64le/", "id": "copr_copr_copr_dev", "name": "Additional repo copr_copr_copr_dev" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4106", "result_dir": "09981752-copr-rpmbuild", "built_packages": "", "id": 9981752, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "copr-pull-requests", "vendor": "Fedora Project COPR (@copr/copr-pull-requests)" } } ] } [2026-01-07 13:53:44,653][ INFO][PID:2917596] Sending fedora-messaging bus message in build.start [2026-01-07 13:53:45,331][ INFO][PID:2917596] Sending fedora-messaging bus message in chroot.start [2026-01-07 13:53:45,369][ INFO][PID:2917596] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9981752-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2026-01-07 13:53:46,323][ INFO][PID:2917596] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18b' (ED25519) to the list of known hosts. [2026-01-07 13:53:46,326][ INFO][PID:2917596] Downloading the builder-live.log file, attempt 1 [2026-01-07 13:53:46,329][ INFO][PID:2917596] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log [2026-01-07 13:53:51,345][ INFO][PID:2917596] Periodic builder liveness probe: alive [2026-01-07 13:53:51,345][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:53:56,346][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:01,349][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:06,350][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:11,351][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:16,354][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:21,355][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:26,356][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:31,358][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:36,359][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:41,365][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:46,366][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:51,369][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:54:56,370][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:01,372][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:06,373][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:11,375][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:16,376][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:21,377][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:26,378][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:31,379][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:36,380][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:41,382][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:46,383][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:51,385][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:55:56,386][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:01,393][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:06,395][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:11,396][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:16,397][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:21,398][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:26,399][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:31,401][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:35,372][ ERROR][PID:2917596] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-07 13:56:36,402][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:41,403][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:45,373][WARNING][PID:2917596] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-07 13:56:46,404][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:51,406][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:56:55,374][ INFO][PID:2917596] Downloading the builder-live.log file, attempt 2 [2026-01-07 13:56:55,376][ INFO][PID:2917596] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log [2026-01-07 13:56:56,408][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:01,409][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:06,410][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:11,413][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:16,423][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:21,424][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:26,425][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:31,427][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:36,428][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:41,435][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:46,436][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:51,438][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:57:55,413][ ERROR][PID:2917596] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-07 13:57:56,443][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:01,445][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:05,659][WARNING][PID:2917596] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-07 13:58:06,446][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:11,447][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:15,662][ INFO][PID:2917596] Downloading the builder-live.log file, attempt 3 [2026-01-07 13:58:15,667][ INFO][PID:2917596] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log [2026-01-07 13:58:16,455][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:21,457][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:26,458][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:31,461][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:36,463][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:41,469][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:46,473][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:51,476][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:58:56,477][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:01,488][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:06,490][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:11,491][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:15,745][ ERROR][PID:2917596] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-07 13:59:16,493][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:21,496][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:25,758][WARNING][PID:2917596] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-01-07 13:59:26,499][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:31,500][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:35,759][ INFO][PID:2917596] Downloading the builder-live.log file, attempt 4 [2026-01-07 13:59:35,761][ INFO][PID:2917596] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18b copr-rpmbuild-log [2026-01-07 13:59:36,502][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:41,503][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:46,504][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:51,505][ INFO][PID:2917596] Checking for cancel request [2026-01-07 13:59:56,506][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:01,509][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:06,512][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:11,523][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:16,524][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:21,526][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:26,528][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:31,529][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:35,829][ ERROR][PID:2917596] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-01-07 14:00:36,531][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:41,532][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:45,909][ ERROR][PID:2917596] Giving up for unstable SSH, failures: 1767794205.373579, 1767794285.65929, 1767794365.7588522, 1767794445.909651 [2026-01-07 14:00:45,910][ ERROR][PID:2917596] Re-try request for task on 'ResallocHost, ticket_id=691266, hostname=2620:52:6:1161:dead:beef:cafe:c18b, name=vmhost_p09_04_prod_00986981_20260107_134548, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-01-07 14:00:45,911][ INFO][PID:2917596] Releasing VM back to pool [2026-01-07 14:00:45,922][ INFO][PID:2917596] Retry #1 (on other host) [2026-01-07 14:00:45,922][ INFO][PID:2917596] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-07 14:00:46,103][ INFO][PID:2917596] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-07 14:00:46,288][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:00:46,290][ INFO][PID:2917596] VM allocation process starts [2026-01-07 14:00:46,302][ INFO][PID:2917596] Trying to allocate VM: ResallocHost, ticket_id=691373, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-07 14:00:49,320][ INFO][PID:2917596] Allocated host ResallocHost, ticket_id=691373, hostname=2620:52:6:1161:dead:beef:cafe:c189, name=vmhost_p09_04_prod_00988177_20260107_135451, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-07 14:00:49,320][ INFO][PID:2917596] Allocating ssh connection to builder [2026-01-07 14:00:49,321][ INFO][PID:2917596] Checking that builder machine is OK [2026-01-07 14:01:49,391][ ERROR][PID:2917596] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c189 port 22: Connection timed out [2026-01-07 14:01:59,392][ ERROR][PID:2917596] Re-try request for task on 'ResallocHost, ticket_id=691373, hostname=2620:52:6:1161:dead:beef:cafe:c189, name=vmhost_p09_04_prod_00988177_20260107_135451, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts [2026-01-07 14:01:59,392][ INFO][PID:2917596] Releasing VM back to pool [2026-01-07 14:01:59,407][ INFO][PID:2917596] Retry #2 (on other host) [2026-01-07 14:01:59,408][ INFO][PID:2917596] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-07 14:01:59,637][ INFO][PID:2917596] Checking that https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le//repodata/repomd.xml exists [2026-01-07 14:01:59,933][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:01:59,935][ INFO][PID:2917596] VM allocation process starts [2026-01-07 14:01:59,953][ INFO][PID:2917596] Trying to allocate VM: ResallocHost, ticket_id=691383, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-07 14:02:02,982][ INFO][PID:2917596] Allocated host ResallocHost, ticket_id=691383, hostname=2620:52:6:1161:dead:beef:cafe:c195, name=vmhost_p09_04_prod_00988390_20260107_135636, requested_tags=['copr_builder', 'arch_ppc64le'] [2026-01-07 14:02:02,982][ INFO][PID:2917596] Allocating ssh connection to builder [2026-01-07 14:02:02,983][ INFO][PID:2917596] Checking that builder machine is OK [2026-01-07 14:02:04,041][ INFO][PID:2917596] Running remote command: copr-builder-ready fedora-rawhide-ppc64le [2026-01-07 14:02:04,583][ INFO][PID:2917596] Red Hat subscription not needed for fedora-rawhide-ppc64le Builder is ready to be used [2026-01-07 14:02:04,585][ INFO][PID:2917596] Filling build.info file with builder info [2026-01-07 14:02:04,594][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:04,596][ INFO][PID:2917596] Not re-notifying FE and msg buses for the new host. [2026-01-07 14:02:04,597][ INFO][PID:2917596] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9981752-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached [2026-01-07 14:02:05,590][ INFO][PID:2917596] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c195' (ED25519) to the list of known hosts. [2026-01-07 14:02:05,591][ INFO][PID:2917596] Downloading the builder-live.log file, attempt 1 [2026-01-07 14:02:05,593][ INFO][PID:2917596] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c195 copr-rpmbuild-log [2026-01-07 14:02:10,592][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:15,593][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:20,594][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:25,596][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:30,599][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:35,600][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:40,601][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:45,603][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:50,606][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:02:55,607][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:00,608][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:05,610][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:10,611][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:15,613][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:20,616][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:25,617][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:30,618][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:35,620][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:40,622][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:45,623][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:50,625][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:03:55,636][ INFO][PID:2917596] Periodic builder liveness probe: alive [2026-01-07 14:03:55,637][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:00,638][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:05,640][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:10,641][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:15,642][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:20,644][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:25,645][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:30,646][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:35,647][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:40,648][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:45,653][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:50,655][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:04:55,657][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:00,661][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:05,662][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:10,665][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:15,669][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:20,673][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:25,678][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:30,679][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:35,681][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:40,682][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:45,683][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:50,689][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:05:55,692][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:00,693][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:05,695][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:10,696][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:15,698][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:20,700][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:25,702][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:30,704][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:35,707][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:40,712][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:45,715][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:50,718][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:06:55,720][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:07:00,721][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:07:05,723][ INFO][PID:2917596] Checking for cancel request [2026-01-07 14:07:07,691][ INFO][PID:2917596] Downloading results from builder [2026-01-07 14:07:07,692][ INFO][PID:2917596] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c195]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild started [2026-01-07 14:07:07,695][ INFO][PID:2917596] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c195]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/ &> /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/build-09981752.rsync.log [2026-01-07 14:07:08,577][ INFO][PID:2917596] rsyncing finished. [2026-01-07 14:07:08,578][ INFO][PID:2917596] Releasing VM back to pool [2026-01-07 14:07:08,592][ INFO][PID:2917596] Searching for 'success' file in resultdir [2026-01-07 14:07:08,593][ INFO][PID:2917596] Going to sign pkgs from source: 9981752-fedora-rawhide-ppc64le in chroot: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le [2026-01-07 14:07:08,594][ INFO][PID:2917596] Calling '/bin/sign -u @copr#copr-pull-requests@copr.fedorahosted.org -p' (attempt #1) [2026-01-07 14:07:08,853][ INFO][PID:2917596] Calling '/bin/sign -4 -h sha256 -u @copr#copr-pull-requests@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.src.rpm' (attempt #1) [2026-01-07 14:07:09,565][ INFO][PID:2917596] signed rpm: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.src.rpm [2026-01-07 14:07:09,567][ INFO][PID:2917596] Calling '/bin/sign -4 -h sha256 -u @copr#copr-pull-requests@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-builder-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm' (attempt #1) [2026-01-07 14:07:10,284][ INFO][PID:2917596] signed rpm: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-builder-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm [2026-01-07 14:07:10,286][ INFO][PID:2917596] Calling '/bin/sign -4 -h sha256 -u @copr#copr-pull-requests@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm' (attempt #1) [2026-01-07 14:07:11,047][ INFO][PID:2917596] signed rpm: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm [2026-01-07 14:07:11,051][ INFO][PID:2917596] Sign done [2026-01-07 14:07:11,055][ INFO][PID:2917596] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.src.rpm [2026-01-07 14:07:11,539][ INFO][PID:2917596] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-builder-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm [2026-01-07 14:07:11,546][ INFO][PID:2917596] [1/3] Uploaded to Pulp: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.src.rpm [2026-01-07 14:07:12,453][ INFO][PID:2917596] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm [2026-01-07 14:07:12,455][ INFO][PID:2917596] [2/3] Uploaded to Pulp: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-builder-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm [2026-01-07 14:07:13,097][ INFO][PID:2917596] [3/3] Uploaded to Pulp: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm [2026-01-07 14:07:13,119][ INFO][PID:2917596] Pulp: get_repository: /api/v3/repositories/rpm/rpm/?name=%40copr%2Fcopr-pull-requests%3Apr%3A4106%2Ffedora-rawhide-ppc64le&offset=0&limit=1 [2026-01-07 14:07:14,109][ INFO][PID:2917596] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019b98c8-79d8-70ff-9389-16684c04a0af/ [2026-01-07 14:07:14,126][ INFO][PID:2917596] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019b98c8-79d8-70ff-9389-16684c04a0af/ [2026-01-07 14:07:19,830][ INFO][PID:2917596] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019b98c8-79d8-70ff-9389-16684c04a0af/ [2026-01-07 14:07:19,832][ INFO][PID:2917596] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019b98c8-79d8-70ff-9389-16684c04a0af/ [2026-01-07 14:07:20,135][ INFO][PID:2917596] Successfully modified Pulp repository content /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019b93d9-9602-7291-8d6d-29f3c25705f4/ [2026-01-07 14:07:20,138][ INFO][PID:2917596] Pulp: publishing /api/v3/publications/rpm/rpm/ /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019b93d9-9602-7291-8d6d-29f3c25705f4/ [2026-01-07 14:07:20,472][ INFO][PID:2917596] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019b98c8-9361-7c2d-a67b-ebbb8dcc1535/ [2026-01-07 14:07:20,476][ INFO][PID:2917596] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019b98c8-9361-7c2d-a67b-ebbb8dcc1535/ [2026-01-07 14:07:25,779][ INFO][PID:2917596] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019b98c8-9361-7c2d-a67b-ebbb8dcc1535/ [2026-01-07 14:07:25,781][ INFO][PID:2917596] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019b98c8-9361-7c2d-a67b-ebbb8dcc1535/ [2026-01-07 14:07:26,011][ INFO][PID:2917596] Successfully created Pulp publication of repository /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019b93d9-9602-7291-8d6d-29f3c25705f4/ [2026-01-07 14:07:26,015][ INFO][PID:2917596] Pulp: add_content: /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019b93d9-9602-7291-8d6d-29f3c25705f4/ (['/api/pulp/public-copr/api/v3/content/rpm/packages/019b98c8-7062-730d-9348-49b8cee85045/', '/api/pulp/public-copr/api/v3/content/rpm/packages/019b98c8-732b-7857-802b-c6a143eeaa9c/', '/api/pulp/public-copr/api/v3/content/rpm/packages/019b98c8-7676-74f0-a18e-afd4cc188f8b/']) [2026-01-07 14:07:26,019][ INFO][PID:2917596] Getting build details [2026-01-07 14:07:26,020][ INFO][PID:2917596] Listing built binary packages in /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild [2026-01-07 14:07:26,022][ INFO][PID:2917596] Built packages: ['copr-builder 1.6.post1', 'copr-rpmbuild 1.6.post1'] [2026-01-07 14:07:26,023][ INFO][PID:2917596] build details: {'built_packages': 'copr-builder 1.6.post1\ncopr-rpmbuild 1.6.post1'} [2026-01-07 14:07:26,024][ INFO][PID:2917596] Retrieving pubkey [2026-01-07 14:07:26,025][ INFO][PID:2917596] Calling '/bin/sign -u @copr#copr-pull-requests@copr.fedorahosted.org -p' (attempt #1) [2026-01-07 14:07:26,301][ INFO][PID:2917596] Added pubkey for user @copr project copr-pull-requests into: /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/pubkey.gpg [2026-01-07 14:07:26,302][ INFO][PID:2917596] Removing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.src.rpm, it is stored in Pulp [2026-01-07 14:07:26,304][ INFO][PID:2917596] Removing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-builder-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm, it is stored in Pulp [2026-01-07 14:07:26,308][ INFO][PID:2917596] Removing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/copr-rpmbuild-1.6.post1-1.git.89.822ccd0.fc44.ppc64le.rpm, it is stored in Pulp [2026-01-07 14:07:26,309][ INFO][PID:2917596] Finished build: id=9981752 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106 chroot=fedora-rawhide-ppc64le [2026-01-07 14:07:26,324][ INFO][PID:2917596] Worker succeeded build, took 321.7277886867523 [2026-01-07 14:07:26,330][ INFO][PID:2917596] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@copr", "project_name": "copr-pull-requests", "project_dirname": "copr-pull-requests:pr:4106", "submitter": null, "ended_on": 1767794846.3245158, "started_on": 1767794524.5967271, "submitted_on": null, "status": 1, "chroot": "fedora-rawhide-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9981752-fedora-rawhide-ppc64le", "build_id": 9981752, "package_name": "copr-rpmbuild", "package_version": "1.6.post1-1.git.89.822ccd0", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/copr-pull-requests:pr:4106/copr-rpmbuild", "git_hash": "35233e32d7b7848287b37d434584d13dcd8032b3", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@copr/copr-pull-requests--7a23753f-fe66-4ef5-b278-b863b7087d84", "results": { "packages": [ { "name": "copr-builder", "epoch": null, "version": "1.6.post1", "release": "1.git.89.822ccd0.fc44", "arch": "ppc64le" }, { "name": "copr-rpmbuild", "epoch": null, "version": "1.6.post1", "release": "1.git.89.822ccd0.fc44", "arch": "src" }, { "name": "copr-rpmbuild", "epoch": null, "version": "1.6.post1", "release": "1.git.89.822ccd0.fc44", "arch": "ppc64le" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests/fedora-rawhide-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/", "id": "copr_coprdir", "name": "Coprdir repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/copr-dev/fedora-rawhide-ppc64le/", "id": "copr_copr_copr_dev", "name": "Additional repo copr_copr_copr_dev" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/copr-pull-requests:pr:4106", "result_dir": "09981752-copr-rpmbuild", "built_packages": "copr-builder 1.6.post1\ncopr-rpmbuild 1.6.post1", "id": 9981752, "mockchain_macros": { "copr_username": "@copr", "copr_projectname": "copr-pull-requests", "vendor": "Fedora Project COPR (@copr/copr-pull-requests)" } } ] } [2026-01-07 14:07:26,533][ INFO][PID:2917596] Sending fedora-messaging bus message in build.end [2026-01-07 14:07:26,619][ INFO][PID:2917596] Compressing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/builder-live.log by gzip [2026-01-07 14:07:26,626][ INFO][PID:2917596] Running command 'gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/builder-live.log' as PID 3037993 [2026-01-07 14:07:26,680][ INFO][PID:2917596] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/builder-live.log) [2026-01-07 14:07:26,682][ INFO][PID:2917596] Compressing /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/backend.log by gzip [2026-01-07 14:07:26,689][ INFO][PID:2917596] Running command 'gzip /var/lib/copr/public_html/results/@copr/copr-pull-requests:pr:4106/fedora-rawhide-ppc64le/09981752-copr-rpmbuild/backend.log' as PID 3038007