[2026-04-10 13:27:14,988][ INFO][PID:293497] Marking build as starting [2026-04-10 13:27:15,112][ INFO][PID:293497] Checking that /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/repodata/repomd.xml exists [2026-04-10 13:27:15,118][ INFO][PID:293497] Checking that /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/repodata/repomd.xml exists [2026-04-10 13:27:15,121][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:27:15,127][ INFO][PID:293497] VM allocation process starts [2026-04-10 13:27:15,158][ INFO][PID:293497] Trying to allocate VM: ResallocHost, ticket_id=2264939, requested_tags=['copr_builder', 'arch_s390x'] [2026-04-10 13:27:20,168][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:27:25,171][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:27:27,231][ INFO][PID:293497] Allocated host ResallocHost, ticket_id=2264939, hostname=3.231.157.114, name=aws_x86_64_spot_prod_05976163_20260410_131836, requested_tags=['copr_builder', 'arch_s390x'] [2026-04-10 13:27:27,232][ INFO][PID:293497] Allocating ssh connection to builder [2026-04-10 13:27:27,233][ INFO][PID:293497] Checking that builder machine is OK [2026-04-10 13:27:27,635][ INFO][PID:293497] Running remote command: copr-builder-ready fedora-43-s390x [2026-04-10 13:27:27,944][ INFO][PID:293497] Red Hat subscription not needed for fedora-43-s390x Builder is ready to be used [2026-04-10 13:27:27,946][ INFO][PID:293497] Filling build.info file with builder info [2026-04-10 13:27:27,959][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:27:27,971][ INFO][PID:293497] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "osloup", "project_name": "postgresql_update_15_2", "project_dirname": "postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6", "submitter": "https://src.fedoraproject.org/user/jplesnik", "ended_on": null, "started_on": 1775827647.970935, "submitted_on": null, "status": 3, "chroot": "fedora-43-s390x", "arch": "s390x", "buildroot_pkgs": [], "task_id": "10311610-fedora-43-s390x", "build_id": 10311610, "package_name": "perl-DBD-Pg", "package_version": "3.20.0-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/perl-DBD-Pg", "git_hash": "8e3bee16d8088705460ebff00fa58f21fbba5bfb", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "osloup/postgresql_update_15_2--https://src.fedoraproject.org/user/jplesnik", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/osloup/postgresql_update_15_2/fedora-43-s390x/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/", "id": "copr_coprdir", "name": "Coprdir repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_s390x" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6", "result_dir": "10311610-perl-DBD-Pg", "built_packages": "", "id": 10311610, "mockchain_macros": { "copr_username": "osloup", "copr_projectname": "postgresql_update_15_2", "vendor": "Fedora Project COPR (osloup/postgresql_update_15_2)" } } ] } [2026-04-10 13:27:28,256][ INFO][PID:293497] Sending fedora-messaging bus message in build.start [2026-04-10 13:27:31,699][ INFO][PID:293497] Sending fedora-messaging bus message in chroot.start [2026-04-10 13:27:31,788][ INFO][PID:293497] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10311610-fedora-43-s390x --chroot fedora-43-s390x --detached [2026-04-10 13:27:32,254][ INFO][PID:293497] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.231.157.114' (ED25519) to the list of known hosts. [2026-04-10 13:27:32,255][ INFO][PID:293497] Downloading the builder-live.log file, attempt 1 [2026-04-10 13:27:32,266][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.231.157.114 copr-rpmbuild-log [2026-04-10 13:27:37,280][ INFO][PID:293497] Periodic builder liveness probe: alive [2026-04-10 13:27:37,283][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:27:42,286][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:27:47,292][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:27:52,300][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:27:57,301][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:02,302][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:07,306][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:12,313][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:17,325][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:22,326][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:27,329][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:32,331][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:37,334][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:42,338][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:47,344][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:52,347][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:28:57,349][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:02,352][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:07,357][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:12,359][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:17,365][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:22,372][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:27,381][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:32,388][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:37,401][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:42,407][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:45,403][ ERROR][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-10 13:29:47,413][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:52,415][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:29:55,405][WARNING][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-10 13:29:57,423][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:02,428][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:05,410][ INFO][PID:293497] Downloading the builder-live.log file, attempt 2 [2026-04-10 13:30:05,412][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.231.157.114 copr-rpmbuild-log [2026-04-10 13:30:07,431][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:12,435][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:17,437][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:22,444][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:27,447][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:32,457][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:37,468][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:42,480][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:47,485][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:52,488][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:30:57,495][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:02,497][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:05,488][ ERROR][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-10 13:31:07,503][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:12,509][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:15,514][WARNING][PID:293497] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-10 13:31:17,519][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:22,524][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:25,515][ INFO][PID:293497] Downloading the builder-live.log file, attempt 3 [2026-04-10 13:31:25,521][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.231.157.114 copr-rpmbuild-log [2026-04-10 13:31:27,528][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:32,529][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:37,532][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:42,536][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:47,543][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:52,559][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:31:57,570][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:02,572][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:07,578][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:12,583][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:17,595][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:22,596][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:25,608][ ERROR][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-10 13:32:27,600][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:32,606][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:35,621][WARNING][PID:293497] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-10 13:32:37,609][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:42,614][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:45,622][ INFO][PID:293497] Downloading the builder-live.log file, attempt 4 [2026-04-10 13:32:45,637][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.231.157.114 copr-rpmbuild-log [2026-04-10 13:32:47,618][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:52,623][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:32:57,628][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:02,640][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:07,641][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:12,643][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:17,647][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:22,680][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:27,685][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:32,687][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:37,691][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:42,694][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:45,772][ ERROR][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-10 13:33:47,696][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:52,699][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:55,780][ ERROR][PID:293497] Giving up for unstable SSH, failures: 1775827795.4058402, 1775827875.514234, 1775827955.6219034, 1775828035.7794635 [2026-04-10 13:33:55,785][ ERROR][PID:293497] Re-try request for task on 'ResallocHost, ticket_id=2264939, hostname=3.231.157.114, name=aws_x86_64_spot_prod_05976163_20260410_131836, requested_tags=['copr_builder', 'arch_s390x']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-04-10 13:33:55,786][ INFO][PID:293497] Releasing VM back to pool [2026-04-10 13:33:55,818][ INFO][PID:293497] Retry #1 (on other host) [2026-04-10 13:33:55,821][ INFO][PID:293497] Checking that /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/repodata/repomd.xml exists [2026-04-10 13:33:55,827][ INFO][PID:293497] Checking that /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/repodata/repomd.xml exists [2026-04-10 13:33:55,830][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:55,833][ INFO][PID:293497] VM allocation process starts [2026-04-10 13:33:55,871][ INFO][PID:293497] Trying to allocate VM: ResallocHost, ticket_id=2265084, requested_tags=['copr_builder', 'arch_s390x'] [2026-04-10 13:33:58,905][ INFO][PID:293497] Allocated host ResallocHost, ticket_id=2265084, hostname=184.72.198.103, name=aws_x86_64_spot_prod_05976111_20260410_131700, requested_tags=['copr_builder', 'arch_s390x'] [2026-04-10 13:33:58,908][ INFO][PID:293497] Allocating ssh connection to builder [2026-04-10 13:33:58,910][ INFO][PID:293497] Checking that builder machine is OK [2026-04-10 13:33:59,134][ INFO][PID:293497] Running remote command: copr-builder-ready fedora-43-s390x [2026-04-10 13:33:59,353][ INFO][PID:293497] Red Hat subscription not needed for fedora-43-s390x Builder is ready to be used [2026-04-10 13:33:59,354][ INFO][PID:293497] Filling build.info file with builder info [2026-04-10 13:33:59,362][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:33:59,365][ INFO][PID:293497] Not re-notifying FE and msg buses for the new host. [2026-04-10 13:33:59,369][ INFO][PID:293497] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10311610-fedora-43-s390x --chroot fedora-43-s390x --detached [2026-04-10 13:33:59,688][ INFO][PID:293497] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '184.72.198.103' (ED25519) to the list of known hosts. [2026-04-10 13:33:59,690][ INFO][PID:293497] Downloading the builder-live.log file, attempt 1 [2026-04-10 13:33:59,700][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@184.72.198.103 copr-rpmbuild-log [2026-04-10 13:34:04,701][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:09,718][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:14,721][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:19,723][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:24,726][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:29,728][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:34,730][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:39,732][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:44,733][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:48,527][ ERROR][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-10 13:34:49,735][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:54,740][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:34:58,534][WARNING][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-10 13:34:59,742][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:04,743][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:08,535][ INFO][PID:293497] Downloading the builder-live.log file, attempt 2 [2026-04-10 13:35:08,553][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@184.72.198.103 copr-rpmbuild-log [2026-04-10 13:35:09,744][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:14,746][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:19,748][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:24,749][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:29,751][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:34,752][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:39,753][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:44,754][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:49,755][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:54,756][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:35:59,757][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:04,758][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:08,644][ ERROR][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-10 13:36:09,762][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:14,765][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:18,659][WARNING][PID:293497] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-10 13:36:19,766][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:24,769][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:28,660][ INFO][PID:293497] Downloading the builder-live.log file, attempt 3 [2026-04-10 13:36:28,662][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@184.72.198.103 copr-rpmbuild-log [2026-04-10 13:36:29,770][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:34,771][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:39,772][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:44,773][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:49,774][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:54,775][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:36:59,778][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:04,779][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:09,781][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:14,784][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:19,785][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:24,787][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:28,731][ ERROR][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-10 13:37:29,793][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:34,797][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:38,834][WARNING][PID:293497] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-04-10 13:37:39,812][ INFO][PID:293497] Periodic builder liveness probe: alive [2026-04-10 13:37:39,813][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:44,815][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:48,835][ INFO][PID:293497] Downloading the builder-live.log file, attempt 4 [2026-04-10 13:37:48,837][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@184.72.198.103 copr-rpmbuild-log [2026-04-10 13:37:49,817][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:54,818][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:37:59,820][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:04,821][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:09,822][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:14,823][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:19,825][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:24,826][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:29,830][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:34,831][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:39,832][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:44,836][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:48,911][ ERROR][PID:293497] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-04-10 13:38:49,837][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:54,838][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:58,917][ ERROR][PID:293497] Giving up for unstable SSH, failures: 1775828098.5344756, 1775828178.6594381, 1775828258.8343644, 1775828338.917051 [2026-04-10 13:38:58,918][ ERROR][PID:293497] Re-try request for task on 'ResallocHost, ticket_id=2265084, hostname=184.72.198.103, name=aws_x86_64_spot_prod_05976111_20260410_131700, requested_tags=['copr_builder', 'arch_s390x']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-04-10 13:38:58,920][ INFO][PID:293497] Releasing VM back to pool [2026-04-10 13:38:58,933][ INFO][PID:293497] Retry #2 (on other host) [2026-04-10 13:38:58,934][ INFO][PID:293497] Checking that /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/repodata/repomd.xml exists [2026-04-10 13:38:58,934][ INFO][PID:293497] Checking that /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/repodata/repomd.xml exists [2026-04-10 13:38:58,935][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:38:58,936][ INFO][PID:293497] VM allocation process starts [2026-04-10 13:38:58,948][ INFO][PID:293497] Trying to allocate VM: ResallocHost, ticket_id=2265142, requested_tags=['copr_builder', 'arch_s390x'] [2026-04-10 13:39:01,985][ INFO][PID:293497] Allocated host ResallocHost, ticket_id=2265142, hostname=3.82.64.241, name=aws_x86_64_normalreserved_prod_05976934_20260410_133617, requested_tags=['copr_builder', 'arch_s390x'] [2026-04-10 13:39:01,986][ INFO][PID:293497] Allocating ssh connection to builder [2026-04-10 13:39:01,986][ INFO][PID:293497] Checking that builder machine is OK [2026-04-10 13:39:02,328][ INFO][PID:293497] Running remote command: copr-builder-ready fedora-43-s390x [2026-04-10 13:39:02,540][ INFO][PID:293497] Red Hat subscription not needed for fedora-43-s390x Builder is ready to be used [2026-04-10 13:39:02,541][ INFO][PID:293497] Filling build.info file with builder info [2026-04-10 13:39:02,553][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:02,554][ INFO][PID:293497] Not re-notifying FE and msg buses for the new host. [2026-04-10 13:39:02,555][ INFO][PID:293497] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10311610-fedora-43-s390x --chroot fedora-43-s390x --detached [2026-04-10 13:39:02,901][ INFO][PID:293497] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.82.64.241' (ED25519) to the list of known hosts. [2026-04-10 13:39:02,902][ INFO][PID:293497] Downloading the builder-live.log file, attempt 1 [2026-04-10 13:39:02,903][ INFO][PID:293497] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.82.64.241 copr-rpmbuild-log [2026-04-10 13:39:07,902][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:12,908][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:17,923][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:22,926][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:27,928][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:32,929][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:37,930][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:42,931][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:47,932][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:52,933][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:39:57,934][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:02,935][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:07,937][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:12,938][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:17,939][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:22,941][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:27,942][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:32,943][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:37,943][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:42,944][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:47,945][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:52,947][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:40:57,948][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:02,949][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:07,950][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:12,951][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:17,952][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:22,953][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:27,954][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:32,955][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:37,968][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:42,969][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:47,970][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:52,971][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:41:57,972][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:02,973][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:07,974][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:12,975][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:17,976][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:22,977][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:27,978][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:32,979][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:37,980][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:42,981][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:47,982][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:52,983][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:42:57,984][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:02,985][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:07,987][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:12,988][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:17,989][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:22,990][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:27,991][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:32,992][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:37,993][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:42,995][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:47,996][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:52,997][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:43:57,999][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:44:03,004][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:44:08,005][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:44:13,006][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:44:18,009][ INFO][PID:293497] Checking for cancel request [2026-04-10 13:44:20,610][ INFO][PID:293497] Downloading results from builder [2026-04-10 13:44:20,611][ INFO][PID:293497] rsyncing of mockbuilder@3.82.64.241:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg started [2026-04-10 13:44:20,612][ INFO][PID:293497] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@3.82.64.241:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/ &> /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/build-10311610.rsync.log [2026-04-10 13:44:21,106][ INFO][PID:293497] rsyncing finished. [2026-04-10 13:44:21,107][ INFO][PID:293497] Releasing VM back to pool [2026-04-10 13:44:21,123][ INFO][PID:293497] Searching for 'success' file in resultdir [2026-04-10 13:44:21,124][ INFO][PID:293497] Going to sign pkgs from source: 10311610-fedora-43-s390x in chroot: /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x [2026-04-10 13:44:21,124][ INFO][PID:293497] Calling '/bin/sign -u osloup#postgresql_update_15_2@copr.fedorahosted.org -p' (attempt #1) [2026-04-10 13:44:21,414][ INFO][PID:293497] Calling '/bin/sign -4 -h sha256 -u osloup#postgresql_update_15_2@copr.fedorahosted.org -r /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-tests-3.20.0-1.fc43.s390x.rpm' (attempt #1) [2026-04-10 13:44:22,172][ INFO][PID:293497] signed rpm: /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-tests-3.20.0-1.fc43.s390x.rpm [2026-04-10 13:44:22,172][ INFO][PID:293497] Calling '/bin/sign -4 -h sha256 -u osloup#postgresql_update_15_2@copr.fedorahosted.org -r /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-debugsource-3.20.0-1.fc43.s390x.rpm' (attempt #1) [2026-04-10 13:44:22,936][ INFO][PID:293497] signed rpm: /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-debugsource-3.20.0-1.fc43.s390x.rpm [2026-04-10 13:44:22,937][ INFO][PID:293497] Calling '/bin/sign -4 -h sha256 -u osloup#postgresql_update_15_2@copr.fedorahosted.org -r /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-debuginfo-3.20.0-1.fc43.s390x.rpm' (attempt #1) [2026-04-10 13:44:23,695][ INFO][PID:293497] signed rpm: /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-debuginfo-3.20.0-1.fc43.s390x.rpm [2026-04-10 13:44:23,696][ INFO][PID:293497] Calling '/bin/sign -4 -h sha256 -u osloup#postgresql_update_15_2@copr.fedorahosted.org -r /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-3.20.0-1.fc43.s390x.rpm' (attempt #1) [2026-04-10 13:44:24,460][ INFO][PID:293497] signed rpm: /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-3.20.0-1.fc43.s390x.rpm [2026-04-10 13:44:24,460][ INFO][PID:293497] Calling '/bin/sign -4 -h sha256 -u osloup#postgresql_update_15_2@copr.fedorahosted.org -r /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-3.20.0-1.fc43.src.rpm' (attempt #1) [2026-04-10 13:44:25,242][ INFO][PID:293497] signed rpm: /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/perl-DBD-Pg-3.20.0-1.fc43.src.rpm [2026-04-10 13:44:25,242][ INFO][PID:293497] Sign done [2026-04-10 13:44:25,243][ INFO][PID:293497] Incremental createrepo run, adding 10311610-perl-DBD-Pg into https://download.copr.fedorainfracloud.org/results/osloup/postgresql_update_15_2/fedora-43-s390x, (auto-create-repo=True) [2026-04-10 13:44:25,244][ INFO][PID:293497] Running command 'copr-repo --batched /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x --add 10311610-perl-DBD-Pg --no-appstream-metadata' as PID 468623 [2026-04-10 13:44:25,832][ INFO][PID:293497] Finished after 0 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x --add 10311610-perl-DBD-Pg --no-appstream-metadata) [2026-04-10 13:44:25,833][ INFO][PID:293497] Getting build details [2026-04-10 13:44:25,834][ INFO][PID:293497] Listing built binary packages in /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg [2026-04-10 13:44:25,834][ INFO][PID:293497] Built packages: ['perl-DBD-Pg-debuginfo 3.20.0', 'perl-DBD-Pg-debugsource 3.20.0', 'perl-DBD-Pg-tests 3.20.0', 'perl-DBD-Pg 3.20.0'] [2026-04-10 13:44:25,835][ INFO][PID:293497] build details: {'built_packages': 'perl-DBD-Pg-debuginfo 3.20.0\nperl-DBD-Pg-debugsource 3.20.0\nperl-DBD-Pg-tests 3.20.0\nperl-DBD-Pg 3.20.0'} [2026-04-10 13:44:25,835][ INFO][PID:293497] Retrieving pubkey [2026-04-10 13:44:25,835][ INFO][PID:293497] Calling '/bin/sign -u osloup#postgresql_update_15_2@copr.fedorahosted.org -p' (attempt #1) [2026-04-10 13:44:26,130][ INFO][PID:293497] Added pubkey for user osloup project postgresql_update_15_2 into: /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/pubkey.gpg [2026-04-10 13:44:26,131][ INFO][PID:293497] Finished build: id=10311610 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6 chroot=fedora-43-s390x [2026-04-10 13:44:26,132][ INFO][PID:293497] Worker succeeded build, took 323.5780785083771 [2026-04-10 13:44:26,133][ INFO][PID:293497] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "osloup", "project_name": "postgresql_update_15_2", "project_dirname": "postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6", "submitter": "https://src.fedoraproject.org/user/jplesnik", "ended_on": 1775828666.1327322, "started_on": 1775828342.5546536, "submitted_on": null, "status": 1, "chroot": "fedora-43-s390x", "arch": "s390x", "buildroot_pkgs": [], "task_id": "10311610-fedora-43-s390x", "build_id": 10311610, "package_name": "perl-DBD-Pg", "package_version": "3.20.0-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/perl-DBD-Pg", "git_hash": "8e3bee16d8088705460ebff00fa58f21fbba5bfb", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "osloup/postgresql_update_15_2--https://src.fedoraproject.org/user/jplesnik", "results": { "packages": [ { "name": "perl-DBD-Pg-debuginfo", "epoch": null, "version": "3.20.0", "release": "1.fc43", "arch": "s390x" }, { "name": "perl-DBD-Pg", "epoch": null, "version": "3.20.0", "release": "1.fc43", "arch": "src" }, { "name": "perl-DBD-Pg-debugsource", "epoch": null, "version": "3.20.0", "release": "1.fc43", "arch": "s390x" }, { "name": "perl-DBD-Pg-tests", "epoch": null, "version": "3.20.0", "release": "1.fc43", "arch": "s390x" }, { "name": "perl-DBD-Pg", "epoch": null, "version": "3.20.0", "release": "1.fc43", "arch": "s390x" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/osloup/postgresql_update_15_2/fedora-43-s390x/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/", "id": "copr_coprdir", "name": "Coprdir repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_s390x" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6", "result_dir": "10311610-perl-DBD-Pg", "built_packages": "perl-DBD-Pg-debuginfo 3.20.0\nperl-DBD-Pg-debugsource 3.20.0\nperl-DBD-Pg-tests 3.20.0\nperl-DBD-Pg 3.20.0", "id": 10311610, "mockchain_macros": { "copr_username": "osloup", "copr_projectname": "postgresql_update_15_2", "vendor": "Fedora Project COPR (osloup/postgresql_update_15_2)" } } ] } [2026-04-10 13:44:26,195][ INFO][PID:293497] Sending fedora-messaging bus message in build.end [2026-04-10 13:44:26,222][ INFO][PID:293497] Compressing /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/builder-live.log by gzip [2026-04-10 13:44:26,223][ INFO][PID:293497] Running command 'gzip /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/builder-live.log' as PID 468733 [2026-04-10 13:44:26,289][ INFO][PID:293497] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/builder-live.log) [2026-04-10 13:44:26,290][ INFO][PID:293497] Compressing /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/backend.log by gzip [2026-04-10 13:44:26,291][ INFO][PID:293497] Running command 'gzip /var/lib/copr/public_html/results/osloup/postgresql_update_15_2:pr:fork-jplesnik-rpms-perl-DBD-Pg-6/fedora-43-s390x/10311610-perl-DBD-Pg/backend.log' as PID 468736