[2026-07-10 15:31:30,516][ INFO][PID:3061076] Marking build as starting [2026-07-10 15:31:30,587][ INFO][PID:3061076] Checking that https://download.copr.fedorainfracloud.org/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64//repodata/repomd.xml exists [2026-07-10 15:31:30,906][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:31:30,933][WARNING][PID:3061076] rpmeta: cannot connect to https://rpmeta.fedoraproject.org, skipping [2026-07-10 15:31:30,934][ INFO][PID:3061076] VM allocation process starts [2026-07-10 15:31:31,119][ INFO][PID:3061076] Trying to allocate VM: ResallocHost, ticket_id=3851077, requested_tags=['arch_x86_64', 'copr_builder'] [2026-07-10 15:31:36,122][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:31:41,123][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:31:46,144][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:31:51,146][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:31:56,151][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:01,153][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:06,155][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:11,158][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:16,162][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:21,167][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:26,168][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:31,170][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:36,172][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:41,180][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:46,183][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:51,186][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:56,188][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:59,258][ INFO][PID:3061076] Allocated host ResallocHost, ticket_id=3851077, hostname=54.224.243.251, name=aws_x86_64_spot_prod_13021497_20260710_153118, requested_tags=['arch_x86_64', 'copr_builder'] [2026-07-10 15:32:59,259][ INFO][PID:3061076] Allocating ssh connection to builder [2026-07-10 15:32:59,260][ INFO][PID:3061076] Checking that builder machine is OK [2026-07-10 15:32:59,579][ INFO][PID:3061076] Installed copr-rpmbuild version: 1.8 [2026-07-10 15:32:59,580][ INFO][PID:3061076] Running remote command: copr-builder-ready fedora-44-x86_64 [2026-07-10 15:32:59,771][ INFO][PID:3061076] Red Hat subscription not needed for fedora-44-x86_64 Builder is ready to be used [2026-07-10 15:32:59,773][ INFO][PID:3061076] Filling build.info file with builder info [2026-07-10 15:32:59,774][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:32:59,778][ INFO][PID:3061076] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "szydell", "project_name": "subsurface-to-ssi-qr", "project_dirname": "subsurface-to-ssi-qr", "submitter": "szydell", "ended_on": null, "started_on": 1783697579.7786067, "submitted_on": null, "status": 3, "chroot": "fedora-44-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "10706546-fedora-44-x86_64", "build_id": 10706546, "package_name": "subsurface-to-ssi-qr", "package_version": "1.1.2-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/szydell/subsurface-to-ssi-qr/subsurface-to-ssi-qr", "git_hash": "c321f0e59c7de5263a968945c28bf6999258eb52", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "szydell/subsurface-to-ssi-qr--szydell", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/szydell/subsurface-to-ssi-qr", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/szydell/subsurface-to-ssi-qr", "result_dir": "10706546-subsurface-to-ssi-qr", "built_packages": "", "rpmeta_prediction": null, "id": 10706546, "mockchain_macros": { "copr_username": "szydell", "copr_projectname": "subsurface-to-ssi-qr", "vendor": "Fedora Project COPR (szydell/subsurface-to-ssi-qr)" } } ] } [2026-07-10 15:32:59,849][ INFO][PID:3061076] Sending fedora-messaging bus message in build.start [2026-07-10 15:33:00,829][ INFO][PID:3061076] Sending fedora-messaging bus message in chroot.start [2026-07-10 15:33:00,859][ INFO][PID:3061076] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10706546-fedora-44-x86_64 --chroot fedora-44-x86_64 --detached [2026-07-10 15:33:01,417][ INFO][PID:3061076] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '54.224.243.251' (ED25519) to the list of known hosts. [2026-07-10 15:33:01,419][ INFO][PID:3061076] Downloading the builder-live.log file, attempt 1 [2026-07-10 15:33:01,425][ INFO][PID:3061076] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.224.243.251 copr-rpmbuild-log [2026-07-10 15:33:06,459][ INFO][PID:3061076] Periodic builder liveness probe: alive [2026-07-10 15:33:06,460][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:11,462][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:16,464][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:21,467][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:26,474][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:31,477][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:36,488][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:41,494][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:46,504][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:51,517][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:33:56,519][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:01,522][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:06,523][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:11,526][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:16,536][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:21,552][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:26,557][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:31,559][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:36,561][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:41,563][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:46,564][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:48,218][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-07-10 15:34:51,566][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:56,567][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:34:59,089][WARNING][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-07-10 15:35:01,578][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:06,583][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:09,104][ INFO][PID:3061076] Downloading the builder-live.log file, attempt 2 [2026-07-10 15:35:09,268][ INFO][PID:3061076] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.224.243.251 copr-rpmbuild-log [2026-07-10 15:35:09,270][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-07-10 15:35:11,584][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:16,587][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:19,461][WARNING][PID:3061076] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-07-10 15:35:21,589][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:26,590][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:29,462][ INFO][PID:3061076] Downloading the builder-live.log file, attempt 3 [2026-07-10 15:35:29,464][ INFO][PID:3061076] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.224.243.251 copr-rpmbuild-log [2026-07-10 15:35:29,475][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-07-10 15:35:31,592][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:36,659][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:39,488][WARNING][PID:3061076] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-07-10 15:35:41,661][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:46,662][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:49,489][ INFO][PID:3061076] Downloading the builder-live.log file, attempt 4 [2026-07-10 15:35:49,491][ INFO][PID:3061076] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.224.243.251 copr-rpmbuild-log [2026-07-10 15:35:51,702][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:35:56,773][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:01,896][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:06,901][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:11,915][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:16,919][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:21,941][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:26,944][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:31,945][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:36,947][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:42,114][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:47,116][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:49,561][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-07-10 15:36:52,126][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:57,132][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:36:59,610][ ERROR][PID:3061076] Giving up for unstable SSH, failures: 1783697699.0898926, 1783697719.4613116, 1783697739.4885538, 1783697819.5949876 [2026-07-10 15:36:59,658][ ERROR][PID:3061076] Re-try request for task on 'ResallocHost, ticket_id=3851077, hostname=54.224.243.251, name=aws_x86_64_spot_prod_13021497_20260710_153118, requested_tags=['arch_x86_64', 'copr_builder']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-07-10 15:36:59,688][ INFO][PID:3061076] VM Release request [2026-07-10 15:37:08,380][ INFO][PID:3061076] Retry #1 (on other host) [2026-07-10 15:37:08,420][ INFO][PID:3061076] Checking that https://download.copr.fedorainfracloud.org/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64//repodata/repomd.xml exists [2026-07-10 15:37:09,471][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:09,732][WARNING][PID:3061076] rpmeta: cannot connect to https://rpmeta.fedoraproject.org, skipping [2026-07-10 15:37:09,733][ INFO][PID:3061076] VM allocation process starts [2026-07-10 15:37:10,414][ INFO][PID:3061076] Trying to allocate VM: ResallocHost, ticket_id=3851202, requested_tags=['arch_x86_64', 'copr_builder'] [2026-07-10 15:37:15,416][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:20,418][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:25,422][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:30,426][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:35,427][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:40,430][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:45,433][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:50,479][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:37:50,712][ INFO][PID:3061076] Allocated host ResallocHost, ticket_id=3851202, hostname=54.224.243.251, name=aws_x86_64_spot_prod_13021497_20260710_153118, requested_tags=['arch_x86_64', 'copr_builder'] [2026-07-10 15:37:50,720][ INFO][PID:3061076] Allocating ssh connection to builder [2026-07-10 15:37:50,734][ INFO][PID:3061076] Checking that builder machine is OK [2026-07-10 15:38:50,847][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 54.224.243.251 port 22: Connection timed out [2026-07-10 15:39:00,980][ ERROR][PID:3061076] Re-try request for task on 'ResallocHost, ticket_id=3851202, hostname=54.224.243.251, name=aws_x86_64_spot_prod_13021497_20260710_153118, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 1 SSH attempts [2026-07-10 15:39:01,013][ INFO][PID:3061076] VM Release request [2026-07-10 15:39:01,178][ INFO][PID:3061076] Retry #2 (on other host) [2026-07-10 15:39:01,179][ INFO][PID:3061076] Checking that https://download.copr.fedorainfracloud.org/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64//repodata/repomd.xml exists [2026-07-10 15:39:01,424][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:39:01,446][WARNING][PID:3061076] rpmeta: cannot connect to https://rpmeta.fedoraproject.org, skipping [2026-07-10 15:39:01,447][ INFO][PID:3061076] VM allocation process starts [2026-07-10 15:39:01,609][ INFO][PID:3061076] Trying to allocate VM: ResallocHost, ticket_id=3851233, requested_tags=['arch_x86_64', 'copr_builder'] [2026-07-10 15:39:06,611][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:39:07,636][ INFO][PID:3061076] Allocated host ResallocHost, ticket_id=3851233, hostname=100.53.124.67, name=aws_x86_64_spot_prod_13021498_20260710_153124, requested_tags=['arch_x86_64', 'copr_builder'] [2026-07-10 15:39:07,638][ INFO][PID:3061076] Allocating ssh connection to builder [2026-07-10 15:39:07,640][ INFO][PID:3061076] Checking that builder machine is OK [2026-07-10 15:40:07,765][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 100.53.124.67 port 22: Connection timed out [2026-07-10 15:40:17,827][ ERROR][PID:3061076] Re-try request for task on 'ResallocHost, ticket_id=3851233, hostname=100.53.124.67, name=aws_x86_64_spot_prod_13021498_20260710_153124, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 1 SSH attempts [2026-07-10 15:40:17,870][ INFO][PID:3061076] VM Release request [2026-07-10 15:40:17,926][ INFO][PID:3061076] Retry #3 (on other host) [2026-07-10 15:40:17,928][ INFO][PID:3061076] Checking that https://download.copr.fedorainfracloud.org/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64//repodata/repomd.xml exists [2026-07-10 15:40:19,074][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:40:19,117][WARNING][PID:3061076] rpmeta: cannot connect to https://rpmeta.fedoraproject.org, skipping [2026-07-10 15:40:19,120][ INFO][PID:3061076] VM allocation process starts [2026-07-10 15:40:19,186][ INFO][PID:3061076] Trying to allocate VM: ResallocHost, ticket_id=3851272, requested_tags=['arch_x86_64', 'copr_builder'] [2026-07-10 15:40:24,194][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:40:29,196][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:40:31,244][ INFO][PID:3061076] Allocated host ResallocHost, ticket_id=3851272, hostname=54.196.224.1, name=aws_x86_64_spot_prod_13021867_20260710_153907, requested_tags=['arch_x86_64', 'copr_builder'] [2026-07-10 15:40:31,245][ INFO][PID:3061076] Allocating ssh connection to builder [2026-07-10 15:40:31,246][ INFO][PID:3061076] Checking that builder machine is OK [2026-07-10 15:40:31,571][ INFO][PID:3061076] Installed copr-rpmbuild version: 1.8 [2026-07-10 15:40:31,676][ INFO][PID:3061076] Running remote command: copr-builder-ready fedora-44-x86_64 [2026-07-10 15:40:31,923][ INFO][PID:3061076] Red Hat subscription not needed for fedora-44-x86_64 Builder is ready to be used [2026-07-10 15:40:31,933][ INFO][PID:3061076] Filling build.info file with builder info [2026-07-10 15:40:31,999][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:40:32,019][ INFO][PID:3061076] Not re-notifying FE and msg buses for the new host. [2026-07-10 15:40:32,032][ INFO][PID:3061076] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10706546-fedora-44-x86_64 --chroot fedora-44-x86_64 --detached [2026-07-10 15:40:32,521][ INFO][PID:3061076] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '54.196.224.1' (ED25519) to the list of known hosts. [2026-07-10 15:40:32,527][ INFO][PID:3061076] Downloading the builder-live.log file, attempt 1 [2026-07-10 15:40:32,537][ INFO][PID:3061076] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.196.224.1 copr-rpmbuild-log [2026-07-10 15:40:37,536][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:40:42,538][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:40:47,539][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:40:52,542][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:40:57,548][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:02,566][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:07,570][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:12,572][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:17,574][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:22,576][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:27,584][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:32,587][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:37,588][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:42,589][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:47,591][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:52,592][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:41:57,595][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:02,605][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:07,606][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:12,608][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:17,610][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:22,611][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:27,613][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:32,614][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:37,616][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:42,621][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:47,623][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:52,625][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:42:56,846][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-07-10 15:42:57,627][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:02,631][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:07,142][WARNING][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-07-10 15:43:07,645][ INFO][PID:3061076] Periodic builder liveness probe: alive [2026-07-10 15:43:07,651][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:12,653][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:17,143][ INFO][PID:3061076] Downloading the builder-live.log file, attempt 2 [2026-07-10 15:43:17,147][ INFO][PID:3061076] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.196.224.1 copr-rpmbuild-log [2026-07-10 15:43:17,156][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-07-10 15:43:17,655][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:22,656][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:27,176][WARNING][PID:3061076] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-07-10 15:43:27,658][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:32,659][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:37,177][ INFO][PID:3061076] Downloading the builder-live.log file, attempt 3 [2026-07-10 15:43:37,181][ INFO][PID:3061076] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.196.224.1 copr-rpmbuild-log [2026-07-10 15:43:37,190][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-07-10 15:43:37,660][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:42,662][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:47,227][WARNING][PID:3061076] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-07-10 15:43:47,664][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:52,665][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:43:57,228][ INFO][PID:3061076] Downloading the builder-live.log file, attempt 4 [2026-07-10 15:43:57,234][ INFO][PID:3061076] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.196.224.1 copr-rpmbuild-log [2026-07-10 15:43:57,666][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:02,814][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:07,816][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:12,817][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:17,819][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:22,986][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:28,832][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:34,149][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:39,648][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:44,649][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:49,657][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:54,658][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:44:57,302][ ERROR][PID:3061076] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-07-10 15:44:59,661][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:45:04,664][ INFO][PID:3061076] Checking for cancel request [2026-07-10 15:45:07,337][ ERROR][PID:3061076] Giving up for unstable SSH, failures: 1783698187.142641, 1783698207.1760762, 1783698227.2271023, 1783698307.3272932 [2026-07-10 15:45:07,377][ ERROR][PID:3061076] Re-try request for task on 'ResallocHost, ticket_id=3851272, hostname=54.196.224.1, name=aws_x86_64_spot_prod_13021867_20260710_153907, requested_tags=['arch_x86_64', 'copr_builder']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-07-10 15:45:07,418][ INFO][PID:3061076] VM Release request [2026-07-10 15:45:07,432][ ERROR][PID:3061076] Backend process error: Three host tried without success: {'100.53.124.67', '54.224.243.251', '54.196.224.1'} [2026-07-10 15:45:07,451][WARNING][PID:3061076] Switching not-finished job state to 'failed' [2026-07-10 15:45:07,452][ INFO][PID:3061076] Worker failed build, took 275.44976329803467 [2026-07-10 15:45:07,456][ INFO][PID:3061076] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "szydell", "project_name": "subsurface-to-ssi-qr", "project_dirname": "subsurface-to-ssi-qr", "submitter": "szydell", "ended_on": 1783698307.4516685, "started_on": 1783698032.0019052, "submitted_on": null, "status": 0, "chroot": "fedora-44-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "10706546-fedora-44-x86_64", "build_id": 10706546, "package_name": "subsurface-to-ssi-qr", "package_version": "1.1.2-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/szydell/subsurface-to-ssi-qr/subsurface-to-ssi-qr", "git_hash": "c321f0e59c7de5263a968945c28bf6999258eb52", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "szydell/subsurface-to-ssi-qr--szydell", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/szydell/subsurface-to-ssi-qr", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/szydell/subsurface-to-ssi-qr", "result_dir": "10706546-subsurface-to-ssi-qr", "built_packages": "", "rpmeta_prediction": null, "id": 10706546, "mockchain_macros": { "copr_username": "szydell", "copr_projectname": "subsurface-to-ssi-qr", "vendor": "Fedora Project COPR (szydell/subsurface-to-ssi-qr)" } } ] } [2026-07-10 15:45:07,518][ INFO][PID:3061076] Sending fedora-messaging bus message in build.end [2026-07-10 15:45:07,881][ INFO][PID:3061076] Compressing /var/lib/copr/public_html/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64/10706546-subsurface-to-ssi-qr/builder-live.log by gzip [2026-07-10 15:45:07,892][ INFO][PID:3061076] Running command 'gzip /var/lib/copr/public_html/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64/10706546-subsurface-to-ssi-qr/builder-live.log' as PID 3165808 [2026-07-10 15:45:07,911][ INFO][PID:3061076] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64/10706546-subsurface-to-ssi-qr/builder-live.log) [2026-07-10 15:45:07,915][ INFO][PID:3061076] Compressing /var/lib/copr/public_html/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64/10706546-subsurface-to-ssi-qr/backend.log by gzip [2026-07-10 15:45:07,917][ INFO][PID:3061076] Running command 'gzip /var/lib/copr/public_html/results/szydell/subsurface-to-ssi-qr/fedora-44-x86_64/10706546-subsurface-to-ssi-qr/backend.log' as PID 3165811