[2025-08-26 17:05:39,895][ INFO][PID:1913287] Marking build as starting [2025-08-26 17:05:39,980][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:05:39,981][ INFO][PID:1913287] VM allocation process starts [2025-08-26 17:05:40,004][ INFO][PID:1913287] Trying to allocate VM: ResallocHost, ticket_id=4764211, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-26 17:05:45,009][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:05:50,013][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:05:55,014][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:00,016][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:05,018][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:10,019][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:15,022][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:20,023][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:25,025][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:30,027][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:35,029][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:40,031][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:45,032][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:50,034][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:06:55,037][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:00,043][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:05,044][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:10,047][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:15,051][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:20,055][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:25,058][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:30,059][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:35,061][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:40,063][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:45,064][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:50,069][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:07:55,071][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:00,072][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:05,073][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:10,075][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:15,078][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:20,082][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:25,085][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:30,086][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:35,090][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:40,092][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:45,093][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:50,094][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:55,095][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:58,138][ INFO][PID:1913287] Allocated host ResallocHost, ticket_id=4764211, hostname=54.221.64.201, name=aws_x86_64_spot_prod_06444283_20250826_170647, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-26 17:08:58,139][ INFO][PID:1913287] Allocating ssh connection to builder [2025-08-26 17:08:58,140][ INFO][PID:1913287] Checking that builder machine is OK [2025-08-26 17:08:58,592][ INFO][PID:1913287] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-08-26 17:08:58,801][ INFO][PID:1913287] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-08-26 17:08:58,802][ INFO][PID:1913287] Filling build.info file with builder info [2025-08-26 17:08:58,803][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:08:58,804][ INFO][PID:1913287] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "python3-scipy-1.11.1", "project_dirname": "python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17", "submitter": "https://src.fedoraproject.org/user/packit", "ended_on": null, "started_on": 1756228138.8041813, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9493062-fedora-rawhide-x86_64", "build_id": 9493062, "package_name": "python-lsp-server", "package_version": "1.13.1-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/python-lsp-server", "git_hash": "b12f2c6029dc02117afb01c56c7202247cd3b97b", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/python3-scipy-1.11.1--https://src.fedoraproject.org/user/packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/fedora-rawhide-x86_64/", "id": "copr_coprdir", "name": "Coprdir repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17", "result_dir": "09493062-python-lsp-server", "built_packages": "", "id": 9493062, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "python3-scipy-1.11.1", "vendor": "Fedora Project COPR (psimovec/python3-scipy-1.11.1)" } } ] } [2025-08-26 17:08:58,862][ INFO][PID:1913287] Sending fedora-messaging bus message in build.start [2025-08-26 17:08:59,588][ INFO][PID:1913287] Sending fedora-messaging bus message in chroot.start [2025-08-26 17:08:59,616][ INFO][PID:1913287] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9493062-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-08-26 17:08:59,961][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 1 [2025-08-26 17:08:59,964][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.221.64.201 copr-rpmbuild-log [2025-08-26 17:09:04,970][ INFO][PID:1913287] Periodic builder liveness probe: alive [2025-08-26 17:09:04,971][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:09,973][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:14,975][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:19,976][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:24,977][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:29,978][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:34,980][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:39,982][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:44,983][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:49,984][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:54,986][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:09:59,988][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:04,989][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:09,990][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:14,991][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:19,992][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:24,995][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:29,996][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:34,997][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:39,998][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:45,000][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:50,002][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:10:55,003][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:00,004][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:05,005][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:10,008][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:15,010][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:20,013][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:25,014][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:30,017][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:35,022][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:40,023][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:45,025][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:50,026][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:11:55,027][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:00,028][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:05,029][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:10,030][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:15,032][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:20,033][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:25,035][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:30,036][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:35,038][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:40,039][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:45,043][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:50,044][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:12:55,046][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:00,047][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:05,049][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:10,050][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:15,056][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:20,061][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:25,065][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:30,068][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:35,069][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:40,071][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:45,073][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:50,075][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:13:55,079][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:00,081][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:05,083][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:10,087][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:15,089][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:20,091][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:25,092][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:30,096][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:35,101][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:40,107][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:45,109][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:50,111][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:14:55,112][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:00,115][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:05,118][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:10,128][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:15,132][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:20,134][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:25,138][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:30,140][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:35,142][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:40,143][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:45,144][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:50,146][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:15:55,148][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:00,149][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:05,150][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:10,153][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:15,154][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:20,156][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:25,157][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:30,159][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:35,160][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:40,161][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:45,162][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:50,164][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:16:55,165][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:00,167][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:05,168][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:10,169][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:15,170][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:20,171][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:25,173][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:30,174][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:35,175][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:40,176][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:45,178][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:50,181][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:17:55,183][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:00,185][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:05,188][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:10,191][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:15,203][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:20,204][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:25,207][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:30,208][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:35,209][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:40,212][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:45,213][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:50,216][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:18:55,217][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:00,219][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:05,228][ INFO][PID:1913287] Periodic builder liveness probe: alive [2025-08-26 17:19:05,229][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:10,232][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:15,234][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:20,235][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:25,236][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:30,238][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:35,239][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:40,240][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:45,244][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:50,246][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:19:55,252][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:00,255][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:05,259][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:10,260][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:15,261][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:20,263][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:25,264][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:30,266][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:35,267][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:40,268][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:45,269][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:50,271][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:20:55,272][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:00,273][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:05,274][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:10,275][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:15,277][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:20,278][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:25,279][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:30,280][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:35,284][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:40,285][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:45,286][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:50,288][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:21:55,289][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:00,290][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:05,291][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:10,292][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:15,294][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:20,294][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:25,296][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:30,297][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:35,298][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:40,300][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:45,301][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:50,302][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:22:52,648][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:22:55,303][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:00,304][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:02,649][WARNING][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:23:05,305][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:10,307][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:12,650][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 2 [2025-08-26 17:23:12,652][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.221.64.201 copr-rpmbuild-log [2025-08-26 17:23:15,308][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:20,309][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:25,310][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:30,312][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:35,313][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:40,315][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:45,320][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:50,321][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:23:55,322][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:00,323][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:05,324][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:10,326][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:12,720][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:24:15,328][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:20,330][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:22,725][WARNING][PID:1913287] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:24:25,332][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:30,333][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:32,727][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 3 [2025-08-26 17:24:32,729][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.221.64.201 copr-rpmbuild-log [2025-08-26 17:24:35,334][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:40,335][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:45,336][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:50,337][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:24:55,339][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:00,340][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:05,341][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:10,342][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:15,344][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:20,345][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:25,346][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:30,347][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:32,796][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:25:35,348][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:40,349][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:42,799][WARNING][PID:1913287] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:25:45,350][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:50,351][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:25:52,800][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 4 [2025-08-26 17:25:52,801][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.221.64.201 copr-rpmbuild-log [2025-08-26 17:25:55,353][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:00,354][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:05,355][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:10,356][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:15,357][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:20,358][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:25,359][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:30,360][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:35,361][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:40,362][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:45,363][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:50,364][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:26:52,870][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:26:55,365][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:00,366][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:02,874][ ERROR][PID:1913287] Giving up for unstable SSH, failures: 1756228982.64966, 1756229062.7253406, 1756229142.7996109, 1756229222.8739092 [2025-08-26 17:27:02,875][ ERROR][PID:1913287] Re-try request for task on 'ResallocHost, ticket_id=4764211, hostname=54.221.64.201, name=aws_x86_64_spot_prod_06444283_20250826_170647, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-08-26 17:27:02,875][ INFO][PID:1913287] Releasing VM back to pool [2025-08-26 17:27:02,885][ INFO][PID:1913287] Retry #1 (on other host) [2025-08-26 17:27:02,886][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:02,886][ INFO][PID:1913287] VM allocation process starts [2025-08-26 17:27:02,901][ INFO][PID:1913287] Trying to allocate VM: ResallocHost, ticket_id=4764563, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-26 17:27:05,921][ INFO][PID:1913287] Allocated host ResallocHost, ticket_id=4764563, hostname=44.200.213.69, name=aws_x86_64_spot_prod_06444750_20250826_172422, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-26 17:27:05,922][ INFO][PID:1913287] Allocating ssh connection to builder [2025-08-26 17:27:05,922][ INFO][PID:1913287] Checking that builder machine is OK [2025-08-26 17:27:06,401][ INFO][PID:1913287] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-08-26 17:27:06,602][ INFO][PID:1913287] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-08-26 17:27:06,603][ INFO][PID:1913287] Filling build.info file with builder info [2025-08-26 17:27:06,607][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:06,608][ INFO][PID:1913287] Not re-notifying FE and msg buses for the new host. [2025-08-26 17:27:06,608][ INFO][PID:1913287] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9493062-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-08-26 17:27:06,928][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 1 [2025-08-26 17:27:06,929][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.213.69 copr-rpmbuild-log [2025-08-26 17:27:11,928][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:16,929][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:21,808][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:27:21,930][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:26,932][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:31,808][WARNING][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:27:31,933][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:36,934][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:41,809][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 2 [2025-08-26 17:27:41,811][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.213.69 copr-rpmbuild-log [2025-08-26 17:27:41,935][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:46,936][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:51,937][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:27:56,940][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:01,941][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:06,942][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:11,943][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:16,944][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:21,946][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:26,947][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:31,948][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:36,949][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:41,878][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:28:41,950][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:46,951][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:51,880][WARNING][PID:1913287] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:28:51,952][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:28:56,953][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:01,881][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 3 [2025-08-26 17:29:01,882][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.213.69 copr-rpmbuild-log [2025-08-26 17:29:01,954][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:06,961][ INFO][PID:1913287] Periodic builder liveness probe: alive [2025-08-26 17:29:06,961][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:11,962][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:16,964][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:21,966][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:26,967][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:31,969][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:36,970][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:41,971][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:46,972][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:51,973][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:29:56,974][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:01,951][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:30:01,975][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:06,976][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:11,952][WARNING][PID:1913287] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:30:11,977][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:16,979][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:21,953][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 4 [2025-08-26 17:30:21,954][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.200.213.69 copr-rpmbuild-log [2025-08-26 17:30:21,980][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:26,981][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:31,982][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:36,984][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:41,985][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:46,986][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:51,988][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:30:56,989][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:01,990][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:06,991][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:11,992][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:16,993][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:21,994][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:22,025][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:31:26,995][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:31,997][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:32,027][ ERROR][PID:1913287] Giving up for unstable SSH, failures: 1756229251.808786, 1756229331.8802767, 1756229411.9524589, 1756229492.0274377 [2025-08-26 17:31:32,028][ ERROR][PID:1913287] Re-try request for task on 'ResallocHost, ticket_id=4764563, hostname=44.200.213.69, name=aws_x86_64_spot_prod_06444750_20250826_172422, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-08-26 17:31:32,028][ INFO][PID:1913287] Releasing VM back to pool [2025-08-26 17:31:32,043][ INFO][PID:1913287] Retry #2 (on other host) [2025-08-26 17:31:32,043][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:32,044][ INFO][PID:1913287] VM allocation process starts [2025-08-26 17:31:32,063][ INFO][PID:1913287] Trying to allocate VM: ResallocHost, ticket_id=4764657, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-26 17:31:37,064][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:42,066][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:47,068][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:52,069][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:31:57,070][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:02,071][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:07,072][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:10,145][ INFO][PID:1913287] Allocated host ResallocHost, ticket_id=4764657, hostname=98.82.3.237, name=aws_x86_64_spot_prod_06444844_20250826_173036, requested_tags=['copr_builder', 'arch_x86_64'] [2025-08-26 17:32:10,146][ INFO][PID:1913287] Allocating ssh connection to builder [2025-08-26 17:32:10,146][ INFO][PID:1913287] Checking that builder machine is OK [2025-08-26 17:32:10,581][ INFO][PID:1913287] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2025-08-26 17:32:10,785][ INFO][PID:1913287] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2025-08-26 17:32:10,786][ INFO][PID:1913287] Filling build.info file with builder info [2025-08-26 17:32:10,788][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:10,789][ INFO][PID:1913287] Not re-notifying FE and msg buses for the new host. [2025-08-26 17:32:10,790][ INFO][PID:1913287] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9493062-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2025-08-26 17:32:11,126][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 1 [2025-08-26 17:32:11,129][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.3.237 copr-rpmbuild-log [2025-08-26 17:32:16,127][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:21,129][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:26,133][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:31,134][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:36,135][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:41,138][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:46,139][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:51,140][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:32:56,141][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:01,142][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:06,143][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:11,145][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:16,146][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:21,147][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:26,148][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:31,150][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:36,151][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:41,152][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:46,153][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:51,155][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:33:56,156][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:01,157][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:06,158][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:11,160][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:16,161][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:21,163][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:26,164][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:31,165][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:36,166][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:41,167][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:46,168][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:49,413][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:34:51,169][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:56,170][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:34:59,414][WARNING][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:35:01,171][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:06,172][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:09,415][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 2 [2025-08-26 17:35:09,416][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.3.237 copr-rpmbuild-log [2025-08-26 17:35:11,174][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:16,175][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:21,176][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:26,177][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:31,178][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:36,179][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:41,180][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:46,181][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:51,182][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:35:56,183][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:01,184][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:06,185][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:09,484][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:36:11,186][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:16,187][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:19,488][WARNING][PID:1913287] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:36:21,188][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:26,190][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:29,489][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 3 [2025-08-26 17:36:29,490][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.3.237 copr-rpmbuild-log [2025-08-26 17:36:31,191][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:36,192][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:41,193][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:46,194][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:51,195][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:36:56,196][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:01,197][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:06,198][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:11,199][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:16,200][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:21,201][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:26,202][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:29,558][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:37:31,203][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:36,204][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:39,560][WARNING][PID:1913287] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-08-26 17:37:41,205][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:46,206][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:49,560][ INFO][PID:1913287] Downloading the builder-live.log file, attempt 4 [2025-08-26 17:37:49,562][ INFO][PID:1913287] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.82.3.237 copr-rpmbuild-log [2025-08-26 17:37:51,207][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:37:56,208][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:01,210][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:06,210][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:11,212][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:16,212][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:21,214][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:26,215][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:31,215][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:36,216][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:41,218][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:46,219][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:49,629][ ERROR][PID:1913287] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-08-26 17:38:51,220][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:56,221][ INFO][PID:1913287] Checking for cancel request [2025-08-26 17:38:59,635][ ERROR][PID:1913287] Giving up for unstable SSH, failures: 1756229699.4146857, 1756229779.488702, 1756229859.5600452, 1756229939.6350543 [2025-08-26 17:38:59,635][ ERROR][PID:1913287] Re-try request for task on 'ResallocHost, ticket_id=4764657, hostname=98.82.3.237, name=aws_x86_64_spot_prod_06444844_20250826_173036, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-08-26 17:38:59,636][ INFO][PID:1913287] Releasing VM back to pool [2025-08-26 17:38:59,645][ ERROR][PID:1913287] Backend process error: Three host tried without success: {'54.221.64.201', '44.200.213.69', '98.82.3.237'} [2025-08-26 17:38:59,646][WARNING][PID:1913287] Switching not-finished job state to 'failed' [2025-08-26 17:38:59,646][ INFO][PID:1913287] Worker failed build, took 408.8564760684967 [2025-08-26 17:38:59,646][ INFO][PID:1913287] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "python3-scipy-1.11.1", "project_dirname": "python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17", "submitter": "https://src.fedoraproject.org/user/packit", "ended_on": 1756229939.646019, "started_on": 1756229530.789543, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9493062-fedora-rawhide-x86_64", "build_id": 9493062, "package_name": "python-lsp-server", "package_version": "1.13.1-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/python-lsp-server", "git_hash": "b12f2c6029dc02117afb01c56c7202247cd3b97b", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/python3-scipy-1.11.1--https://src.fedoraproject.org/user/packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/fedora-rawhide-x86_64/", "id": "copr_coprdir", "name": "Coprdir repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17", "result_dir": "09493062-python-lsp-server", "built_packages": "", "id": 9493062, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "python3-scipy-1.11.1", "vendor": "Fedora Project COPR (psimovec/python3-scipy-1.11.1)" } } ] } [2025-08-26 17:38:59,695][ INFO][PID:1913287] Sending fedora-messaging bus message in build.end [2025-08-26 17:38:59,718][ INFO][PID:1913287] Compressing /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/fedora-rawhide-x86_64/09493062-python-lsp-server/builder-live.log by gzip [2025-08-26 17:38:59,719][ INFO][PID:1913287] Running command 'gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/fedora-rawhide-x86_64/09493062-python-lsp-server/builder-live.log' as PID 2151365 [2025-08-26 17:38:59,721][ INFO][PID:1913287] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/fedora-rawhide-x86_64/09493062-python-lsp-server/builder-live.log) [2025-08-26 17:38:59,721][ INFO][PID:1913287] Compressing /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/fedora-rawhide-x86_64/09493062-python-lsp-server/backend.log by gzip [2025-08-26 17:38:59,722][ INFO][PID:1913287] Running command 'gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1:pr:fork-packit-rpms-python-lsp-server-17/fedora-rawhide-x86_64/09493062-python-lsp-server/backend.log' as PID 2151366