[2025-12-09 10:53:34,803][ INFO][PID:740199] Marking build as starting [2025-12-09 10:53:34,870][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:53:34,872][ INFO][PID:740199] VM allocation process starts [2025-12-09 10:53:34,883][ INFO][PID:740199] Trying to allocate VM: ResallocHost, ticket_id=234244, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-09 10:53:37,900][ INFO][PID:740199] Allocated host ResallocHost, ticket_id=234244, hostname=34.239.185.131, name=aws_x86_64_spot_prod_00251314_20251209_104237, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-09 10:53:37,900][ INFO][PID:740199] Allocating ssh connection to builder [2025-12-09 10:53:37,901][ INFO][PID:740199] Checking that builder machine is OK [2025-12-09 10:53:38,226][ INFO][PID:740199] Running remote command: copr-builder-ready fedora-43-x86_64 [2025-12-09 10:53:38,415][ INFO][PID:740199] Red Hat subscription not needed for fedora-43-x86_64 Builder is ready to be used [2025-12-09 10:53:38,415][ INFO][PID:740199] Filling build.info file with builder info [2025-12-09 10:53:38,416][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:53:38,418][ INFO][PID:740199] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "openblas-0.3.23", "project_dirname": "openblas-0.3.23", "submitter": "https://src.fedoraproject.org/user/music", "ended_on": null, "started_on": 1765277618.417891, "submitted_on": null, "status": 3, "chroot": "fedora-43-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9889551-fedora-43-x86_64", "build_id": 9889551, "package_name": "snakemake", "package_version": "9.14.4-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/snakemake", "git_hash": "a834930c25e151fd0f50c0c2a5847f632da0aa01", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/music", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-43-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23", "result_dir": "09889551-snakemake", "built_packages": "", "id": 9889551, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "openblas-0.3.23", "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)" } } ] } [2025-12-09 10:53:38,473][ INFO][PID:740199] Sending fedora-messaging bus message in build.start [2025-12-09 10:53:39,130][ INFO][PID:740199] Sending fedora-messaging bus message in chroot.start [2025-12-09 10:53:39,157][ INFO][PID:740199] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9889551-fedora-43-x86_64 --chroot fedora-43-x86_64 --detached [2025-12-09 10:53:39,465][ INFO][PID:740199] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '34.239.185.131' (ED25519) to the list of known hosts. [2025-12-09 10:53:39,466][ INFO][PID:740199] Downloading the builder-live.log file, attempt 1 [2025-12-09 10:53:39,467][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.239.185.131 copr-rpmbuild-log [2025-12-09 10:53:44,473][ INFO][PID:740199] Periodic builder liveness probe: alive [2025-12-09 10:53:44,473][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:53:49,474][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:53:54,475][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:53:59,476][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:04,477][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:09,478][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:14,479][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:19,480][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:24,481][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:29,482][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:34,483][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:39,484][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:44,485][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:49,486][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:54,487][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:54:59,488][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:04,489][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:09,490][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:14,491][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:19,492][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:24,493][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:29,494][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:34,495][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:39,496][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:44,496][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:49,497][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:54,498][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:55:59,499][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:04,500][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:09,501][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:14,502][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:19,503][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:24,504][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:29,505][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:34,506][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:39,507][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:44,508][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:49,508][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:54,509][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:56:59,510][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:04,511][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:09,513][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:14,514][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:19,515][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:24,515][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:29,516][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:34,517][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:39,518][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:44,519][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:49,520][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:54,521][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:57:59,523][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:04,523][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:09,524][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:14,525][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:19,526][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:24,528][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:29,530][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:34,531][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:39,532][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:44,532][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:49,533][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:54,534][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:58:59,535][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:04,539][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:09,543][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:14,544][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:19,546][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:24,548][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:29,549][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:34,550][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:38,432][ ERROR][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-09 10:59:39,551][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:44,552][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:48,433][WARNING][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-09 10:59:49,553][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:54,554][ INFO][PID:740199] Checking for cancel request [2025-12-09 10:59:58,434][ INFO][PID:740199] Downloading the builder-live.log file, attempt 2 [2025-12-09 10:59:58,436][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.239.185.131 copr-rpmbuild-log [2025-12-09 10:59:59,556][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:04,557][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:09,558][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:14,559][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:19,560][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:24,563][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:29,565][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:34,566][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:39,568][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:44,569][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:49,570][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:54,571][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:00:58,505][ ERROR][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-09 11:00:59,572][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:04,573][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:08,510][WARNING][PID:740199] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-09 11:01:09,574][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:14,575][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:18,511][ INFO][PID:740199] Downloading the builder-live.log file, attempt 3 [2025-12-09 11:01:18,512][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.239.185.131 copr-rpmbuild-log [2025-12-09 11:01:19,576][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:24,577][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:29,578][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:34,600][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:39,602][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:44,605][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:49,607][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:54,608][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:01:59,610][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:04,611][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:09,612][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:14,613][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:18,579][ ERROR][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-09 11:02:19,614][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:24,615][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:28,584][WARNING][PID:740199] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-09 11:02:29,616][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:34,617][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:38,584][ INFO][PID:740199] Downloading the builder-live.log file, attempt 4 [2025-12-09 11:02:38,585][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.239.185.131 copr-rpmbuild-log [2025-12-09 11:02:39,618][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:44,619][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:49,620][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:54,621][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:02:59,622][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:04,623][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:09,624][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:14,625][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:19,626][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:24,627][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:29,629][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:34,631][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:38,653][ ERROR][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-09 11:03:39,632][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:44,638][ INFO][PID:740199] Periodic builder liveness probe: alive [2025-12-09 11:03:44,638][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:48,657][ ERROR][PID:740199] Giving up for unstable SSH, failures: 1765277988.4338028, 1765278068.5101202, 1765278148.584007, 1765278228.656958 [2025-12-09 11:03:48,658][ ERROR][PID:740199] Re-try request for task on 'ResallocHost, ticket_id=234244, hostname=34.239.185.131, name=aws_x86_64_spot_prod_00251314_20251209_104237, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-09 11:03:48,659][ INFO][PID:740199] Releasing VM back to pool [2025-12-09 11:03:48,669][ INFO][PID:740199] Retry #1 (on other host) [2025-12-09 11:03:48,670][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:48,670][ INFO][PID:740199] VM allocation process starts [2025-12-09 11:03:48,686][ INFO][PID:740199] Trying to allocate VM: ResallocHost, ticket_id=234386, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-09 11:03:51,704][ INFO][PID:740199] Allocated host ResallocHost, ticket_id=234386, hostname=3.91.8.103, name=aws_x86_64_spot_prod_00251671_20251209_110027, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-09 11:03:51,705][ INFO][PID:740199] Allocating ssh connection to builder [2025-12-09 11:03:51,705][ INFO][PID:740199] Checking that builder machine is OK [2025-12-09 11:03:52,060][ INFO][PID:740199] Running remote command: copr-builder-ready fedora-43-x86_64 [2025-12-09 11:03:52,270][ INFO][PID:740199] Red Hat subscription not needed for fedora-43-x86_64 Builder is ready to be used [2025-12-09 11:03:52,270][ INFO][PID:740199] Filling build.info file with builder info [2025-12-09 11:03:52,277][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:03:52,279][ INFO][PID:740199] Not re-notifying FE and msg buses for the new host. [2025-12-09 11:03:52,279][ INFO][PID:740199] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9889551-fedora-43-x86_64 --chroot fedora-43-x86_64 --detached [2025-12-09 11:03:52,639][ INFO][PID:740199] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.91.8.103' (ED25519) to the list of known hosts. [2025-12-09 11:03:52,640][ INFO][PID:740199] Downloading the builder-live.log file, attempt 1 [2025-12-09 11:03:52,641][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.91.8.103 copr-rpmbuild-log [2025-12-09 11:03:57,640][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:02,641][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:07,642][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:12,643][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:17,644][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:22,645][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:27,646][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:32,647][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:37,648][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:42,649][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:47,650][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:52,651][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:04:57,652][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:02,653][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:07,654][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:12,656][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:17,657][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:22,659][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:27,660][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:32,661][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:37,662][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:42,663][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:47,664][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:52,666][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:05:57,667][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:02,668][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:07,670][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:12,671][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:17,672][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:22,673][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:27,674][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:32,675][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:37,676][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:42,677][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:47,678][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:52,679][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:06:57,680][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:02,681][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:07,682][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:12,684][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:17,685][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:22,688][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:27,690][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:32,694][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:37,697][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:42,699][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:47,700][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:52,701][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:07:57,702][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:02,703][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:07,706][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:12,707][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:17,708][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:22,709][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:27,710][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:32,711][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:37,712][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:42,713][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:47,714][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:52,716][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:08:57,717][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:02,720][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:07,721][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:12,722][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:17,723][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:22,724][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:27,724][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:32,725][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:37,727][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:42,728][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:47,728][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:52,730][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:09:57,732][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:02,733][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:07,734][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:12,735][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:17,736][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:22,737][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:27,738][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:32,739][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:37,740][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:42,741][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:47,742][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:52,743][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:10:57,744][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:02,745][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:07,746][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:12,748][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:17,750][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:22,755][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:27,756][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:32,761][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:37,763][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:42,766][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:47,767][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:52,769][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:11:57,777][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:02,778][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:07,780][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:12,783][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:17,787][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:22,788][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:27,790][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:32,791][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:37,793][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:40,754][ ERROR][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-09 11:12:42,794][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:47,795][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:50,756][WARNING][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-09 11:12:52,800][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:12:57,802][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:00,756][ INFO][PID:740199] Downloading the builder-live.log file, attempt 2 [2025-12-09 11:13:00,758][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.91.8.103 copr-rpmbuild-log [2025-12-09 11:13:02,803][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:07,805][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:12,809][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:17,810][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:22,811][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:27,812][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:32,814][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:37,816][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:42,817][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:47,827][ INFO][PID:740199] Periodic builder liveness probe: alive [2025-12-09 11:13:47,828][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:52,829][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:13:57,830][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:00,826][ ERROR][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-09 11:14:02,832][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:07,834][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:10,830][WARNING][PID:740199] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-09 11:14:12,835][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:17,837][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:20,830][ INFO][PID:740199] Downloading the builder-live.log file, attempt 3 [2025-12-09 11:14:20,832][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.91.8.103 copr-rpmbuild-log [2025-12-09 11:14:22,839][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:27,840][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:32,842][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:37,850][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:42,851][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:47,852][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:52,854][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:14:57,858][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:02,860][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:07,862][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:12,863][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:17,865][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:20,901][ ERROR][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-09 11:15:22,867][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:27,869][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:30,902][WARNING][PID:740199] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-09 11:15:32,870][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:37,871][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:40,903][ INFO][PID:740199] Downloading the builder-live.log file, attempt 4 [2025-12-09 11:15:40,904][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.91.8.103 copr-rpmbuild-log [2025-12-09 11:15:42,875][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:47,876][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:52,877][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:15:57,879][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:02,880][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:07,881][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:12,883][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:17,886][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:22,887][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:27,888][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:32,890][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:37,891][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:40,971][ ERROR][PID:740199] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-09 11:16:42,892][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:47,893][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:50,977][ ERROR][PID:740199] Giving up for unstable SSH, failures: 1765278770.7559888, 1765278850.8299973, 1765278930.9023554, 1765279010.9777634 [2025-12-09 11:16:50,978][ ERROR][PID:740199] Re-try request for task on 'ResallocHost, ticket_id=234386, hostname=3.91.8.103, name=aws_x86_64_spot_prod_00251671_20251209_110027, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2025-12-09 11:16:50,979][ INFO][PID:740199] Releasing VM back to pool [2025-12-09 11:16:50,992][ INFO][PID:740199] Retry #2 (on other host) [2025-12-09 11:16:50,992][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:50,993][ INFO][PID:740199] VM allocation process starts [2025-12-09 11:16:51,005][ INFO][PID:740199] Trying to allocate VM: ResallocHost, ticket_id=234663, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-09 11:16:56,008][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:57,029][ INFO][PID:740199] Allocated host ResallocHost, ticket_id=234663, hostname=100.30.203.177, name=aws_x86_64_normalreserved_prod_00251955_20251209_111332, requested_tags=['copr_builder', 'arch_x86_64'] [2025-12-09 11:16:57,030][ INFO][PID:740199] Allocating ssh connection to builder [2025-12-09 11:16:57,030][ INFO][PID:740199] Checking that builder machine is OK [2025-12-09 11:16:57,375][ INFO][PID:740199] Running remote command: copr-builder-ready fedora-43-x86_64 [2025-12-09 11:16:57,585][ INFO][PID:740199] Red Hat subscription not needed for fedora-43-x86_64 Builder is ready to be used [2025-12-09 11:16:57,586][ INFO][PID:740199] Filling build.info file with builder info [2025-12-09 11:16:57,587][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:16:57,588][ INFO][PID:740199] Not re-notifying FE and msg buses for the new host. [2025-12-09 11:16:57,589][ INFO][PID:740199] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9889551-fedora-43-x86_64 --chroot fedora-43-x86_64 --detached [2025-12-09 11:16:57,941][ INFO][PID:740199] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '100.30.203.177' (ED25519) to the list of known hosts. [2025-12-09 11:16:57,942][ INFO][PID:740199] Downloading the builder-live.log file, attempt 1 [2025-12-09 11:16:57,943][ INFO][PID:740199] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.30.203.177 copr-rpmbuild-log [2025-12-09 11:17:02,942][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:07,943][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:12,944][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:17,945][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:22,946][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:27,948][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:32,949][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:37,950][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:42,950][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:47,951][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:52,952][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:17:57,953][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:02,955][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:07,956][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:12,957][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:17,958][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:22,962][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:28,127][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:33,128][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:38,129][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:43,130][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:48,131][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:53,132][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:18:58,133][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:03,134][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:08,135][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:13,136][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:18,137][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:23,138][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:28,139][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:33,141][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:38,142][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:43,143][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:48,144][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:53,146][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:19:58,147][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:03,148][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:08,149][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:13,150][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:18,152][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:23,153][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:28,154][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:33,155][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:38,156][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:43,158][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:48,159][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:53,160][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:20:58,161][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:03,162][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:08,163][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:13,164][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:18,165][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:23,166][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:28,167][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:33,168][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:38,169][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:43,170][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:48,171][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:53,172][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:21:58,174][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:03,175][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:08,176][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:13,177][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:18,178][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:23,179][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:28,180][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:33,181][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:38,182][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:43,185][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:48,186][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:53,187][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:22:58,188][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:03,189][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:08,190][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:13,191][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:18,192][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:23,193][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:28,194][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:33,195][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:38,196][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:43,197][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:48,206][ INFO][PID:740199] Periodic builder liveness probe: alive [2025-12-09 11:23:48,207][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:53,207][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:23:58,208][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:03,211][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:08,212][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:13,214][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:18,216][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:23,220][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:28,223][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:33,231][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:38,233][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:43,236][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:48,238][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:53,239][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:24:58,241][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:03,243][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:08,245][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:13,248][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:18,249][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:23,250][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:28,255][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:33,256][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:38,257][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:43,258][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:48,259][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:53,260][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:25:58,261][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:03,262][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:08,263][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:13,265][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:18,266][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:23,267][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:28,270][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:33,274][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:38,275][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:43,277][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:48,282][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:53,290][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:26:58,293][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:03,295][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:08,298][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:13,299][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:18,310][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:23,316][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:28,320][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:33,322][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:38,326][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:43,331][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:48,338][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:53,347][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:27:58,352][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:03,353][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:08,354][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:13,357][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:18,358][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:23,364][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:28,365][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:33,366][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:38,367][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:43,370][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:48,371][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:53,373][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:28:58,374][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:03,375][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:08,376][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:13,377][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:18,378][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:23,384][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:28,385][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:33,386][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:38,386][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:43,388][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:48,394][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:53,395][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:29:58,396][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:30:03,401][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:30:08,402][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:30:13,403][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:30:18,404][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:30:23,405][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:30:28,406][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:30:33,407][ INFO][PID:740199] Checking for cancel request [2025-12-09 11:30:37,927][ INFO][PID:740199] Downloading results from builder [2025-12-09 11:30:37,971][ INFO][PID:740199] rsyncing of mockbuilder@100.30.203.177:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake started [2025-12-09 11:30:38,021][ INFO][PID:740199] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@100.30.203.177:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/ &> /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/build-09889551.rsync.log [2025-12-09 11:30:38,301][ INFO][PID:740199] rsyncing finished. [2025-12-09 11:30:38,310][ INFO][PID:740199] Releasing VM back to pool [2025-12-09 11:30:38,514][ INFO][PID:740199] Searching for 'success' file in resultdir [2025-12-09 11:30:38,534][ INFO][PID:740199] Going to sign pkgs from source: 9889551-fedora-43-x86_64 in chroot: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64 [2025-12-09 11:30:38,554][ INFO][PID:740199] Calling '/bin/sign -u psimovec#openblas-0.3.23@copr.fedorahosted.org -p' (attempt #1) [2025-12-09 11:30:38,759][ INFO][PID:740199] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/snakemake-9.14.4-1.fc44.noarch.rpm' (attempt #1) [2025-12-09 11:30:39,436][ INFO][PID:740199] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/snakemake-9.14.4-1.fc44.noarch.rpm [2025-12-09 11:30:39,437][ INFO][PID:740199] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/snakemake-9.14.4-1.fc44.src.rpm' (attempt #1) [2025-12-09 11:30:40,430][ INFO][PID:740199] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/snakemake-9.14.4-1.fc44.src.rpm [2025-12-09 11:30:40,431][ INFO][PID:740199] Calling '/bin/sign -4 -h sha256 -u psimovec#openblas-0.3.23@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/snakemake+reports-9.14.4-1.fc44.noarch.rpm' (attempt #1) [2025-12-09 11:30:41,162][ INFO][PID:740199] signed rpm: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/snakemake+reports-9.14.4-1.fc44.noarch.rpm [2025-12-09 11:30:41,162][ INFO][PID:740199] Sign done [2025-12-09 11:30:41,182][ INFO][PID:740199] Incremental createrepo run, adding 09889551-snakemake into https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-43-x86_64, (auto-create-repo=True) [2025-12-09 11:30:41,230][ INFO][PID:740199] Running command 'copr-repo --batched /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64 --add 09889551-snakemake --no-appstream-metadata' as PID 973247 [2025-12-09 11:30:52,260][ INFO][PID:740199] Finished after 11 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64 --add 09889551-snakemake --no-appstream-metadata) [2025-12-09 11:30:52,278][ INFO][PID:740199] Getting build details [2025-12-09 11:30:52,279][ INFO][PID:740199] Listing built binary packages in /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake [2025-12-09 11:30:52,280][ INFO][PID:740199] Built packages: ['snakemake+reports 9.14.4', 'snakemake 9.14.4'] [2025-12-09 11:30:52,377][ INFO][PID:740199] build details: {'built_packages': 'snakemake+reports 9.14.4\nsnakemake 9.14.4'} [2025-12-09 11:30:52,381][ INFO][PID:740199] Retrieving pubkey [2025-12-09 11:30:52,383][ INFO][PID:740199] Calling '/bin/sign -u psimovec#openblas-0.3.23@copr.fedorahosted.org -p' (attempt #1) [2025-12-09 11:30:52,662][ INFO][PID:740199] Added pubkey for user psimovec project openblas-0.3.23 into: /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/pubkey.gpg [2025-12-09 11:30:52,668][ INFO][PID:740199] Finished build: id=9889551 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/openblas-0.3.23 chroot=fedora-43-x86_64 [2025-12-09 11:30:52,752][ INFO][PID:740199] Worker succeeded build, took 835.1624701023102 [2025-12-09 11:30:52,763][ INFO][PID:740199] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "psimovec", "project_name": "openblas-0.3.23", "project_dirname": "openblas-0.3.23", "submitter": "https://src.fedoraproject.org/user/music", "ended_on": 1765279852.7510824, "started_on": 1765279017.5886123, "submitted_on": null, "status": 1, "chroot": "fedora-43-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9889551-fedora-43-x86_64", "build_id": 9889551, "package_name": "snakemake", "package_version": "9.14.4-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/openblas-0.3.23/snakemake", "git_hash": "a834930c25e151fd0f50c0c2a5847f632da0aa01", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "psimovec/openblas-0.3.23--https://src.fedoraproject.org/user/music", "results": { "packages": [ { "name": "snakemake", "epoch": null, "version": "9.14.4", "release": "1.fc44", "arch": "src" }, { "name": "snakemake+reports", "epoch": null, "version": "9.14.4", "release": "1.fc44", "arch": "noarch" }, { "name": "snakemake", "epoch": null, "version": "9.14.4", "release": "1.fc44", "arch": "noarch" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23/fedora-43-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/psimovec/openblas-0.3.23", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/openblas-0.3.23", "result_dir": "09889551-snakemake", "built_packages": "snakemake+reports 9.14.4\nsnakemake 9.14.4", "id": 9889551, "mockchain_macros": { "copr_username": "psimovec", "copr_projectname": "openblas-0.3.23", "vendor": "Fedora Project COPR (psimovec/openblas-0.3.23)" } } ] } [2025-12-09 11:30:53,526][ INFO][PID:740199] Sending fedora-messaging bus message in build.end [2025-12-09 11:30:53,942][ INFO][PID:740199] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/builder-live.log by gzip [2025-12-09 11:30:53,946][ INFO][PID:740199] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/builder-live.log' as PID 974706 [2025-12-09 11:30:53,986][ INFO][PID:740199] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/builder-live.log) [2025-12-09 11:30:53,987][ INFO][PID:740199] Compressing /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/backend.log by gzip [2025-12-09 11:30:53,988][ INFO][PID:740199] Running command 'gzip /var/lib/copr/public_html/results/psimovec/openblas-0.3.23/fedora-43-x86_64/09889551-snakemake/backend.log' as PID 974708