[2026-03-09 14:30:57,869][ INFO][PID:3471933] Marking build as starting [2026-03-09 14:30:57,934][ INFO][PID:3471933] Checking that /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/repodata/repomd.xml exists [2026-03-09 14:30:57,971][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:30:57,971][ INFO][PID:3471933] VM allocation process starts [2026-03-09 14:30:58,011][ INFO][PID:3471933] Trying to allocate VM: ResallocHost, ticket_id=1778051, requested_tags=['copr_builder', 'arch_x86_64'] [2026-03-09 14:31:03,020][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:08,021][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:10,082][ INFO][PID:3471933] Allocated host ResallocHost, ticket_id=1778051, hostname=3.91.175.29, name=aws_x86_64_spot_prod_02942164_20260309_142508, requested_tags=['copr_builder', 'arch_x86_64'] [2026-03-09 14:31:10,082][ INFO][PID:3471933] Allocating ssh connection to builder [2026-03-09 14:31:10,083][ INFO][PID:3471933] Checking that builder machine is OK [2026-03-09 14:31:10,404][ INFO][PID:3471933] Running remote command: copr-builder-ready rhel-9-x86_64 [2026-03-09 14:31:10,617][ INFO][PID:3471933] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2026-03-09 14:31:10,618][ INFO][PID:3471933] Filling build.info file with builder info [2026-03-09 14:31:10,619][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:10,619][ INFO][PID:3471933] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@theforeman", "project_name": "develop", "project_dirname": "develop", "submitter": "packit", "ended_on": null, "started_on": 1773066670.6197968, "submitted_on": null, "status": 3, "chroot": "rhel-9-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "10203999-rhel-9-x86_64", "build_id": 10203999, "package_name": "foreman", "package_version": "3.19.0-0.20260309143012356626.develop.9010.g697368892", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@theforeman/develop/foreman", "git_hash": "c8dd63fb8936b13372c09059c6661fb0f5465c82", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@theforeman/develop--packit", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@theforeman/develop/rhel-9-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://yum.theforeman.org/releases/nightly/el9/x86_64/", "id": "https_yum_theforeman_org_releases_nightly_el9_x86_64", "name": "Additional repo https_yum_theforeman_org_releases_nightly_el9_x86_64" } ], "background": false, "fedora_review": false, "isolation": "default", "modules": { "toggle": [ { "enable": "foreman-devel:el9" } ] }, "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@theforeman/develop", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@theforeman/develop", "result_dir": "10203999-foreman", "built_packages": "", "id": 10203999, "mockchain_macros": { "copr_username": "@theforeman", "copr_projectname": "develop", "vendor": "Fedora Project COPR (@theforeman/develop)" } } ] } [2026-03-09 14:31:10,661][ INFO][PID:3471933] Sending fedora-messaging bus message in build.start [2026-03-09 14:31:11,231][ INFO][PID:3471933] Sending fedora-messaging bus message in chroot.start [2026-03-09 14:31:11,261][ INFO][PID:3471933] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10203999-rhel-9-x86_64 --chroot rhel-9-x86_64 --detached [2026-03-09 14:31:11,586][ INFO][PID:3471933] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '3.91.175.29' (ED25519) to the list of known hosts. [2026-03-09 14:31:11,587][ INFO][PID:3471933] Downloading the builder-live.log file, attempt 1 [2026-03-09 14:31:11,589][ INFO][PID:3471933] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.91.175.29 copr-rpmbuild-log [2026-03-09 14:31:16,596][ INFO][PID:3471933] Periodic builder liveness probe: alive [2026-03-09 14:31:16,596][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:21,597][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:26,598][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:31,599][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:36,601][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:41,602][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:46,603][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:51,604][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:31:56,605][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:01,606][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:06,607][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:11,608][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:16,609][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:21,610][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:26,611][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:31,612][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:36,614][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:41,616][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:46,617][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:51,618][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:32:56,618][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:01,619][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:06,620][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:11,621][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:16,623][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:21,624][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:26,648][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:31,649][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:36,650][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:41,651][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:46,653][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:51,654][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:33:56,655][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:01,656][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:06,657][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:11,658][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:16,659][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:21,660][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:25,639][ ERROR][PID:3471933] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-03-09 14:34:26,661][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:31,964][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:35,640][WARNING][PID:3471933] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-03-09 14:34:36,965][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:41,966][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:45,641][ INFO][PID:3471933] Downloading the builder-live.log file, attempt 2 [2026-03-09 14:34:45,643][ INFO][PID:3471933] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.91.175.29 copr-rpmbuild-log [2026-03-09 14:34:46,967][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:51,968][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:34:56,970][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:01,971][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:06,972][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:11,973][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:16,975][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:21,976][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:26,977][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:31,978][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:36,980][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:41,983][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:45,715][ ERROR][PID:3471933] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-03-09 14:35:47,075][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:52,077][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:35:55,721][WARNING][PID:3471933] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-03-09 14:35:57,079][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:02,081][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:05,723][ INFO][PID:3471933] Downloading the builder-live.log file, attempt 3 [2026-03-09 14:36:05,724][ INFO][PID:3471933] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.91.175.29 copr-rpmbuild-log [2026-03-09 14:36:07,090][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:12,091][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:17,092][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:22,093][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:27,094][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:32,096][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:37,099][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:42,157][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:47,158][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:52,164][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:36:57,166][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:02,169][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:05,793][ ERROR][PID:3471933] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-03-09 14:37:07,171][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:12,173][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:15,811][WARNING][PID:3471933] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2026-03-09 14:37:17,208][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:22,209][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:25,813][ INFO][PID:3471933] Downloading the builder-live.log file, attempt 4 [2026-03-09 14:37:25,814][ INFO][PID:3471933] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.91.175.29 copr-rpmbuild-log [2026-03-09 14:37:27,210][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:32,212][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:37,214][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:42,215][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:47,216][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:52,241][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:37:57,243][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:02,245][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:07,246][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:12,248][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:17,250][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:22,251][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:25,883][ ERROR][PID:3471933] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2026-03-09 14:38:27,266][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:32,269][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:35,900][ ERROR][PID:3471933] Giving up for unstable SSH, failures: 1773066875.6406682, 1773066955.7212567, 1773067035.8118181, 1773067115.9004238 [2026-03-09 14:38:35,901][ ERROR][PID:3471933] Re-try request for task on 'ResallocHost, ticket_id=1778051, hostname=3.91.175.29, name=aws_x86_64_spot_prod_02942164_20260309_142508, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2026-03-09 14:38:35,902][ INFO][PID:3471933] Releasing VM back to pool [2026-03-09 14:38:35,913][ INFO][PID:3471933] Retry #1 (on other host) [2026-03-09 14:38:35,914][ INFO][PID:3471933] Checking that /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/repodata/repomd.xml exists [2026-03-09 14:38:35,914][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:35,915][ INFO][PID:3471933] VM allocation process starts [2026-03-09 14:38:35,925][ INFO][PID:3471933] Trying to allocate VM: ResallocHost, ticket_id=1778177, requested_tags=['copr_builder', 'arch_x86_64'] [2026-03-09 14:38:40,926][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:45,927][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:47,956][ INFO][PID:3471933] Allocated host ResallocHost, ticket_id=1778177, hostname=52.206.210.235, name=aws_x86_64_normal_prod_02942147_20260309_142421, requested_tags=['copr_builder', 'arch_x86_64'] [2026-03-09 14:38:47,957][ INFO][PID:3471933] Allocating ssh connection to builder [2026-03-09 14:38:47,957][ INFO][PID:3471933] Checking that builder machine is OK [2026-03-09 14:38:48,307][ INFO][PID:3471933] Running remote command: copr-builder-ready rhel-9-x86_64 [2026-03-09 14:38:48,520][ INFO][PID:3471933] Checking Red Hat subscription (attempt #1) Red Hat subscription active Builder is ready to be used [2026-03-09 14:38:48,521][ INFO][PID:3471933] Filling build.info file with builder info [2026-03-09 14:38:48,538][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:48,540][ INFO][PID:3471933] Not re-notifying FE and msg buses for the new host. [2026-03-09 14:38:48,540][ INFO][PID:3471933] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10203999-rhel-9-x86_64 --chroot rhel-9-x86_64 --detached [2026-03-09 14:38:48,877][ INFO][PID:3471933] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '52.206.210.235' (ED25519) to the list of known hosts. [2026-03-09 14:38:48,878][ INFO][PID:3471933] Downloading the builder-live.log file, attempt 1 [2026-03-09 14:38:48,879][ INFO][PID:3471933] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.206.210.235 copr-rpmbuild-log [2026-03-09 14:38:53,878][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:38:58,880][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:03,880][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:08,882][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:13,883][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:18,886][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:23,888][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:28,890][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:33,891][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:38,893][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:43,894][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:48,898][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:53,900][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:39:58,905][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:03,906][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:08,907][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:13,908][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:18,909][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:23,910][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:28,912][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:33,914][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:38,915][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:43,916][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:48,917][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:53,918][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:40:58,920][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:03,921][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:08,926][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:13,927][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:18,936][ INFO][PID:3471933] Periodic builder liveness probe: alive [2026-03-09 14:41:18,937][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:23,938][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:28,939][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:33,940][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:38,941][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:43,942][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:48,943][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:53,944][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:41:58,945][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:03,946][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:08,947][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:13,948][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:18,950][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:23,966][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:28,967][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:33,990][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:38,991][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:43,992][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:48,993][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:53,994][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:42:58,997][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:04,000][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:09,001][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:14,005][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:19,009][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:24,014][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:29,015][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:34,017][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:39,018][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:44,019][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:49,021][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:54,022][ INFO][PID:3471933] Checking for cancel request [2026-03-09 14:43:54,539][ INFO][PID:3471933] Downloading results from builder [2026-03-09 14:43:54,540][ INFO][PID:3471933] rsyncing of mockbuilder@52.206.210.235:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman started [2026-03-09 14:43:54,541][ INFO][PID:3471933] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@52.206.210.235:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/ &> /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/build-10203999.rsync.log [2026-03-09 14:43:55,580][ INFO][PID:3471933] rsyncing finished. [2026-03-09 14:43:55,580][ INFO][PID:3471933] Releasing VM back to pool [2026-03-09 14:43:55,595][ INFO][PID:3471933] Searching for 'success' file in resultdir [2026-03-09 14:43:55,596][ INFO][PID:3471933] Going to sign pkgs from source: 10203999-rhel-9-x86_64 in chroot: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64 [2026-03-09 14:43:55,596][ INFO][PID:3471933] Calling '/bin/sign -u @theforeman#develop@copr.fedorahosted.org -p' (attempt #1) [2026-03-09 14:43:55,784][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:43:57,727][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:43:57,727][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-build-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:43:58,427][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-build-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:43:58,428][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-service-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:43:59,315][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-service-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:43:59,315][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.src.rpm' (attempt #1) [2026-03-09 14:44:00,356][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.src.rpm [2026-03-09 14:44:00,357][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-dynflow-sidekiq-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:01,116][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-dynflow-sidekiq-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:01,117][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-debug-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:01,900][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-debug-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:01,901][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-journald-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:02,581][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-journald-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:02,583][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-postgresql-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:03,250][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-postgresql-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:03,252][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-telemetry-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:03,952][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-telemetry-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:03,954][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-pcp-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:04,674][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-pcp-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:04,675][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-cli-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:05,372][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-cli-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:05,373][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-ec2-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:06,254][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-ec2-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:06,255][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-vmware-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:06,895][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-vmware-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:06,895][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-redis-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:07,597][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-redis-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:07,598][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-console-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:08,379][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-console-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:08,380][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-assets-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:09,047][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-assets-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:09,048][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-openstack-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:09,729][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-openstack-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:09,733][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-plugin-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:10,434][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-plugin-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:10,436][ INFO][PID:3471933] Calling '/bin/sign -4 -h sha256 -u @theforeman#develop@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-libvirt-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm' (attempt #1) [2026-03-09 14:44:11,128][ INFO][PID:3471933] signed rpm: /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/foreman-libvirt-3.19.0-0.20260309143012356626.develop.9010.g697368892.el9.noarch.rpm [2026-03-09 14:44:11,130][ INFO][PID:3471933] Sign done [2026-03-09 14:44:11,131][ INFO][PID:3471933] Incremental createrepo run, adding 10203999-foreman into https://download.copr.fedorainfracloud.org/results/@theforeman/develop/rhel-9-x86_64, (auto-create-repo=True) [2026-03-09 14:44:11,134][ INFO][PID:3471933] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64 --add 10203999-foreman --no-appstream-metadata' as PID 3556432 [2026-03-09 14:44:25,674][ INFO][PID:3471933] Finished after 14 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64 --add 10203999-foreman --no-appstream-metadata) [2026-03-09 14:44:25,715][ INFO][PID:3471933] Getting build details [2026-03-09 14:44:25,719][ INFO][PID:3471933] Listing built binary packages in /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman [2026-03-09 14:44:25,720][ INFO][PID:3471933] Built packages: ['foreman-console 3.19.0', 'foreman-plugin 3.19.0', 'foreman-redis 3.19.0', 'foreman-assets 3.19.0', 'foreman-pcp 3.19.0', 'foreman-dynflow-sidekiq 3.19.0', 'foreman-telemetry 3.19.0', 'foreman-service 3.19.0', 'foreman-vmware 3.19.0', 'foreman-postgresql 3.19.0', 'foreman-cli 3.19.0', 'foreman-journald 3.19.0', 'foreman-ec2 3.19.0', 'foreman-build 3.19.0', 'foreman-debug 3.19.0', 'foreman 3.19.0', 'foreman-openstack 3.19.0', 'foreman-libvirt 3.19.0'] [2026-03-09 14:44:25,721][ INFO][PID:3471933] build details: {'built_packages': 'foreman-console 3.19.0\nforeman-plugin 3.19.0\nforeman-redis 3.19.0\nforeman-assets 3.19.0\nforeman-pcp 3.19.0\nforeman-dynflow-sidekiq 3.19.0\nforeman-telemetry 3.19.0\nforeman-service 3.19.0\nforeman-vmware 3.19.0\nforeman-postgresql 3.19.0\nforeman-cli 3.19.0\nforeman-journald 3.19.0\nforeman-ec2 3.19.0\nforeman-build 3.19.0\nforeman-debug 3.19.0\nforeman 3.19.0\nforeman-openstack 3.19.0\nforeman-libvirt 3.19.0'} [2026-03-09 14:44:25,722][ INFO][PID:3471933] Retrieving pubkey [2026-03-09 14:44:25,723][ INFO][PID:3471933] Calling '/bin/sign -u @theforeman#develop@copr.fedorahosted.org -p' (attempt #1) [2026-03-09 14:44:26,084][ INFO][PID:3471933] Added pubkey for user @theforeman project develop into: /var/lib/copr/public_html/results/@theforeman/develop/pubkey.gpg [2026-03-09 14:44:26,089][ INFO][PID:3471933] Finished build: id=10203999 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@theforeman/develop chroot=rhel-9-x86_64 [2026-03-09 14:44:26,094][ INFO][PID:3471933] Worker succeeded build, took 337.55422019958496 [2026-03-09 14:44:26,096][ INFO][PID:3471933] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@theforeman", "project_name": "develop", "project_dirname": "develop", "submitter": "packit", "ended_on": 1773067466.0942106, "started_on": 1773067128.5399904, "submitted_on": null, "status": 1, "chroot": "rhel-9-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "10203999-rhel-9-x86_64", "build_id": 10203999, "package_name": "foreman", "package_version": "3.19.0-0.20260309143012356626.develop.9010.g697368892", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@theforeman/develop/foreman", "git_hash": "c8dd63fb8936b13372c09059c6661fb0f5465c82", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@theforeman/develop--packit", "results": { "packages": [ { "name": "foreman-console", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-plugin", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-redis", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-assets", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-pcp", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-dynflow-sidekiq", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "src" }, { "name": "foreman-telemetry", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-service", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-vmware", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-postgresql", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-cli", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-journald", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-ec2", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-build", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-debug", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-openstack", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" }, { "name": "foreman-libvirt", "epoch": null, "version": "3.19.0", "release": "0.20260309143012356626.develop.9010.g697368892.el9", "arch": "noarch" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@theforeman/develop/rhel-9-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "https://yum.theforeman.org/releases/nightly/el9/x86_64/", "id": "https_yum_theforeman_org_releases_nightly_el9_x86_64", "name": "Additional repo https_yum_theforeman_org_releases_nightly_el9_x86_64" } ], "background": false, "fedora_review": false, "isolation": "default", "modules": { "toggle": [ { "enable": "foreman-devel:el9" } ] }, "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@theforeman/develop", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@theforeman/develop", "result_dir": "10203999-foreman", "built_packages": "foreman-console 3.19.0\nforeman-plugin 3.19.0\nforeman-redis 3.19.0\nforeman-assets 3.19.0\nforeman-pcp 3.19.0\nforeman-dynflow-sidekiq 3.19.0\nforeman-telemetry 3.19.0\nforeman-service 3.19.0\nforeman-vmware 3.19.0\nforeman-postgresql 3.19.0\nforeman-cli 3.19.0\nforeman-journald 3.19.0\nforeman-ec2 3.19.0\nforeman-build 3.19.0\nforeman-debug 3.19.0\nforeman 3.19.0\nforeman-openstack 3.19.0\nforeman-libvirt 3.19.0", "id": 10203999, "mockchain_macros": { "copr_username": "@theforeman", "copr_projectname": "develop", "vendor": "Fedora Project COPR (@theforeman/develop)" } } ] } [2026-03-09 14:44:26,157][ INFO][PID:3471933] Sending fedora-messaging bus message in build.end [2026-03-09 14:44:26,193][ INFO][PID:3471933] Compressing /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/builder-live.log by gzip [2026-03-09 14:44:26,196][ INFO][PID:3471933] Running command 'gzip /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/builder-live.log' as PID 3558939 [2026-03-09 14:44:26,326][ INFO][PID:3471933] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/builder-live.log) [2026-03-09 14:44:26,328][ INFO][PID:3471933] Compressing /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/backend.log by gzip [2026-03-09 14:44:26,331][ INFO][PID:3471933] Running command 'gzip /var/lib/copr/public_html/results/@theforeman/develop/rhel-9-x86_64/10203999-foreman/backend.log' as PID 3558957