[2026-04-07 21:07:55,879][  INFO][PID:3297108] Marking build as starting
[2026-04-07 21:07:56,025][  INFO][PID:3297108] Checking that https://download.copr.fedorainfracloud.org/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64//repodata/repomd.xml exists
[2026-04-07 21:07:56,563][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:07:56,567][  INFO][PID:3297108] VM allocation process starts
[2026-04-07 21:07:56,619][  INFO][PID:3297108] Trying to allocate VM: ResallocHost, ticket_id=2212009, requested_tags=['copr_builder', 'arch_x86_64']
[2026-04-07 21:07:59,665][  INFO][PID:3297108] Allocated host ResallocHost, ticket_id=2212009, hostname=54.159.221.217, name=aws_x86_64_spot_prod_05913469_20260407_210234, requested_tags=['copr_builder', 'arch_x86_64']
[2026-04-07 21:07:59,667][  INFO][PID:3297108] Allocating ssh connection to builder
[2026-04-07 21:07:59,668][  INFO][PID:3297108] Checking that builder machine is OK
[2026-04-07 21:08:00,015][  INFO][PID:3297108] Running remote command: copr-builder-ready rhel-9-x86_64
[2026-04-07 21:08:00,241][  INFO][PID:3297108] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2026-04-07 21:08:00,241][  INFO][PID:3297108] Filling build.info file with builder info
[2026-04-07 21:08:00,242][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:00,247][  INFO][PID:3297108] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 36000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@dotnet-sig",
            "project_name": "dotnet-preview",
            "project_dirname": "dotnet-preview",
            "submitter": "omajid",
            "ended_on": null,
            "started_on": 1775596080.2476456,
            "submitted_on": null,
            "status": 3,
            "chroot": "rhel-9-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "10298563-rhel-9-x86_64",
            "build_id": 10298563,
            "package_name": "dotnet11.0",
            "package_version": "11.0.100~preview.2.final-0.1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@dotnet-sig/dotnet-preview/dotnet11.0",
            "git_hash": "161c2aa0cc262b03d4e095c79d43af9a1a87d2b0",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@dotnet-sig/dotnet-preview--omajid",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 1,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@dotnet-sig/dotnet-preview",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@dotnet-sig/dotnet-preview",
            "result_dir": "10298563-dotnet11.0",
            "built_packages": "",
            "id": 10298563,
            "mockchain_macros": {
                "copr_username": "@dotnet-sig",
                "copr_projectname": "dotnet-preview",
                "vendor": "Fedora Project COPR (@dotnet-sig/dotnet-preview)"
            }
        }
    ]
}
[2026-04-07 21:08:00,373][  INFO][PID:3297108] Sending fedora-messaging bus message in build.start
[2026-04-07 21:08:01,133][  INFO][PID:3297108] Sending fedora-messaging bus message in chroot.start
[2026-04-07 21:08:01,164][  INFO][PID:3297108] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10298563-rhel-9-x86_64 --chroot rhel-9-x86_64 --detached
[2026-04-07 21:08:01,512][  INFO][PID:3297108] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '54.159.221.217' (ED25519) to the list of known hosts.


[2026-04-07 21:08:01,513][  INFO][PID:3297108] Downloading the builder-live.log file, attempt 1
[2026-04-07 21:08:01,515][  INFO][PID:3297108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.159.221.217 copr-rpmbuild-log
[2026-04-07 21:08:06,531][  INFO][PID:3297108] Periodic builder liveness probe: alive
[2026-04-07 21:08:06,532][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:11,543][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:16,552][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:21,554][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:26,555][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:31,556][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:36,563][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:41,571][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:46,573][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:51,574][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:08:56,578][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:01,581][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:06,584][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:11,587][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:16,590][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:21,592][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:26,597][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:31,599][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:36,600][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:41,601][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:46,602][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:51,603][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:09:56,605][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:01,607][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:06,608][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:11,609][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:16,614][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:21,615][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:26,616][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:31,618][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:36,620][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:41,624][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:46,625][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:51,631][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:10:56,634][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:01,636][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:06,638][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:11,641][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:16,642][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:21,644][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:26,646][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:31,647][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:36,672][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:41,673][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:46,674][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:51,675][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:11:56,676][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:01,677][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:06,679][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:11,680][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:16,681][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:21,682][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:26,683][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:31,684][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:36,685][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:41,686][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:46,701][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:51,703][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:12:56,705][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:01,706][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:06,709][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:11,711][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:16,713][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:21,723][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:26,724][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:31,725][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:36,726][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:41,727][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:46,728][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:51,729][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:13:56,730][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:01,731][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:06,732][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:11,732][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:16,733][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:21,734][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:26,735][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:31,736][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:36,737][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:41,738][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:46,739][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:51,740][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:14:56,741][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:01,742][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:06,743][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:11,744][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:16,746][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:21,747][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:26,748][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:31,749][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:36,749][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:41,750][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:46,751][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:51,752][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:15:56,753][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:01,754][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:06,755][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:11,756][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:16,757][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:21,758][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:26,759][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:31,760][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:36,761][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:41,762][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:46,763][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:51,764][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:16:56,779][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:01,780][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:06,782][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:11,786][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:16,787][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:21,788][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:26,789][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:31,790][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:36,791][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:41,792][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:46,793][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:51,794][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:17:56,795][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:01,796][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:06,803][  INFO][PID:3297108] Periodic builder liveness probe: alive
[2026-04-07 21:18:06,804][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:11,805][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:16,806][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:21,808][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:26,813][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:31,816][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:36,817][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:41,818][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:46,819][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:51,820][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:18:56,821][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:01,822][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:06,823][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:11,824][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:16,825][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:21,854][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:26,856][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:31,857][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:36,858][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:41,859][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:46,860][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:51,861][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:19:56,862][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:01,863][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:06,864][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:11,865][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:16,866][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:21,869][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:25,413][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-04-07 21:20:26,870][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:31,871][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:35,413][WARNING][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-04-07 21:20:36,872][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:41,873][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:45,414][  INFO][PID:3297108] Downloading the builder-live.log file, attempt 2
[2026-04-07 21:20:45,416][  INFO][PID:3297108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.159.221.217 copr-rpmbuild-log
[2026-04-07 21:20:46,875][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:51,876][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:20:56,910][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:01,931][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:06,933][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:11,933][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:17,004][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:22,005][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:27,007][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:32,008][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:37,009][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:42,010][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:45,481][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-04-07 21:21:47,356][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:52,360][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:21:55,492][WARNING][PID:3297108] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-04-07 21:21:57,369][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:02,371][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:05,493][  INFO][PID:3297108] Downloading the builder-live.log file, attempt 3
[2026-04-07 21:22:05,494][  INFO][PID:3297108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.159.221.217 copr-rpmbuild-log
[2026-04-07 21:22:07,373][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:12,374][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:17,377][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:22,378][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:27,379][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:32,380][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:37,457][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:42,501][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:47,556][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:52,562][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:22:57,637][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:02,647][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:05,548][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-04-07 21:23:07,760][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:12,843][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:15,580][WARNING][PID:3297108] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-04-07 21:23:17,910][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:22,936][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:25,580][  INFO][PID:3297108] Downloading the builder-live.log file, attempt 4
[2026-04-07 21:23:25,582][  INFO][PID:3297108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.159.221.217 copr-rpmbuild-log
[2026-04-07 21:23:27,938][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:32,939][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:37,989][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:43,163][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:48,260][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:53,507][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:23:58,703][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:03,874][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:09,315][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:14,466][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:19,740][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:24,909][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:25,648][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-04-07 21:24:29,910][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:34,911][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:35,652][ ERROR][PID:3297108] Giving up for unstable SSH, failures: 1775596835.4137077, 1775596915.4921436, 1775596995.5800524, 1775597075.651947
[2026-04-07 21:24:35,653][ ERROR][PID:3297108] Re-try request for task on 'ResallocHost, ticket_id=2212009, hostname=54.159.221.217, name=aws_x86_64_spot_prod_05913469_20260407_210234, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-04-07 21:24:35,653][  INFO][PID:3297108] Releasing VM back to pool
[2026-04-07 21:24:35,705][  INFO][PID:3297108] Retry #1 (on other host)
[2026-04-07 21:24:35,705][  INFO][PID:3297108] Checking that https://download.copr.fedorainfracloud.org/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64//repodata/repomd.xml exists
[2026-04-07 21:24:35,888][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:24:35,889][  INFO][PID:3297108] VM allocation process starts
[2026-04-07 21:24:35,898][  INFO][PID:3297108] Trying to allocate VM: ResallocHost, ticket_id=2212218, requested_tags=['copr_builder', 'arch_x86_64']
[2026-04-07 21:24:38,910][  INFO][PID:3297108] Allocated host ResallocHost, ticket_id=2212218, hostname=44.213.60.130, name=aws_x86_64_spot_prod_05913690_20260407_211632, requested_tags=['copr_builder', 'arch_x86_64']
[2026-04-07 21:24:38,910][  INFO][PID:3297108] Allocating ssh connection to builder
[2026-04-07 21:24:38,911][  INFO][PID:3297108] Checking that builder machine is OK
[2026-04-07 21:25:38,980][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 44.213.60.130 port 22: Connection timed out

[2026-04-07 21:25:48,988][ ERROR][PID:3297108] Re-try request for task on 'ResallocHost, ticket_id=2212218, hostname=44.213.60.130, name=aws_x86_64_spot_prod_05913690_20260407_211632, requested_tags=['copr_builder', 'arch_x86_64']': Unable to finish after 1 SSH attempts
[2026-04-07 21:25:48,988][  INFO][PID:3297108] Releasing VM back to pool
[2026-04-07 21:25:49,002][  INFO][PID:3297108] Retry #2 (on other host)
[2026-04-07 21:25:49,003][  INFO][PID:3297108] Checking that https://download.copr.fedorainfracloud.org/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64//repodata/repomd.xml exists
[2026-04-07 21:25:49,213][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:25:49,214][  INFO][PID:3297108] VM allocation process starts
[2026-04-07 21:25:49,245][  INFO][PID:3297108] Trying to allocate VM: ResallocHost, ticket_id=2212229, requested_tags=['copr_builder', 'arch_x86_64']
[2026-04-07 21:25:52,264][  INFO][PID:3297108] Allocated host ResallocHost, ticket_id=2212229, hostname=100.55.5.184, name=aws_x86_64_spot_prod_05913710_20260407_211817, requested_tags=['copr_builder', 'arch_x86_64']
[2026-04-07 21:25:52,264][  INFO][PID:3297108] Allocating ssh connection to builder
[2026-04-07 21:25:52,265][  INFO][PID:3297108] Checking that builder machine is OK
[2026-04-07 21:25:52,576][  INFO][PID:3297108] Running remote command: copr-builder-ready rhel-9-x86_64
[2026-04-07 21:25:52,770][  INFO][PID:3297108] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2026-04-07 21:25:52,771][  INFO][PID:3297108] Filling build.info file with builder info
[2026-04-07 21:25:53,414][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:25:53,415][  INFO][PID:3297108] Not re-notifying FE and msg buses for the new host.
[2026-04-07 21:25:53,416][  INFO][PID:3297108] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10298563-rhel-9-x86_64 --chroot rhel-9-x86_64 --detached
[2026-04-07 21:25:53,735][  INFO][PID:3297108] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '100.55.5.184' (ED25519) to the list of known hosts.


[2026-04-07 21:25:53,736][  INFO][PID:3297108] Downloading the builder-live.log file, attempt 1
[2026-04-07 21:25:53,737][  INFO][PID:3297108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.55.5.184 copr-rpmbuild-log
[2026-04-07 21:25:58,736][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:03,737][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:08,738][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:13,739][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:18,740][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:23,741][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:28,742][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:33,743][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:38,744][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:44,208][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:49,415][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:26:54,778][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:00,044][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:05,213][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:10,331][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:15,441][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:20,443][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:26,258][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:31,260][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:36,290][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:41,291][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:46,292][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:51,312][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:27:56,329][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:01,330][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:06,355][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:11,364][  INFO][PID:3297108] Periodic builder liveness probe: alive
[2026-04-07 21:28:11,365][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:16,365][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:21,366][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:26,367][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:31,368][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:36,369][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:41,374][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:46,375][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:51,376][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:28:56,377][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:01,378][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:06,380][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:11,381][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:16,404][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:21,405][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:26,406][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:31,407][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:36,408][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:41,409][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:46,411][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:51,412][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:29:56,413][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:01,413][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:06,415][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:11,416][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:16,418][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:21,418][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:26,419][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:31,422][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:36,425][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:41,426][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:46,427][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:51,428][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:30:56,429][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:01,430][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:06,431][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:11,431][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:16,433][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:21,434][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:26,435][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:31,436][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:36,437][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:41,438][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:46,459][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:51,461][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:31:56,463][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:01,465][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:06,466][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:11,468][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:16,511][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:21,512][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:26,513][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:31,513][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:36,514][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:41,515][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:46,588][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:51,589][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:32:56,590][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:01,590][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:06,792][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:11,793][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:16,794][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:21,796][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:26,870][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:31,871][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:36,872][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:41,905][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:46,906][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:51,906][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:33:56,908][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:01,909][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:06,910][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:11,911][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:16,912][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:21,913][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:26,914][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:31,915][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:36,916][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:41,917][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:46,918][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:51,919][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:34:56,920][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:01,556][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-04-07 21:35:01,921][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:06,922][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:11,675][WARNING][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-04-07 21:35:11,923][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:16,924][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:21,739][  INFO][PID:3297108] Downloading the builder-live.log file, attempt 2
[2026-04-07 21:35:21,740][  INFO][PID:3297108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.55.5.184 copr-rpmbuild-log
[2026-04-07 21:35:22,131][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:22,302][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-04-07 21:35:27,134][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:32,142][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:32,321][WARNING][PID:3297108] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-04-07 21:35:37,144][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:42,145][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:42,321][  INFO][PID:3297108] Downloading the builder-live.log file, attempt 3
[2026-04-07 21:35:42,323][  INFO][PID:3297108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.55.5.184 copr-rpmbuild-log
[2026-04-07 21:35:42,331][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-04-07 21:35:47,147][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:52,347][WARNING][PID:3297108] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-04-07 21:35:52,478][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:35:57,480][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:02,397][  INFO][PID:3297108] Downloading the builder-live.log file, attempt 4
[2026-04-07 21:36:02,482][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:02,546][  INFO][PID:3297108] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@100.55.5.184 copr-rpmbuild-log
[2026-04-07 21:36:07,483][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:12,583][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:17,584][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:22,585][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:27,586][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:32,587][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:37,612][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:42,613][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:47,614][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:52,615][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:36:57,615][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:37:02,612][ ERROR][PID:3297108] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-04-07 21:37:02,616][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:37:07,617][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:37:12,618][  INFO][PID:3297108] Checking for cancel request
[2026-04-07 21:37:12,620][ ERROR][PID:3297108] Giving up for unstable SSH, failures: 1775597711.675279, 1775597732.321148, 1775597752.3471925, 1775597832.619856
[2026-04-07 21:37:12,620][ ERROR][PID:3297108] Re-try request for task on 'ResallocHost, ticket_id=2212229, hostname=100.55.5.184, name=aws_x86_64_spot_prod_05913710_20260407_211817, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-04-07 21:37:12,621][  INFO][PID:3297108] Releasing VM back to pool
[2026-04-07 21:37:12,632][ ERROR][PID:3297108] Backend process error: Three host tried without success: {'100.55.5.184', '54.159.221.217', '44.213.60.130'}
[2026-04-07 21:37:12,633][WARNING][PID:3297108] Switching not-finished job state to 'failed'
[2026-04-07 21:37:12,633][  INFO][PID:3297108] Worker failed build, took 679.2172224521637
[2026-04-07 21:37:12,634][  INFO][PID:3297108] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 36000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@dotnet-sig",
            "project_name": "dotnet-preview",
            "project_dirname": "dotnet-preview",
            "submitter": "omajid",
            "ended_on": 1775597832.633031,
            "started_on": 1775597153.4158084,
            "submitted_on": null,
            "status": 0,
            "chroot": "rhel-9-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "10298563-rhel-9-x86_64",
            "build_id": 10298563,
            "package_name": "dotnet11.0",
            "package_version": "11.0.100~preview.2.final-0.1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@dotnet-sig/dotnet-preview/dotnet11.0",
            "git_hash": "161c2aa0cc262b03d4e095c79d43af9a1a87d2b0",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@dotnet-sig/dotnet-preview--omajid",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 1,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@dotnet-sig/dotnet-preview",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@dotnet-sig/dotnet-preview",
            "result_dir": "10298563-dotnet11.0",
            "built_packages": "",
            "id": 10298563,
            "mockchain_macros": {
                "copr_username": "@dotnet-sig",
                "copr_projectname": "dotnet-preview",
                "vendor": "Fedora Project COPR (@dotnet-sig/dotnet-preview)"
            }
        }
    ]
}
[2026-04-07 21:37:12,706][  INFO][PID:3297108] Sending fedora-messaging bus message in build.end
[2026-04-07 21:37:12,739][  INFO][PID:3297108] Compressing /var/lib/copr/public_html/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64/10298563-dotnet11.0/builder-live.log by gzip
[2026-04-07 21:37:12,740][  INFO][PID:3297108] Running command 'gzip /var/lib/copr/public_html/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64/10298563-dotnet11.0/builder-live.log' as PID 3432924
[2026-04-07 21:37:12,774][  INFO][PID:3297108] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64/10298563-dotnet11.0/builder-live.log)
[2026-04-07 21:37:12,775][  INFO][PID:3297108] Compressing /var/lib/copr/public_html/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64/10298563-dotnet11.0/backend.log by gzip
[2026-04-07 21:37:12,777][  INFO][PID:3297108] Running command 'gzip /var/lib/copr/public_html/results/@dotnet-sig/dotnet-preview/rhel-9-x86_64/10298563-dotnet11.0/backend.log' as PID 3432925