[2026-04-20 13:17:45,796][  INFO][PID:1961668] Marking build as starting
[2026-04-20 13:17:45,968][  INFO][PID:1961668] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/repodata/repomd.xml exists
[2026-04-20 13:17:46,007][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:17:46,008][  INFO][PID:1961668] VM allocation process starts
[2026-04-20 13:17:46,101][  INFO][PID:1961668] Trying to allocate VM: ResallocHost, ticket_id=2421882, requested_tags=['arch_x86_64', 'copr_builder']
[2026-04-20 13:17:51,107][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:17:52,212][  INFO][PID:1961668] Allocated host ResallocHost, ticket_id=2421882, hostname=2620:52:6:1161:dead:beef:cafe:c114, name=vmhost_x86_02_prod_07130868_20260420_131457, requested_tags=['arch_x86_64', 'copr_builder']
[2026-04-20 13:17:52,213][  INFO][PID:1961668] Allocating ssh connection to builder
[2026-04-20 13:17:52,214][  INFO][PID:1961668] Checking that builder machine is OK
[2026-04-20 13:17:52,813][  INFO][PID:1961668] Installed copr-rpmbuild version: 1.8
[2026-04-20 13:17:52,814][  INFO][PID:1961668] Running remote command: copr-builder-ready fedora-rawhide-x86_64
[2026-04-20 13:17:53,215][  INFO][PID:1961668] Red Hat subscription not needed for fedora-rawhide-x86_64
Builder is ready to be used

[2026-04-20 13:17:53,216][  INFO][PID:1961668] Filling build.info file with builder info
[2026-04-20 13:17:53,216][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:17:53,217][  INFO][PID:1961668] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@osbuild",
            "project_name": "osbuild-composer",
            "project_dirname": "osbuild-composer",
            "submitter": "packit",
            "ended_on": null,
            "started_on": 1776691073.2173877,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "10348571-fedora-rawhide-x86_64",
            "build_id": 10348571,
            "package_name": "osbuild-composer",
            "package_version": "169-1.20260420131710720793.main.5.g2414f89f9",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@osbuild/osbuild-composer/osbuild-composer",
            "git_hash": "87eda2b1e79ec817d3018c004f7dc0fbf7111936",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@osbuild/osbuild-composer--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@osbuild/osbuild-composer",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer",
            "result_dir": "10348571-osbuild-composer",
            "built_packages": "",
            "id": 10348571,
            "mockchain_macros": {
                "copr_username": "@osbuild",
                "copr_projectname": "osbuild-composer",
                "vendor": "Fedora Project COPR (@osbuild/osbuild-composer)"
            }
        }
    ]
}
[2026-04-20 13:17:53,285][  INFO][PID:1961668] Sending fedora-messaging bus message in build.start
[2026-04-20 13:17:53,910][  INFO][PID:1961668] Sending fedora-messaging bus message in chroot.start
[2026-04-20 13:17:53,940][  INFO][PID:1961668] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10348571-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached
[2026-04-20 13:17:54,696][  INFO][PID:1961668] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c114' (ED25519) to the list of known hosts.


[2026-04-20 13:17:54,697][  INFO][PID:1961668] Downloading the builder-live.log file, attempt 1
[2026-04-20 13:17:54,699][  INFO][PID:1961668] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c114 copr-rpmbuild-log
[2026-04-20 13:17:59,714][  INFO][PID:1961668] Periodic builder liveness probe: alive
[2026-04-20 13:17:59,715][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:04,716][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:09,718][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:14,719][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:19,721][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:24,724][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:29,726][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:34,727][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:39,739][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:44,743][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:49,745][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:54,748][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:18:59,749][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:04,750][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:09,751][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:14,753][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:19,761][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:24,765][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:29,767][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:34,770][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:39,772][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:44,773][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:49,774][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:54,777][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:19:59,780][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:04,782][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:09,789][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:14,794][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:19,796][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:24,799][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:29,802][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:34,810][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:39,821][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:44,893][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:49,894][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:54,896][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:20:59,900][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:04,903][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:09,907][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:14,913][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:19,918][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:24,941][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:29,943][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:34,959][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:39,962][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:44,966][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:49,969][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:54,985][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:21:59,987][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:04,995][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:10,005][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:15,006][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:20,008][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:25,012][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:30,014][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:35,017][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:40,020][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:45,025][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:50,027][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:22:55,046][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:00,049][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:05,050][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:10,051][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:15,053][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:20,055][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:25,058][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:30,088][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:35,089][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:40,090][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:45,094][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:50,095][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:23:55,097][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:00,098][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:05,101][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:10,108][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:15,110][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:20,117][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:25,120][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:30,126][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:35,127][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:40,129][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:45,130][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:50,132][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:24:55,136][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:00,140][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:05,141][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:10,145][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:15,146][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:20,148][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:25,149][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:30,150][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:35,152][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:40,153][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:45,157][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:50,166][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:25:55,168][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:00,169][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:05,174][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:10,176][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:15,180][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:20,182][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:25,183][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:30,184][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:35,185][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:40,188][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:45,190][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:50,191][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:26:55,200][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:00,203][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:05,213][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:10,214][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:15,227][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:20,232][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:25,324][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:30,329][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:35,332][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:40,336][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:45,337][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:50,340][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:27:55,342][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:00,435][  INFO][PID:1961668] Periodic builder liveness probe: alive
[2026-04-20 13:28:00,438][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:05,448][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:10,449][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:15,450][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:20,464][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:25,474][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:30,477][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:35,482][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:40,494][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:45,496][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:50,504][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:28:55,510][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:00,518][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:05,520][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:10,529][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:15,535][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:20,543][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:25,546][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:30,570][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:35,574][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:40,576][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:45,581][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:50,583][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:29:55,592][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:00,595][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:05,597][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:10,601][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:15,603][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:20,608][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:25,610][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:30,617][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:35,629][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:40,636][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:45,639][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:50,646][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:30:55,648][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:00,656][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:05,660][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:10,664][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:15,668][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:20,670][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:25,674][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:30,677][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:35,680][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:40,683][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:45,685][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:50,687][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:31:55,689][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:00,692][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:05,693][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:10,695][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:15,698][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:20,702][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:25,706][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:30,708][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:35,713][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:40,716][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:45,718][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:50,730][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:32:55,733][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:33:00,735][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:33:05,747][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:33:10,757][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:33:11,142][  INFO][PID:1961668] Downloading results from builder
[2026-04-20 13:33:11,144][  INFO][PID:1961668] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c114]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer started
[2026-04-20 13:33:11,149][  INFO][PID:1961668] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c114]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/ &> /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/build-10348571.rsync.log
[2026-04-20 13:33:11,677][  INFO][PID:1961668] rsyncing finished.
[2026-04-20 13:33:11,678][ ERROR][PID:1961668] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:33:11,694][ ERROR][PID:1961668] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:33:21,704][  INFO][PID:1961668] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c114]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer started
[2026-04-20 13:33:21,706][  INFO][PID:1961668] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c114]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/ &> /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/build-10348571.rsync.log
[2026-04-20 13:33:22,035][  INFO][PID:1961668] rsyncing finished.
[2026-04-20 13:33:22,036][ ERROR][PID:1961668] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:33:22,037][ ERROR][PID:1961668] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:33:32,040][  INFO][PID:1961668] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c114]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer started
[2026-04-20 13:33:32,044][  INFO][PID:1961668] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c114]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/ &> /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/build-10348571.rsync.log
[2026-04-20 13:33:32,385][  INFO][PID:1961668] rsyncing finished.
[2026-04-20 13:33:32,390][ ERROR][PID:1961668] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:33:32,392][ ERROR][PID:1961668] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-04-20 13:33:42,396][ ERROR][PID:1961668] Re-try request for task on 'ResallocHost, ticket_id=2421882, hostname=2620:52:6:1161:dead:beef:cafe:c114, name=vmhost_x86_02_prod_07130868_20260420_131457, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 3 SSH attempts
[2026-04-20 13:33:42,396][  INFO][PID:1961668] VM Release request
[2026-04-20 13:33:42,431][  INFO][PID:1961668] Retry #1 (on other host)
[2026-04-20 13:33:42,435][  INFO][PID:1961668] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/repodata/repomd.xml exists
[2026-04-20 13:33:42,503][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:33:42,505][  INFO][PID:1961668] VM allocation process starts
[2026-04-20 13:33:42,557][  INFO][PID:1961668] Trying to allocate VM: ResallocHost, ticket_id=2422514, requested_tags=['arch_x86_64', 'copr_builder']
[2026-04-20 13:33:47,561][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:33:52,562][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:33:57,565][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:02,567][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:07,569][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:12,577][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:17,578][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:22,589][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:27,590][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:32,591][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:37,601][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:42,617][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:47,620][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:52,627][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:34:57,630][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:02,634][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:07,640][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:10,839][  INFO][PID:1961668] Allocated host ResallocHost, ticket_id=2422514, hostname=34.231.244.244, name=aws_x86_64_normalreserved_prod_07132512_20260420_133259, requested_tags=['arch_x86_64', 'copr_builder']
[2026-04-20 13:35:10,840][  INFO][PID:1961668] Allocating ssh connection to builder
[2026-04-20 13:35:10,840][  INFO][PID:1961668] Checking that builder machine is OK
[2026-04-20 13:35:10,984][ ERROR][PID:1961668] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '34.231.244.244' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@34.231.244.244: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:35:20,984][ ERROR][PID:1961668] Re-try request for task on 'ResallocHost, ticket_id=2422514, hostname=34.231.244.244, name=aws_x86_64_normalreserved_prod_07132512_20260420_133259, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:35:20,989][  INFO][PID:1961668] VM Release request
[2026-04-20 13:35:21,059][  INFO][PID:1961668] Retry #2 (on other host)
[2026-04-20 13:35:21,062][  INFO][PID:1961668] Checking that /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/repodata/repomd.xml exists
[2026-04-20 13:35:21,064][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:21,068][  INFO][PID:1961668] VM allocation process starts
[2026-04-20 13:35:21,137][  INFO][PID:1961668] Trying to allocate VM: ResallocHost, ticket_id=2422590, requested_tags=['arch_x86_64', 'copr_builder']
[2026-04-20 13:35:26,138][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:31,143][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:36,146][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:41,162][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:46,166][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:51,171][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:35:56,176][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:01,179][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:06,194][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:11,198][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:16,208][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:21,211][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:26,213][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:31,218][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:36,220][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:41,225][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:46,241][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:51,246][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:36:56,248][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:01,251][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:06,262][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:11,287][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:16,288][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:21,294][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:26,296][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:31,301][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:36,307][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:41,318][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:46,324][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:51,332][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:37:56,334][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:01,342][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:06,346][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:11,352][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:16,362][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:21,397][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:26,401][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:31,411][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:36,415][  INFO][PID:1961668] Checking for cancel request
[2026-04-20 13:38:39,705][  INFO][PID:1961668] Allocated host ResallocHost, ticket_id=2422590, hostname=54.227.82.36, name=aws_x86_64_spot_prod_07132637_20260420_133454, requested_tags=['arch_x86_64', 'copr_builder']
[2026-04-20 13:38:39,706][  INFO][PID:1961668] Allocating ssh connection to builder
[2026-04-20 13:38:39,707][  INFO][PID:1961668] Checking that builder machine is OK
[2026-04-20 13:38:39,870][ ERROR][PID:1961668] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '54.227.82.36' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@54.227.82.36: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:38:49,876][ ERROR][PID:1961668] Re-try request for task on 'ResallocHost, ticket_id=2422590, hostname=54.227.82.36, name=aws_x86_64_spot_prod_07132637_20260420_133454, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:38:49,884][  INFO][PID:1961668] VM Release request
[2026-04-20 13:38:49,928][ ERROR][PID:1961668] Backend process error: Three host tried without success: {'34.231.244.244', '2620:52:6:1161:dead:beef:cafe:c114', '54.227.82.36'}
[2026-04-20 13:38:49,933][WARNING][PID:1961668] Switching not-finished job state to 'failed'
[2026-04-20 13:38:49,941][  INFO][PID:1961668] Worker failed build, took 1256.716358423233
[2026-04-20 13:38:49,942][  INFO][PID:1961668] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@osbuild",
            "project_name": "osbuild-composer",
            "project_dirname": "osbuild-composer",
            "submitter": "packit",
            "ended_on": 1776692329.933746,
            "started_on": 1776691073.2173877,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "10348571-fedora-rawhide-x86_64",
            "build_id": 10348571,
            "package_name": "osbuild-composer",
            "package_version": "169-1.20260420131710720793.main.5.g2414f89f9",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@osbuild/osbuild-composer/osbuild-composer",
            "git_hash": "87eda2b1e79ec817d3018c004f7dc0fbf7111936",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@osbuild/osbuild-composer--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@osbuild/osbuild-composer",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@osbuild/osbuild-composer",
            "result_dir": "10348571-osbuild-composer",
            "built_packages": "",
            "id": 10348571,
            "mockchain_macros": {
                "copr_username": "@osbuild",
                "copr_projectname": "osbuild-composer",
                "vendor": "Fedora Project COPR (@osbuild/osbuild-composer)"
            }
        }
    ]
}
[2026-04-20 13:38:50,122][  INFO][PID:1961668] Sending fedora-messaging bus message in build.end
[2026-04-20 13:38:50,269][  INFO][PID:1961668] Compressing /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/builder-live.log by gzip
[2026-04-20 13:38:50,271][  INFO][PID:1961668] Running command 'gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/builder-live.log' as PID 2243834
[2026-04-20 13:38:50,327][  INFO][PID:1961668] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/builder-live.log)
[2026-04-20 13:38:50,334][  INFO][PID:1961668] Compressing /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/backend.log by gzip
[2026-04-20 13:38:50,336][  INFO][PID:1961668] Running command 'gzip /var/lib/copr/public_html/results/@osbuild/osbuild-composer/fedora-rawhide-x86_64/10348571-osbuild-composer/backend.log' as PID 2243850