[2026-08-18 16:03:00,223][  INFO][PID:1068902] Marking build as starting
[2026-08-18 16:03:00,785][  INFO][PID:1068902] Checking that /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/repodata/repomd.xml exists
[2026-08-18 16:03:00,789][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:00,878][  INFO][PID:1068902] rpmeta: package vulkan-lunarg-tools not known to model, skipping
[2026-08-18 16:03:00,888][  INFO][PID:1068902] VM allocation process starts
[2026-08-18 16:03:00,923][  INFO][PID:1068902] Trying to allocate VM: ResallocHost, ticket_id=4501745, requested_tags=['arch_x86_64', 'copr_builder']
[2026-08-18 16:03:05,937][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:10,946][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:15,948][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:20,953][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:25,958][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:30,971][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:35,976][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:40,978][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:46,006][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:51,014][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:03:56,029][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:01,037][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:06,044][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:11,049][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:16,052][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:21,062][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:26,069][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:31,077][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:36,087][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:41,101][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:46,108][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:51,111][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:04:56,123][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:01,129][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:06,131][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:11,139][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:16,147][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:19,426][  INFO][PID:1068902] Allocated host ResallocHost, ticket_id=4501745, hostname=3.81.28.139, name=aws_x86_64_spot_prod_19341337_20260818_160240, requested_tags=['arch_x86_64', 'copr_builder']
[2026-08-18 16:05:19,431][  INFO][PID:1068902] Allocating ssh connection to builder
[2026-08-18 16:05:19,435][  INFO][PID:1068902] Checking that builder machine is OK
[2026-08-18 16:05:19,870][  INFO][PID:1068902] Installed copr-rpmbuild version: 1.9
[2026-08-18 16:05:19,880][  INFO][PID:1068902] Running remote command: copr-builder-ready fedora-45-x86_64
[2026-08-18 16:05:20,303][  INFO][PID:1068902] Red Hat subscription not needed for fedora-45-x86_64
Builder is ready to be used

[2026-08-18 16:05:20,307][  INFO][PID:1068902] Filling build.info file with builder info
[2026-08-18 16:05:20,309][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:20,326][  INFO][PID:1068902] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "tellowkrinkle",
            "project_name": "vulkan-tools",
            "project_dirname": "vulkan-tools",
            "submitter": "tellowkrinkle",
            "ended_on": null,
            "started_on": 1787069120.3263006,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-45-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "10876842-fedora-45-x86_64",
            "build_id": 10876842,
            "package_name": "vulkan-lunarg-tools",
            "package_version": "1.4.357.0-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/tellowkrinkle/vulkan-tools/vulkan-lunarg-tools",
            "git_hash": "718b9657fedf2d6070c9503dfd28dafa1987f53e",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "tellowkrinkle/vulkan-tools--tellowkrinkle",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": 105,
                    "repo_gpgcheck": false
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": 105,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/tellowkrinkle/vulkan-tools",
            "result_dir": "10876842-vulkan-lunarg-tools",
            "built_packages": "",
            "rpmeta_prediction": null,
            "id": 10876842,
            "mockchain_macros": {
                "copr_username": "tellowkrinkle",
                "copr_projectname": "vulkan-tools",
                "vendor": "Fedora Project COPR (tellowkrinkle/vulkan-tools)"
            }
        }
    ]
}
[2026-08-18 16:05:20,467][  INFO][PID:1068902] Sending fedora-messaging bus message in build.start
[2026-08-18 16:05:23,300][  INFO][PID:1068902] Sending fedora-messaging bus message in chroot.start
[2026-08-18 16:05:23,441][  INFO][PID:1068902] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10876842-fedora-45-x86_64 --chroot fedora-45-x86_64 --detached
[2026-08-18 16:05:24,165][  INFO][PID:1068902] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '3.81.28.139' (ED25519) to the list of known hosts.


[2026-08-18 16:05:24,170][  INFO][PID:1068902] Downloading the builder-live.log file, attempt 1
[2026-08-18 16:05:24,187][  INFO][PID:1068902] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.81.28.139 copr-rpmbuild-log
[2026-08-18 16:05:29,197][  INFO][PID:1068902] Periodic builder liveness probe: alive
[2026-08-18 16:05:29,200][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:34,210][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:39,217][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:44,222][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:49,227][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:54,259][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:05:59,264][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:03,848][ ERROR][PID:1068902] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-08-18 16:06:04,279][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:09,293][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:13,935][WARNING][PID:1068902] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-08-18 16:06:14,309][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:19,322][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:23,958][  INFO][PID:1068902] Downloading the builder-live.log file, attempt 2
[2026-08-18 16:06:23,993][  INFO][PID:1068902] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.81.28.139 copr-rpmbuild-log
[2026-08-18 16:06:24,340][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:29,359][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:34,368][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:39,375][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:44,378][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:49,388][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:54,412][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:06:59,427][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:04,434][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:09,450][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:14,487][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:19,510][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:24,113][ ERROR][PID:1068902] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-08-18 16:07:24,557][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:29,562][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:34,206][WARNING][PID:1068902] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-08-18 16:07:34,571][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:39,595][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:44,213][  INFO][PID:1068902] Downloading the builder-live.log file, attempt 3
[2026-08-18 16:07:44,223][  INFO][PID:1068902] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.81.28.139 copr-rpmbuild-log
[2026-08-18 16:07:44,599][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:49,629][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:54,647][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:07:59,658][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:04,665][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:09,695][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:14,732][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:19,747][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:24,755][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:29,796][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:34,832][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:40,000][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:44,377][ ERROR][PID:1068902] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-08-18 16:08:45,037][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:50,087][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:08:54,667][WARNING][PID:1068902] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-08-18 16:08:55,106][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:00,134][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:04,677][  INFO][PID:1068902] Downloading the builder-live.log file, attempt 4
[2026-08-18 16:09:04,712][  INFO][PID:1068902] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.81.28.139 copr-rpmbuild-log
[2026-08-18 16:09:05,157][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:10,179][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:15,197][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:20,222][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:25,258][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:30,275][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:35,293][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:40,309][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:45,327][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:50,356][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:09:55,399][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:00,413][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:04,905][ ERROR][PID:1068902] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-08-18 16:10:05,428][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:10,431][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:15,280][ ERROR][PID:1068902] Giving up for unstable SSH, failures: 1787069173.934946, 1787069254.206791, 1787069334.6672873, 1787069415.2802248
[2026-08-18 16:10:15,448][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:15,511][ ERROR][PID:1068902] Re-try request for task on 'ResallocHost, ticket_id=4501745, hostname=3.81.28.139, name=aws_x86_64_spot_prod_19341337_20260818_160240, requested_tags=['arch_x86_64', 'copr_builder']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-08-18 16:10:15,712][  INFO][PID:1068902] VM Release request
[2026-08-18 16:10:15,803][  INFO][PID:1068902] Retry #1 (on other host)
[2026-08-18 16:10:15,808][  INFO][PID:1068902] Checking that /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/repodata/repomd.xml exists
[2026-08-18 16:10:15,813][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:16,124][  INFO][PID:1068902] rpmeta: package vulkan-lunarg-tools not known to model, skipping
[2026-08-18 16:10:16,170][  INFO][PID:1068902] VM allocation process starts
[2026-08-18 16:10:16,245][  INFO][PID:1068902] Trying to allocate VM: ResallocHost, ticket_id=4501965, requested_tags=['arch_x86_64', 'copr_builder']
[2026-08-18 16:10:21,257][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:26,267][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:31,282][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:36,322][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:41,337][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:46,346][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:51,358][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:10:56,363][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:01,381][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:06,402][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:11,439][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:16,472][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:21,482][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:26,497][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:31,505][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:36,515][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:41,534][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:46,545][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:51,588][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:11:56,640][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:12:01,646][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:12:06,656][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:12:11,662][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:12:16,671][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:12:21,672][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:12:26,683][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:12:31,701][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:12:34,911][  INFO][PID:1068902] Allocated host ResallocHost, ticket_id=4501965, hostname=3.81.28.139, name=aws_x86_64_spot_prod_19341337_20260818_160240, requested_tags=['arch_x86_64', 'copr_builder']
[2026-08-18 16:12:34,914][  INFO][PID:1068902] Allocating ssh connection to builder
[2026-08-18 16:12:34,917][  INFO][PID:1068902] Checking that builder machine is OK
[2026-08-18 16:13:35,057][ ERROR][PID:1068902] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 3.81.28.139 port 22: Connection timed out

[2026-08-18 16:13:45,145][ ERROR][PID:1068902] Re-try request for task on 'ResallocHost, ticket_id=4501965, hostname=3.81.28.139, name=aws_x86_64_spot_prod_19341337_20260818_160240, requested_tags=['arch_x86_64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-08-18 16:13:45,318][  INFO][PID:1068902] VM Release request
[2026-08-18 16:13:45,406][  INFO][PID:1068902] Retry #2 (on other host)
[2026-08-18 16:13:45,413][  INFO][PID:1068902] Checking that /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/repodata/repomd.xml exists
[2026-08-18 16:13:45,419][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:13:45,634][  INFO][PID:1068902] rpmeta: package vulkan-lunarg-tools not known to model, skipping
[2026-08-18 16:13:45,653][  INFO][PID:1068902] VM allocation process starts
[2026-08-18 16:13:45,733][  INFO][PID:1068902] Trying to allocate VM: ResallocHost, ticket_id=4502013, requested_tags=['arch_x86_64', 'copr_builder']
[2026-08-18 16:13:50,744][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:13:55,750][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:00,762][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:05,778][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:10,787][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:15,797][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:20,808][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:25,817][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:30,823][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:35,838][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:40,851][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:45,862][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:50,867][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:54,103][  INFO][PID:1068902] Allocated host ResallocHost, ticket_id=4502013, hostname=2620:52:6:1161:dead:beef:cafe:c11f, name=vmhost_x86_04_prod_19341308_20260818_160222, requested_tags=['arch_x86_64', 'copr_builder']
[2026-08-18 16:14:54,107][  INFO][PID:1068902] Allocating ssh connection to builder
[2026-08-18 16:14:54,114][  INFO][PID:1068902] Checking that builder machine is OK
[2026-08-18 16:14:54,865][  INFO][PID:1068902] Installed copr-rpmbuild version: 1.9
[2026-08-18 16:14:54,869][  INFO][PID:1068902] Running remote command: copr-builder-ready fedora-45-x86_64
[2026-08-18 16:14:55,385][  INFO][PID:1068902] Red Hat subscription not needed for fedora-45-x86_64
Builder is ready to be used

[2026-08-18 16:14:55,389][  INFO][PID:1068902] Filling build.info file with builder info
[2026-08-18 16:14:55,393][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:14:55,402][  INFO][PID:1068902] Not re-notifying FE and msg buses for the new host.
[2026-08-18 16:14:55,412][  INFO][PID:1068902] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10876842-fedora-45-x86_64 --chroot fedora-45-x86_64 --detached
[2026-08-18 16:14:56,271][  INFO][PID:1068902] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c11f' (ED25519) to the list of known hosts.


[2026-08-18 16:14:56,277][  INFO][PID:1068902] Downloading the builder-live.log file, attempt 1
[2026-08-18 16:14:56,294][  INFO][PID:1068902] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c11f copr-rpmbuild-log
[2026-08-18 16:15:01,278][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:06,286][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:11,292][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:16,299][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:21,306][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:26,328][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:31,399][  INFO][PID:1068902] Periodic builder liveness probe: alive
[2026-08-18 16:15:31,406][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:36,424][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:41,425][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:46,428][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:51,439][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:15:56,442][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:01,453][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:06,469][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:11,478][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:16,494][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:21,503][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:26,520][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:31,539][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:36,547][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:41,564][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:46,583][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:51,594][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:16:56,727][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:01,754][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:06,759][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:11,766][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:16,773][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:21,777][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:26,793][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:31,811][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:36,812][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:41,828][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:46,852][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:51,861][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:17:56,870][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:01,876][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:06,880][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:11,900][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:16,904][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:21,910][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:26,913][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:31,920][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:36,926][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:41,930][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:46,937][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:51,946][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:18:56,953][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:01,958][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:06,964][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:11,968][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:16,973][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:21,983][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:26,987][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:31,991][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:37,024][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:42,058][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:47,078][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:52,080][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:19:57,099][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:02,115][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:07,121][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:12,136][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:17,142][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:22,164][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:27,171][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:32,180][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:37,186][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:42,190][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:47,203][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:52,207][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:20:57,215][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:02,219][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:07,221][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:12,227][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:17,253][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:22,258][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:27,263][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:32,280][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:37,291][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:42,294][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:47,353][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:52,365][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:21:57,370][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:02,378][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:07,395][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:12,398][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:17,417][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:22,435][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:27,439][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:32,453][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:37,471][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:42,484][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:47,496][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:52,506][  INFO][PID:1068902] Checking for cancel request
[2026-08-18 16:22:56,060][  INFO][PID:1068902] Downloading results from builder
[2026-08-18 16:22:56,062][  INFO][PID:1068902] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c11f]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools started
[2026-08-18 16:22:56,073][  INFO][PID:1068902] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c11f]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/ &> /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/build-10876842.rsync.log
[2026-08-18 16:22:57,829][  INFO][PID:1068902] rsyncing finished.
[2026-08-18 16:22:57,836][  INFO][PID:1068902] VM Release request
[2026-08-18 16:22:57,853][  INFO][PID:1068902] Searching for 'success' file in resultdir
[2026-08-18 16:22:57,854][  INFO][PID:1068902] Going to sign pkgs from source: 10876842-fedora-45-x86_64 in chroot: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64
[2026-08-18 16:22:57,855][  INFO][PID:1068902] Calling '/bin/sign -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -p' (attempt #1)
[2026-08-18 16:22:58,172][  INFO][PID:1068902] Calling '/bin/sign -4 -h sha256 -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-extra-layers-debuginfo-1.4.357.0-1.fc45.x86_64.rpm' (attempt #1)
[2026-08-18 16:22:59,300][  INFO][PID:1068902] signed rpm: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-extra-layers-debuginfo-1.4.357.0-1.fc45.x86_64.rpm
[2026-08-18 16:22:59,301][  INFO][PID:1068902] Calling '/bin/sign -4 -h sha256 -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-extra-tools-1.4.357.0-1.fc45.x86_64.rpm' (attempt #1)
[2026-08-18 16:23:00,080][  INFO][PID:1068902] signed rpm: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-extra-tools-1.4.357.0-1.fc45.x86_64.rpm
[2026-08-18 16:23:00,083][  INFO][PID:1068902] Calling '/bin/sign -4 -h sha256 -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-extra-layers-1.4.357.0-1.fc45.x86_64.rpm' (attempt #1)
[2026-08-18 16:23:00,955][  INFO][PID:1068902] signed rpm: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-extra-layers-1.4.357.0-1.fc45.x86_64.rpm
[2026-08-18 16:23:00,964][  INFO][PID:1068902] Calling '/bin/sign -4 -h sha256 -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-lunarg-tools-debugsource-1.4.357.0-1.fc45.x86_64.rpm' (attempt #1)
[2026-08-18 16:23:02,106][  INFO][PID:1068902] signed rpm: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-lunarg-tools-debugsource-1.4.357.0-1.fc45.x86_64.rpm
[2026-08-18 16:23:02,108][  INFO][PID:1068902] Calling '/bin/sign -4 -h sha256 -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-lunarg-tools-1.4.357.0-1.fc45.src.rpm' (attempt #1)
[2026-08-18 16:23:03,678][  INFO][PID:1068902] signed rpm: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-lunarg-tools-1.4.357.0-1.fc45.src.rpm
[2026-08-18 16:23:03,704][  INFO][PID:1068902] Calling '/bin/sign -4 -h sha256 -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-extra-tools-debuginfo-1.4.357.0-1.fc45.x86_64.rpm' (attempt #1)
[2026-08-18 16:23:06,182][  INFO][PID:1068902] signed rpm: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-extra-tools-debuginfo-1.4.357.0-1.fc45.x86_64.rpm
[2026-08-18 16:23:06,206][  INFO][PID:1068902] Calling '/bin/sign -4 -h sha256 -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -r /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-lunarg-tools-debuginfo-1.4.357.0-1.fc45.x86_64.rpm' (attempt #1)
[2026-08-18 16:23:07,817][  INFO][PID:1068902] signed rpm: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/vulkan-lunarg-tools-debuginfo-1.4.357.0-1.fc45.x86_64.rpm
[2026-08-18 16:23:07,818][  INFO][PID:1068902] Sign done
[2026-08-18 16:23:07,819][  INFO][PID:1068902] Incremental createrepo run, adding 10876842-vulkan-lunarg-tools into https://download.copr.fedorainfracloud.org/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64, (auto-create-repo=True)
[2026-08-18 16:23:07,826][  INFO][PID:1068902] Running command 'copr-repo --batched /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64 --add 10876842-vulkan-lunarg-tools --no-appstream-metadata' as PID 1306365
[2026-08-18 16:23:11,268][  INFO][PID:1068902] Finished after 3 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64 --add 10876842-vulkan-lunarg-tools --no-appstream-metadata)
[2026-08-18 16:23:11,286][  INFO][PID:1068902] Getting build details
[2026-08-18 16:23:11,291][  INFO][PID:1068902] Listing built binary packages in /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools
[2026-08-18 16:23:11,303][  INFO][PID:1068902] Built packages:
['vulkan-lunarg-tools-debugsource 1.4.357.0', 'vulkan-extra-tools 1.4.357.0', 'vulkan-extra-layers 1.4.357.0', 'vulkan-extra-tools-debuginfo 1.4.357.0', 'vulkan-lunarg-tools-debuginfo 1.4.357.0', 'vulkan-extra-layers-debuginfo 1.4.357.0']
[2026-08-18 16:23:11,313][  INFO][PID:1068902] build details: {'built_packages': 'vulkan-lunarg-tools-debugsource 1.4.357.0\nvulkan-extra-tools 1.4.357.0\nvulkan-extra-layers 1.4.357.0\nvulkan-extra-tools-debuginfo 1.4.357.0\nvulkan-lunarg-tools-debuginfo 1.4.357.0\nvulkan-extra-layers-debuginfo 1.4.357.0'}
[2026-08-18 16:23:11,317][  INFO][PID:1068902] Retrieving pubkey
[2026-08-18 16:23:11,319][  INFO][PID:1068902] Calling '/bin/sign -u tellowkrinkle#vulkan-tools@copr.fedorahosted.org -p' (attempt #1)
[2026-08-18 16:23:11,690][  INFO][PID:1068902] Added pubkey for user tellowkrinkle project vulkan-tools into: /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/pubkey.gpg
[2026-08-18 16:23:11,691][  INFO][PID:1068902] Finished build: id=10876842 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools chroot=fedora-45-x86_64 
[2026-08-18 16:23:11,716][  INFO][PID:1068902] Worker succeeded build, took 496.31375527381897
[2026-08-18 16:23:11,732][  INFO][PID:1068902] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "tellowkrinkle",
            "project_name": "vulkan-tools",
            "project_dirname": "vulkan-tools",
            "submitter": "tellowkrinkle",
            "ended_on": 1787070191.7162092,
            "started_on": 1787069695.402454,
            "submitted_on": null,
            "status": 1,
            "chroot": "fedora-45-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "10876842-fedora-45-x86_64",
            "build_id": 10876842,
            "package_name": "vulkan-lunarg-tools",
            "package_version": "1.4.357.0-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/tellowkrinkle/vulkan-tools/vulkan-lunarg-tools",
            "git_hash": "718b9657fedf2d6070c9503dfd28dafa1987f53e",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "tellowkrinkle/vulkan-tools--tellowkrinkle",
            "results": {
                "packages": [
                    {
                        "name": "vulkan-lunarg-tools-debugsource",
                        "epoch": null,
                        "version": "1.4.357.0",
                        "release": "1.fc45",
                        "arch": "x86_64"
                    },
                    {
                        "name": "vulkan-extra-tools",
                        "epoch": null,
                        "version": "1.4.357.0",
                        "release": "1.fc45",
                        "arch": "x86_64"
                    },
                    {
                        "name": "vulkan-extra-layers",
                        "epoch": null,
                        "version": "1.4.357.0",
                        "release": "1.fc45",
                        "arch": "x86_64"
                    },
                    {
                        "name": "vulkan-extra-tools-debuginfo",
                        "epoch": null,
                        "version": "1.4.357.0",
                        "release": "1.fc45",
                        "arch": "x86_64"
                    },
                    {
                        "name": "vulkan-lunarg-tools-debuginfo",
                        "epoch": null,
                        "version": "1.4.357.0",
                        "release": "1.fc45",
                        "arch": "x86_64"
                    },
                    {
                        "name": "vulkan-extra-layers-debuginfo",
                        "epoch": null,
                        "version": "1.4.357.0",
                        "release": "1.fc45",
                        "arch": "x86_64"
                    },
                    {
                        "name": "vulkan-lunarg-tools",
                        "epoch": null,
                        "version": "1.4.357.0",
                        "release": "1.fc45",
                        "arch": "src"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": 105,
                    "repo_gpgcheck": false
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": 105,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/tellowkrinkle/vulkan-tools",
            "result_dir": "10876842-vulkan-lunarg-tools",
            "built_packages": "vulkan-lunarg-tools-debugsource 1.4.357.0\nvulkan-extra-tools 1.4.357.0\nvulkan-extra-layers 1.4.357.0\nvulkan-extra-tools-debuginfo 1.4.357.0\nvulkan-lunarg-tools-debuginfo 1.4.357.0\nvulkan-extra-layers-debuginfo 1.4.357.0",
            "rpmeta_prediction": null,
            "id": 10876842,
            "mockchain_macros": {
                "copr_username": "tellowkrinkle",
                "copr_projectname": "vulkan-tools",
                "vendor": "Fedora Project COPR (tellowkrinkle/vulkan-tools)"
            }
        }
    ]
}
[2026-08-18 16:23:11,961][  INFO][PID:1068902] Sending fedora-messaging bus message in build.end
[2026-08-18 16:23:12,168][  INFO][PID:1068902] Compressing /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/builder-live.log by gzip
[2026-08-18 16:23:12,176][  INFO][PID:1068902] Running command 'gzip /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/builder-live.log' as PID 1307305
[2026-08-18 16:23:12,319][  INFO][PID:1068902] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/builder-live.log)
[2026-08-18 16:23:12,336][  INFO][PID:1068902] Compressing /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/backend.log by gzip
[2026-08-18 16:23:12,346][  INFO][PID:1068902] Running command 'gzip /var/lib/copr/public_html/results/tellowkrinkle/vulkan-tools/fedora-45-x86_64/10876842-vulkan-lunarg-tools/backend.log' as PID 1307334