[2026-04-20 13:53:18,424][  INFO][PID:2441539] Marking build as starting
[2026-04-20 13:53:18,615][  INFO][PID:2441539] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 13:53:19,656][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:53:19,699][  INFO][PID:2441539] VM allocation process starts
[2026-04-20 13:53:19,748][  INFO][PID:2441539] Trying to allocate VM: ResallocHost, ticket_id=2423136, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:53:24,782][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:53:29,797][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:53:34,799][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:53:39,809][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:53:44,831][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:53:49,859][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:53:54,871][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:53:59,873][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:04,877][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:09,891][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:14,897][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:19,902][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:24,914][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:29,919][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:34,930][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:39,938][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:44,945][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:49,948][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:54,954][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:54:59,959][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:04,969][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:09,973][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:14,983][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:19,991][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:24,999][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:30,004][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:35,009][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:38,514][  INFO][PID:2441539] Allocated host ResallocHost, ticket_id=2423136, hostname=52.207.223.245, name=aws_aarch64_eserved_prod_07133603_20260420_135145, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:55:38,521][  INFO][PID:2441539] Allocating ssh connection to builder
[2026-04-20 13:55:38,525][  INFO][PID:2441539] Checking that builder machine is OK
[2026-04-20 13:55:38,813][ ERROR][PID:2441539] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '52.207.223.245' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@52.207.223.245: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:55:48,856][ ERROR][PID:2441539] Re-try request for task on 'ResallocHost, ticket_id=2423136, hostname=52.207.223.245, name=aws_aarch64_eserved_prod_07133603_20260420_135145, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:55:48,941][  INFO][PID:2441539] VM Release request
[2026-04-20 13:55:49,009][  INFO][PID:2441539] Retry #1 (on other host)
[2026-04-20 13:55:49,011][  INFO][PID:2441539] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 13:55:51,559][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:55:51,600][  INFO][PID:2441539] VM allocation process starts
[2026-04-20 13:55:51,706][  INFO][PID:2441539] Trying to allocate VM: ResallocHost, ticket_id=2423239, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:55:56,726][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:01,736][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:06,754][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:11,767][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:16,784][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:21,786][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:26,789][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:31,793][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:36,802][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:41,817][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:46,830][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:51,836][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:56:56,839][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:01,841][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:06,846][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:11,850][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:16,853][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:21,864][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:26,867][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:31,874][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:36,881][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:40,319][  INFO][PID:2441539] Allocated host ResallocHost, ticket_id=2423239, hostname=34.229.117.69, name=aws_aarch64_eserved_prod_07133748_20260420_135455, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:57:40,322][  INFO][PID:2441539] Allocating ssh connection to builder
[2026-04-20 13:57:40,325][  INFO][PID:2441539] Checking that builder machine is OK
[2026-04-20 13:57:40,483][ ERROR][PID:2441539] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '34.229.117.69' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@34.229.117.69: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:57:50,504][ ERROR][PID:2441539] Re-try request for task on 'ResallocHost, ticket_id=2423239, hostname=34.229.117.69, name=aws_aarch64_eserved_prod_07133748_20260420_135455, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:57:50,581][  INFO][PID:2441539] VM Release request
[2026-04-20 13:57:50,622][  INFO][PID:2441539] Retry #2 (on other host)
[2026-04-20 13:57:50,632][  INFO][PID:2441539] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 13:57:51,442][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:57:51,469][  INFO][PID:2441539] VM allocation process starts
[2026-04-20 13:57:51,520][  INFO][PID:2441539] Trying to allocate VM: ResallocHost, ticket_id=2423314, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:57:56,534][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:01,541][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:06,563][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:11,575][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:16,603][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:21,631][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:26,657][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:31,685][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:36,707][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:41,726][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:46,728][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:51,732][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:58:56,737][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:01,741][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:06,744][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:11,750][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:16,758][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:21,773][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:26,788][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:31,792][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:36,796][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:41,799][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:46,806][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:51,808][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 13:59:56,817][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:01,976][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:06,994][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:12,010][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:17,017][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:22,018][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:27,039][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:32,059][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:37,078][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:42,118][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:47,121][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:52,125][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:00:57,133][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:01:02,141][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:01:07,154][  INFO][PID:2441539] Checking for cancel request
[2026-04-20 14:01:10,357][  INFO][PID:2441539] Allocated host ResallocHost, ticket_id=2423314, hostname=3.80.199.122, name=aws_aarch64_eserved_prod_07133903_20260420_135753, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 14:01:10,389][  INFO][PID:2441539] Allocating ssh connection to builder
[2026-04-20 14:01:10,391][  INFO][PID:2441539] Checking that builder machine is OK
[2026-04-20 14:01:10,782][ ERROR][PID:2441539] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '3.80.199.122' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@3.80.199.122: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 14:01:21,145][ ERROR][PID:2441539] Re-try request for task on 'ResallocHost, ticket_id=2423314, hostname=3.80.199.122, name=aws_aarch64_eserved_prod_07133903_20260420_135753, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 14:01:21,362][  INFO][PID:2441539] VM Release request
[2026-04-20 14:01:21,482][ ERROR][PID:2441539] Backend process error: Three host tried without success: {'3.80.199.122', '52.207.223.245', '34.229.117.69'}
[2026-04-20 14:01:21,776][WARNING][PID:2441539] Switching not-finished job state to 'failed'
[2026-04-20 14:01:21,796][  INFO][PID:2441539] Worker failed build, took 483.5384347438812
[2026-04-20 14:01:21,812][  INFO][PID:2441539] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "xsnrg",
            "project_name": "indi-3rdparty-bleeding",
            "project_dirname": "indi-3rdparty-bleeding",
            "submitter": "https://api.github.com/users/knro",
            "ended_on": 1776693681.776675,
            "started_on": 1776693198.2382402,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-43-aarch64",
            "arch": "aarch64",
            "buildroot_pkgs": [],
            "task_id": "10348521-fedora-43-aarch64",
            "build_id": 10348521,
            "package_name": "indi-aok",
            "package_version": "2.1.3.git-20260420131440",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/xsnrg/indi-3rdparty-bleeding/indi-aok",
            "git_hash": "a74771f411b974aca4b5302a3f27599a247e72b3",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "xsnrg/indi-3rdparty-bleeding--https://api.github.com/users/knro",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 1,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/libindi-bleeding/fedora-43-aarch64/",
                    "id": "copr_xsnrg_libindi_bleeding",
                    "name": "Additional repo copr_xsnrg_libindi_bleeding"
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/",
                    "id": "copr_xsnrg_indi_3rdparty_bleeding",
                    "name": "Additional repo copr_xsnrg_indi_3rdparty_bleeding"
                }
            ],
            "background": true,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_aarch64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding",
            "result_dir": "10348521-indi-aok",
            "built_packages": "",
            "id": 10348521,
            "mockchain_macros": {
                "copr_username": "xsnrg",
                "copr_projectname": "indi-3rdparty-bleeding",
                "vendor": "Fedora Project COPR (xsnrg/indi-3rdparty-bleeding)"
            }
        }
    ]
}
[2026-04-20 14:01:22,259][  INFO][PID:2441539] Sending fedora-messaging bus message in build.end
[2026-04-20 14:01:24,596][  INFO][PID:2441539] Compressing /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348521-indi-aok/builder-live.log by gzip
[2026-04-20 14:01:24,604][  INFO][PID:2441539] Running command 'gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348521-indi-aok/builder-live.log' as PID 2556736
[2026-04-20 14:01:24,614][  INFO][PID:2441539] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348521-indi-aok/builder-live.log)
stdout:

stderr:
gzip: /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348521-indi-aok/builder-live.log: No such file or directory


[2026-04-20 14:01:24,629][ ERROR][PID:2441539] Unable to compress file /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348521-indi-aok/builder-live.log
[2026-04-20 14:01:24,870][  INFO][PID:2441539] Compressing /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348521-indi-aok/backend.log by gzip
[2026-04-20 14:01:24,895][  INFO][PID:2441539] Running command 'gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348521-indi-aok/backend.log' as PID 2556824