[2026-04-20 13:27:12,155][  INFO][PID:2075307] Marking build as starting
[2026-04-20 13:27:12,374][  INFO][PID:2075307] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 13:27:12,951][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:27:12,953][  INFO][PID:2075307] VM allocation process starts
[2026-04-20 13:27:13,004][  INFO][PID:2075307] Trying to allocate VM: ResallocHost, ticket_id=2422234, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:27:18,007][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:27:23,048][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:27:28,100][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:27:33,232][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:27:38,292][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:27:43,297][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:27:48,329][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:27:51,287][  INFO][PID:2075307] Allocated host ResallocHost, ticket_id=2422234, hostname=52.205.253.155, name=aws_aarch64_normalreserved_prod_07129637_20260420_130701, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:27:51,288][  INFO][PID:2075307] Allocating ssh connection to builder
[2026-04-20 13:27:51,289][  INFO][PID:2075307] Checking that builder machine is OK
[2026-04-20 13:27:51,406][ ERROR][PID:2075307] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '52.205.253.155' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@52.205.253.155: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:28:01,410][ ERROR][PID:2075307] Re-try request for task on 'ResallocHost, ticket_id=2422234, hostname=52.205.253.155, name=aws_aarch64_normalreserved_prod_07129637_20260420_130701, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:28:01,429][  INFO][PID:2075307] VM Release request
[2026-04-20 13:28:01,480][  INFO][PID:2075307] Retry #1 (on other host)
[2026-04-20 13:28:01,482][  INFO][PID:2075307] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 13:28:02,722][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:02,736][  INFO][PID:2075307] VM allocation process starts
[2026-04-20 13:28:02,777][  INFO][PID:2075307] Trying to allocate VM: ResallocHost, ticket_id=2422278, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:28:07,781][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:12,783][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:17,786][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:22,795][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:27,799][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:32,801][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:37,826][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:42,866][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:47,869][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:52,896][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:28:57,900][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:02,907][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:07,908][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:12,912][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:17,917][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:22,924][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:27,926][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:32,935][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:37,939][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:43,024][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:48,035][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:53,037][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:29:58,091][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:03,106][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:08,108][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:13,115][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:18,118][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:23,120][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:28,124][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:33,131][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:38,135][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:43,180][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:48,181][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:53,298][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:30:58,302][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:03,310][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:08,313][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:13,319][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:18,321][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:23,323][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:28,326][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:33,331][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:38,349][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:43,350][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:48,351][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:53,353][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:31:58,354][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:03,356][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:08,359][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:13,361][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:18,362][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:21,318][  INFO][PID:2075307] Allocated host ResallocHost, ticket_id=2422278, hostname=13.222.52.218, name=aws_aarch64_normalreserved_prod_07132356_20260420_132949, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:32:21,320][  INFO][PID:2075307] Allocating ssh connection to builder
[2026-04-20 13:32:21,321][  INFO][PID:2075307] Checking that builder machine is OK
[2026-04-20 13:32:21,469][ ERROR][PID:2075307] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '13.222.52.218' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@13.222.52.218: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:32:31,470][ ERROR][PID:2075307] Re-try request for task on 'ResallocHost, ticket_id=2422278, hostname=13.222.52.218, name=aws_aarch64_normalreserved_prod_07132356_20260420_132949, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:32:31,471][  INFO][PID:2075307] VM Release request
[2026-04-20 13:32:31,496][  INFO][PID:2075307] Retry #2 (on other host)
[2026-04-20 13:32:31,497][  INFO][PID:2075307] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 13:32:32,367][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:32,369][  INFO][PID:2075307] VM allocation process starts
[2026-04-20 13:32:32,390][  INFO][PID:2075307] Trying to allocate VM: ResallocHost, ticket_id=2422469, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:32:37,393][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:42,395][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:47,408][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:52,417][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:32:57,418][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:02,426][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:07,441][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:12,451][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:17,455][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:22,458][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:27,463][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:32,467][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:37,469][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:42,471][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:47,472][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:52,473][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:33:57,477][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:02,485][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:07,487][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:12,491][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:17,496][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:22,498][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:27,503][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:32,505][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:37,508][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:42,518][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:47,524][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:52,527][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:34:57,532][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:02,539][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:07,545][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:12,558][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:17,568][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:22,579][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:27,585][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:32,587][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:37,590][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:42,593][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:47,597][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:52,598][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:35:57,600][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:02,606][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:07,619][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:12,620][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:17,621][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:22,634][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:27,635][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:32,639][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:37,645][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:42,657][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:47,668][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:52,670][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:36:57,682][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:02,692][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:07,722][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:12,723][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:17,729][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:22,738][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:27,754][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:32,762][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:37,766][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:42,768][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:47,776][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:52,777][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:37:57,782][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:02,788][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:07,790][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:12,801][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:17,817][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:22,823][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:27,833][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:32,845][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:37,849][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:42,853][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:47,858][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:52,870][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:38:57,878][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:02,888][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:07,889][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:12,892][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:17,897][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:22,906][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:27,908][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:32,910][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:37,915][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:42,917][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:47,924][  INFO][PID:2075307] Checking for cancel request
[2026-04-20 13:39:50,879][  INFO][PID:2075307] Allocated host ResallocHost, ticket_id=2422469, hostname=32.192.199.1, name=aws_aarch64_normalreserved_prod_07132713_20260420_133618, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:39:50,880][  INFO][PID:2075307] Allocating ssh connection to builder
[2026-04-20 13:39:50,881][  INFO][PID:2075307] Checking that builder machine is OK
[2026-04-20 13:39:51,012][ ERROR][PID:2075307] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '32.192.199.1' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@32.192.199.1: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:40:01,014][ ERROR][PID:2075307] Re-try request for task on 'ResallocHost, ticket_id=2422469, hostname=32.192.199.1, name=aws_aarch64_normalreserved_prod_07132713_20260420_133618, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:40:01,017][  INFO][PID:2075307] VM Release request
[2026-04-20 13:40:01,064][ ERROR][PID:2075307] Backend process error: Three host tried without success: {'13.222.52.218', '32.192.199.1', '52.205.253.155'}
[2026-04-20 13:40:01,065][WARNING][PID:2075307] Switching not-finished job state to 'failed'
[2026-04-20 13:40:01,067][  INFO][PID:2075307] Worker failed build, took 768.9179058074951
[2026-04-20 13:40:01,069][  INFO][PID:2075307] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "xsnrg",
            "project_name": "indi-3rdparty-bleeding",
            "project_dirname": "indi-3rdparty-bleeding",
            "submitter": "https://api.github.com/users/knro",
            "ended_on": 1776692401.065684,
            "started_on": 1776691632.1477783,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-43-aarch64",
            "arch": "aarch64",
            "buildroot_pkgs": [],
            "task_id": "10348511-fedora-43-aarch64",
            "build_id": 10348511,
            "package_name": "libtoupcam",
            "package_version": "2.1.3.git-20260420131447",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/xsnrg/indi-3rdparty-bleeding/libtoupcam",
            "git_hash": "7b0fb0e1a4ad998f28184a316416d4d57afec811",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "xsnrg/indi-3rdparty-bleeding--https://api.github.com/users/knro",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 1,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/libindi-bleeding/fedora-43-aarch64/",
                    "id": "copr_xsnrg_libindi_bleeding",
                    "name": "Additional repo copr_xsnrg_libindi_bleeding"
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/",
                    "id": "copr_xsnrg_indi_3rdparty_bleeding",
                    "name": "Additional repo copr_xsnrg_indi_3rdparty_bleeding"
                }
            ],
            "background": true,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_aarch64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding",
            "result_dir": "10348511-libtoupcam",
            "built_packages": "",
            "id": 10348511,
            "mockchain_macros": {
                "copr_username": "xsnrg",
                "copr_projectname": "indi-3rdparty-bleeding",
                "vendor": "Fedora Project COPR (xsnrg/indi-3rdparty-bleeding)"
            }
        }
    ]
}
[2026-04-20 13:40:01,185][  INFO][PID:2075307] Sending fedora-messaging bus message in build.end
[2026-04-20 13:40:03,293][  INFO][PID:2075307] Compressing /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348511-libtoupcam/builder-live.log by gzip
[2026-04-20 13:40:03,296][  INFO][PID:2075307] Running command 'gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348511-libtoupcam/builder-live.log' as PID 2262692
[2026-04-20 13:40:03,299][  INFO][PID:2075307] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348511-libtoupcam/builder-live.log)
stdout:

stderr:
gzip: /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348511-libtoupcam/builder-live.log: No such file or directory


[2026-04-20 13:40:03,301][ ERROR][PID:2075307] Unable to compress file /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348511-libtoupcam/builder-live.log
[2026-04-20 13:40:03,302][  INFO][PID:2075307] Compressing /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348511-libtoupcam/backend.log by gzip
[2026-04-20 13:40:03,308][  INFO][PID:2075307] Running command 'gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348511-libtoupcam/backend.log' as PID 2262698