[2026-04-20 13:56:02,879][  INFO][PID:2479103] Marking build as starting
[2026-04-20 13:56:02,992][  INFO][PID:2479103] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 13:56:04,390][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:04,415][  INFO][PID:2479103] VM allocation process starts
[2026-04-20 13:56:04,500][  INFO][PID:2479103] Trying to allocate VM: ResallocHost, ticket_id=2423249, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:56:09,528][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:14,535][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:19,553][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:24,558][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:29,561][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:34,567][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:39,572][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:44,577][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:49,587][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:54,592][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:56:59,596][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:04,599][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:09,611][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:14,617][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:19,620][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:24,667][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:29,671][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:34,681][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:39,687][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:44,693][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:49,703][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:54,724][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:57:59,739][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:04,741][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:09,747][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:14,760][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:19,767][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:23,057][  INFO][PID:2479103] Allocated host ResallocHost, ticket_id=2423249, hostname=107.23.247.33, name=aws_aarch64_eserved_prod_07133766_20260420_135520, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:58:23,058][  INFO][PID:2479103] Allocating ssh connection to builder
[2026-04-20 13:58:23,066][  INFO][PID:2479103] Checking that builder machine is OK
[2026-04-20 13:58:23,498][ ERROR][PID:2479103] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '107.23.247.33' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@107.23.247.33: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 13:58:33,681][ ERROR][PID:2479103] Re-try request for task on 'ResallocHost, ticket_id=2423249, hostname=107.23.247.33, name=aws_aarch64_eserved_prod_07133766_20260420_135520, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 13:58:33,703][  INFO][PID:2479103] VM Release request
[2026-04-20 13:58:33,779][  INFO][PID:2479103] Retry #1 (on other host)
[2026-04-20 13:58:33,787][  INFO][PID:2479103] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 13:58:34,200][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:34,216][  INFO][PID:2479103] VM allocation process starts
[2026-04-20 13:58:34,342][  INFO][PID:2479103] Trying to allocate VM: ResallocHost, ticket_id=2423362, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 13:58:39,355][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:44,366][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:49,369][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:54,376][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:58:59,398][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:04,403][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:09,406][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:14,412][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:19,417][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:24,421][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:29,427][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:34,445][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:39,462][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:44,467][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:49,481][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:54,489][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 13:59:59,492][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:04,496][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:09,512][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:14,516][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:19,566][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:24,575][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:29,580][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:34,618][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:39,643][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:44,661][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:49,676][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:54,691][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:00:59,705][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:04,726][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:09,736][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:14,773][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:19,828][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:24,840][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:29,884][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:34,938][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:39,960][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:44,970][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:49,977][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:01:55,002][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:00,076][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:05,096][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:10,099][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:15,103][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:20,105][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:25,135][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:30,182][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:35,210][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:40,221][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:45,236][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:50,252][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:02:55,265][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:00,285][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:05,294][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:10,296][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:15,303][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:20,306][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:25,321][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:30,324][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:35,328][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:40,339][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:45,348][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:50,358][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:03:53,274][  INFO][PID:2479103] Allocated host ResallocHost, ticket_id=2423362, hostname=54.152.104.239, name=aws_aarch64_reserved_prod_07133980_20260420_135956, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 14:03:53,275][  INFO][PID:2479103] Allocating ssh connection to builder
[2026-04-20 14:03:53,278][  INFO][PID:2479103] Checking that builder machine is OK
[2026-04-20 14:03:53,619][ ERROR][PID:2479103] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '54.152.104.239' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@54.152.104.239: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 14:04:03,903][ ERROR][PID:2479103] Re-try request for task on 'ResallocHost, ticket_id=2423362, hostname=54.152.104.239, name=aws_aarch64_reserved_prod_07133980_20260420_135956, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 14:04:04,004][  INFO][PID:2479103] VM Release request
[2026-04-20 14:04:04,087][  INFO][PID:2479103] Retry #2 (on other host)
[2026-04-20 14:04:04,088][  INFO][PID:2479103] Checking that https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64//repodata/repomd.xml exists
[2026-04-20 14:04:04,943][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:04,951][  INFO][PID:2479103] VM allocation process starts
[2026-04-20 14:04:05,024][  INFO][PID:2479103] Trying to allocate VM: ResallocHost, ticket_id=2423510, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 14:04:10,037][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:15,041][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:20,056][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:25,075][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:30,076][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:35,078][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:40,092][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:45,098][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:50,113][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:04:55,132][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:00,152][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:05,166][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:10,201][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:15,203][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:20,217][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:25,242][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:30,249][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:35,257][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:40,265][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:45,276][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:50,278][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:05:55,281][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:00,284][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:05,289][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:10,292][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:15,295][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:20,304][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:25,309][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:30,314][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:35,317][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:40,321][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:45,325][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:50,329][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:06:55,333][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:07:00,342][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:07:05,353][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:07:10,365][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:07:15,375][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:07:20,386][  INFO][PID:2479103] Checking for cancel request
[2026-04-20 14:07:23,989][  INFO][PID:2479103] Allocated host ResallocHost, ticket_id=2423510, hostname=54.234.171.179, name=aws_aarch64_reserved_prod_07134149_20260420_140322, requested_tags=['arch_aarch64', 'copr_builder']
[2026-04-20 14:07:24,020][  INFO][PID:2479103] Allocating ssh connection to builder
[2026-04-20 14:07:24,029][  INFO][PID:2479103] Checking that builder machine is OK
[2026-04-20 14:07:24,725][ ERROR][PID:2479103] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
Warning: Permanently added '54.234.171.179' (ED25519) to the list of known hosts.
Load key "/home/copr/.ssh/id_rsa": Permission denied
Permission denied, please try again.
Permission denied, please try again.
mockbuilder@54.234.171.179: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).

[2026-04-20 14:07:35,150][ ERROR][PID:2479103] Re-try request for task on 'ResallocHost, ticket_id=2423510, hostname=54.234.171.179, name=aws_aarch64_reserved_prod_07134149_20260420_140322, requested_tags=['arch_aarch64', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-20 14:07:35,178][  INFO][PID:2479103] VM Release request
[2026-04-20 14:07:35,283][ ERROR][PID:2479103] Backend process error: Three host tried without success: {'54.234.171.179', '107.23.247.33', '54.152.104.239'}
[2026-04-20 14:07:35,324][WARNING][PID:2479103] Switching not-finished job state to 'failed'
[2026-04-20 14:07:35,325][  INFO][PID:2479103] Worker failed build, took 692.5262370109558
[2026-04-20 14:07:35,326][  INFO][PID:2479103] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "xsnrg",
            "project_name": "indi-3rdparty-bleeding",
            "project_dirname": "indi-3rdparty-bleeding",
            "submitter": "https://api.github.com/users/knro",
            "ended_on": 1776694055.3244655,
            "started_on": 1776693362.7982285,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-43-aarch64",
            "arch": "aarch64",
            "buildroot_pkgs": [],
            "task_id": "10348523-fedora-43-aarch64",
            "build_id": 10348523,
            "package_name": "indi-armadillo-platypus",
            "package_version": "2.1.3.git-20260420131436",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/xsnrg/indi-3rdparty-bleeding/indi-armadillo-platypus",
            "git_hash": "cc8c9b441cc18c94ac0f3bdcd9a11970d082dd57",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "xsnrg/indi-3rdparty-bleeding--https://api.github.com/users/knro",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 1,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/libindi-bleeding/fedora-43-aarch64/",
                    "id": "copr_xsnrg_libindi_bleeding",
                    "name": "Additional repo copr_xsnrg_libindi_bleeding"
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/",
                    "id": "copr_xsnrg_indi_3rdparty_bleeding",
                    "name": "Additional repo copr_xsnrg_indi_3rdparty_bleeding"
                }
            ],
            "background": true,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_aarch64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/xsnrg/indi-3rdparty-bleeding",
            "result_dir": "10348523-indi-armadillo-platypus",
            "built_packages": "",
            "id": 10348523,
            "mockchain_macros": {
                "copr_username": "xsnrg",
                "copr_projectname": "indi-3rdparty-bleeding",
                "vendor": "Fedora Project COPR (xsnrg/indi-3rdparty-bleeding)"
            }
        }
    ]
}
[2026-04-20 14:07:35,428][  INFO][PID:2479103] Sending fedora-messaging bus message in build.end
[2026-04-20 14:07:43,553][  INFO][PID:2479103] Compressing /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348523-indi-armadillo-platypus/builder-live.log by gzip
[2026-04-20 14:07:43,570][  INFO][PID:2479103] Running command 'gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348523-indi-armadillo-platypus/builder-live.log' as PID 2672508
[2026-04-20 14:07:43,579][  INFO][PID:2479103] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348523-indi-armadillo-platypus/builder-live.log)
stdout:

stderr:
gzip: /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348523-indi-armadillo-platypus/builder-live.log: No such file or directory


[2026-04-20 14:07:43,586][ ERROR][PID:2479103] Unable to compress file /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348523-indi-armadillo-platypus/builder-live.log
[2026-04-20 14:07:43,694][  INFO][PID:2479103] Compressing /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348523-indi-armadillo-platypus/backend.log by gzip
[2026-04-20 14:07:43,720][  INFO][PID:2479103] Running command 'gzip /var/lib/copr/public_html/results/xsnrg/indi-3rdparty-bleeding/fedora-43-aarch64/10348523-indi-armadillo-platypus/backend.log' as PID 2672522