[2025-01-08 14:55:07,272][  INFO][PID:52308] Marking build as starting
[2025-01-08 14:55:07,296][  INFO][PID:52308] Trying to allocate VM
[2025-01-08 14:55:10,328][  INFO][PID:52308] Allocated host ResallocHost, ticket_id=568892, hostname=54.173.155.15
[2025-01-08 14:55:10,329][  INFO][PID:52308] Allocating ssh connection to builder
[2025-01-08 14:55:10,330][  INFO][PID:52308] Checking that builder machine is OK
[2025-01-08 14:55:10,688][  INFO][PID:52308] Running remote command: copr-builder-ready fedora-40-x86_64
[2025-01-08 14:55:10,910][  INFO][PID:52308] Red Hat subscription not needed for fedora-40-x86_64
Builder is ready to be used

[2025-01-08 14:55:10,911][  INFO][PID:52308] Filling build.info file with builder info
[2025-01-08 14:55:10,915][  INFO][PID:52308] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "churchyard",
            "project_name": "jinja2-3.1.5-f40",
            "project_dirname": "jinja2-3.1.5-f40:custom:isolated",
            "submitter": "churchyard",
            "ended_on": null,
            "started_on": 1736348110.9146278,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-40-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "8487458-fedora-40-x86_64",
            "build_id": 8487458,
            "package_name": "python-xbout",
            "package_version": "0.3.6-2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/churchyard/jinja2-3.1.5-f40:custom:isolated/python-xbout",
            "git_hash": "f488f4b4de142836f9446ba597e4861ef1adec34",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "churchyard/jinja2-3.1.5-f40--churchyard",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/churchyard/jinja2-3.1.5-f40/fedora-40-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/f40-build/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_f40_build_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_f40_build_latest_basearch"
                }
            ],
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/churchyard/jinja2-3.1.5-f40:custom:isolated",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/churchyard/jinja2-3.1.5-f40:custom:isolated",
            "result_dir": "08487458-python-xbout",
            "built_packages": "",
            "id": 8487458,
            "mockchain_macros": {
                "copr_username": "churchyard",
                "copr_projectname": "jinja2-3.1.5-f40",
                "vendor": "Fedora Project COPR (churchyard/jinja2-3.1.5-f40)"
            }
        }
    ]
}
[2025-01-08 14:55:10,946][  INFO][PID:52308] Sending fedora-messaging bus message in build.start
[2025-01-08 14:55:11,582][  INFO][PID:52308] Sending fedora-messaging bus message in chroot.start
[2025-01-08 14:55:11,600][  INFO][PID:52308] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8487458-fedora-40-x86_64 --chroot fedora-40-x86_64 --detached
[2025-01-08 14:55:11,978][  INFO][PID:52308] Downloading the builder-live.log file, attempt 1
[2025-01-08 14:55:11,980][  INFO][PID:52308] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.173.155.15 copr-rpmbuild-log
[2025-01-08 14:55:16,987][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 15:05:17,098][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 15:15:17,243][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 15:25:17,355][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 15:35:17,434][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 15:45:17,509][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 15:55:17,585][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 16:05:17,659][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 16:15:17,755][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 16:25:17,839][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 16:35:17,921][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 16:45:18,012][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 16:55:18,123][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 17:05:18,332][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 17:10:29,910][ ERROR][PID:52308] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-08 17:10:39,911][WARNING][PID:52308] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-08 17:10:49,912][  INFO][PID:52308] Downloading the builder-live.log file, attempt 2
[2025-01-08 17:10:49,913][  INFO][PID:52308] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.173.155.15 copr-rpmbuild-log
[2025-01-08 17:10:49,922][ ERROR][PID:52308] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-08 17:10:59,923][WARNING][PID:52308] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-08 17:11:09,923][  INFO][PID:52308] Downloading the builder-live.log file, attempt 3
[2025-01-08 17:11:09,924][  INFO][PID:52308] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.173.155.15 copr-rpmbuild-log
[2025-01-08 17:12:09,992][ ERROR][PID:52308] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-08 17:12:19,995][WARNING][PID:52308] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-08 17:12:29,996][  INFO][PID:52308] Downloading the builder-live.log file, attempt 4
[2025-01-08 17:12:29,998][  INFO][PID:52308] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.173.155.15 copr-rpmbuild-log
[2025-01-08 17:13:30,068][ ERROR][PID:52308] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-08 17:13:40,073][ ERROR][PID:52308] Giving up for unstable SSH, failures: 1736356239.9115195, 1736356259.9230201, 1736356339.9953818, 1736356420.0733204
[2025-01-08 17:13:40,075][ ERROR][PID:52308] Re-try request for task on 'ResallocHost, ticket_id=568892, hostname=54.173.155.15': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-01-08 17:13:40,078][  INFO][PID:52308] Releasing VM back to pool
[2025-01-08 17:13:40,122][  INFO][PID:52308] Retry #1 (on other host)
[2025-01-08 17:13:40,125][  INFO][PID:52308] Trying to allocate VM
[2025-01-08 17:15:58,511][  INFO][PID:52308] Allocated host ResallocHost, ticket_id=570872, hostname=98.84.127.42
[2025-01-08 17:15:58,521][  INFO][PID:52308] Allocating ssh connection to builder
[2025-01-08 17:15:58,522][  INFO][PID:52308] Checking that builder machine is OK
[2025-01-08 17:15:58,878][  INFO][PID:52308] Running remote command: copr-builder-ready fedora-40-x86_64
[2025-01-08 17:15:59,104][  INFO][PID:52308] Red Hat subscription not needed for fedora-40-x86_64
Builder is ready to be used

[2025-01-08 17:15:59,106][  INFO][PID:52308] Filling build.info file with builder info
[2025-01-08 17:15:59,115][  INFO][PID:52308] Not re-notifying FE and msg buses for the new host.
[2025-01-08 17:15:59,116][  INFO][PID:52308] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8487458-fedora-40-x86_64 --chroot fedora-40-x86_64 --detached
[2025-01-08 17:15:59,487][  INFO][PID:52308] Downloading the builder-live.log file, attempt 1
[2025-01-08 17:15:59,489][  INFO][PID:52308] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.84.127.42 copr-rpmbuild-log
[2025-01-08 17:16:04,507][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 17:26:04,600][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 17:36:04,674][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 17:46:04,744][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 17:56:04,815][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 18:06:04,937][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 18:16:05,051][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 18:26:05,138][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 18:36:05,215][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 18:46:05,305][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 18:56:05,385][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 19:06:05,467][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 19:16:05,596][  INFO][PID:52308] Periodic builder liveness probe: alive
[2025-01-08 19:17:23,983][  INFO][PID:52308] Cancel request succeeded
out:
success
err:
Warning: Permanently added '98.84.127.42' (ED25519) to the list of known hosts.

[2025-01-08 19:17:23,984][ ERROR][PID:52308] Build was canceled
[2025-01-08 19:17:23,985][  INFO][PID:52308] Releasing VM back to pool
[2025-01-08 19:17:24,010][WARNING][PID:52308] Switching not-finished job state to 'failed'
[2025-01-08 19:17:24,010][  INFO][PID:52308] Worker failed build, took 7284.894488573074
[2025-01-08 19:17:24,011][  INFO][PID:52308] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "churchyard",
            "project_name": "jinja2-3.1.5-f40",
            "project_dirname": "jinja2-3.1.5-f40:custom:isolated",
            "submitter": "churchyard",
            "ended_on": 1736363844.0099752,
            "started_on": 1736356559.1154866,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-40-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "8487458-fedora-40-x86_64",
            "build_id": 8487458,
            "package_name": "python-xbout",
            "package_version": "0.3.6-2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/churchyard/jinja2-3.1.5-f40:custom:isolated/python-xbout",
            "git_hash": "f488f4b4de142836f9446ba597e4861ef1adec34",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "churchyard/jinja2-3.1.5-f40--churchyard",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/churchyard/jinja2-3.1.5-f40/fedora-40-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/f40-build/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_f40_build_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_f40_build_latest_basearch"
                }
            ],
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/churchyard/jinja2-3.1.5-f40:custom:isolated",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/churchyard/jinja2-3.1.5-f40:custom:isolated",
            "result_dir": "08487458-python-xbout",
            "built_packages": "",
            "id": 8487458,
            "mockchain_macros": {
                "copr_username": "churchyard",
                "copr_projectname": "jinja2-3.1.5-f40",
                "vendor": "Fedora Project COPR (churchyard/jinja2-3.1.5-f40)"
            }
        }
    ]
}
[2025-01-08 19:17:24,041][  INFO][PID:52308] Sending fedora-messaging bus message in build.end
[2025-01-08 19:17:24,075][  INFO][PID:52308] Compressing /var/lib/copr/public_html/results/churchyard/jinja2-3.1.5-f40:custom:isolated/fedora-40-x86_64/08487458-python-xbout/builder-live.log by gzip
[2025-01-08 19:17:24,077][  INFO][PID:52308] Running command 'gzip /var/lib/copr/public_html/results/churchyard/jinja2-3.1.5-f40:custom:isolated/fedora-40-x86_64/08487458-python-xbout/builder-live.log' as PID 1281307
[2025-01-08 19:17:24,111][  INFO][PID:52308] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/churchyard/jinja2-3.1.5-f40:custom:isolated/fedora-40-x86_64/08487458-python-xbout/builder-live.log)
[2025-01-08 19:17:24,112][  INFO][PID:52308] Compressing /var/lib/copr/public_html/results/churchyard/jinja2-3.1.5-f40:custom:isolated/fedora-40-x86_64/08487458-python-xbout/backend.log by gzip
[2025-01-08 19:17:24,114][  INFO][PID:52308] Running command 'gzip /var/lib/copr/public_html/results/churchyard/jinja2-3.1.5-f40:custom:isolated/fedora-40-x86_64/08487458-python-xbout/backend.log' as PID 1281310