[2025-12-24 10:15:51,041][  INFO][PID:930633] Marking build as starting
[2025-12-24 10:15:51,184][  INFO][PID:930633] Checking that /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/repodata/repomd.xml exists
[2025-12-24 10:15:51,185][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:15:51,185][  INFO][PID:930633] VM allocation process starts
[2025-12-24 10:15:51,194][  INFO][PID:930633] Trying to allocate VM: ResallocHost, ticket_id=493624, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-24 10:15:54,249][  INFO][PID:930633] Allocated host ResallocHost, ticket_id=493624, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_00748295_20251224_095209, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-24 10:15:54,249][  INFO][PID:930633] Allocating ssh connection to builder
[2025-12-24 10:15:54,249][  INFO][PID:930633] Checking that builder machine is OK
[2025-12-24 10:15:55,135][  INFO][PID:930633] Running remote command: copr-builder-ready epel-10-ppc64le
[2025-12-24 10:15:55,617][  INFO][PID:930633] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-12-24 10:15:55,617][  INFO][PID:930633] Filling build.info file with builder info
[2025-12-24 10:15:55,618][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:15:55,618][  INFO][PID:930633] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "celenity",
            "project_name": "copr",
            "project_dirname": "copr",
            "submitter": "celenity",
            "ended_on": null,
            "started_on": 1766571355.6187632,
            "submitted_on": null,
            "status": 3,
            "chroot": "epel-10-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9947656-epel-10-ppc64le",
            "build_id": 9947656,
            "package_name": "dove-flatpak",
            "package_version": "202512231-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/celenity/copr/dove-flatpak",
            "git_hash": "08f842eed81447b3229424e40e9d95088380fbaa",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "celenity/copr--celenity",
            "results": null,
            "appstream": true,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/celenity/copr/epel-10-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/celenity/copr",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/celenity/copr",
            "result_dir": "09947656-dove-flatpak",
            "built_packages": "",
            "id": 9947656,
            "mockchain_macros": {
                "copr_username": "celenity",
                "copr_projectname": "copr",
                "vendor": "Fedora Project COPR (celenity/copr)"
            }
        }
    ]
}
[2025-12-24 10:15:55,765][  INFO][PID:930633] Sending fedora-messaging bus message in build.start
[2025-12-24 10:15:56,260][  INFO][PID:930633] Sending fedora-messaging bus message in chroot.start
[2025-12-24 10:15:56,287][  INFO][PID:930633] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9947656-epel-10-ppc64le --chroot epel-10-ppc64le --detached
[2025-12-24 10:16:56,355][ ERROR][PID:930633] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c14d port 22: Connection timed out

[2025-12-24 10:17:06,355][ ERROR][PID:930633] Re-try request for task on 'ResallocHost, ticket_id=493624, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_00748295_20251224_095209, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']': Unable to finish after 1 SSH attempts
[2025-12-24 10:17:06,356][  INFO][PID:930633] Releasing VM back to pool
[2025-12-24 10:17:06,364][  INFO][PID:930633] Retry #1 (on other host)
[2025-12-24 10:17:06,364][  INFO][PID:930633] Checking that /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/repodata/repomd.xml exists
[2025-12-24 10:17:06,365][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:06,365][  INFO][PID:930633] VM allocation process starts
[2025-12-24 10:17:06,399][  INFO][PID:930633] Trying to allocate VM: ResallocHost, ticket_id=493630, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-24 10:17:09,462][  INFO][PID:930633] Allocated host ResallocHost, ticket_id=493630, hostname=2620:52:6:1161:dead:beef:cafe:c18d, name=vmhost_p09_04_prod_00748298_20251224_095237, requested_tags=['copr_builder', 'arch_ppc64le', 'arch_power9']
[2025-12-24 10:17:09,463][  INFO][PID:930633] Allocating ssh connection to builder
[2025-12-24 10:17:09,463][  INFO][PID:930633] Checking that builder machine is OK
[2025-12-24 10:17:10,388][  INFO][PID:930633] Running remote command: copr-builder-ready epel-10-ppc64le
[2025-12-24 10:17:10,936][  INFO][PID:930633] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-12-24 10:17:10,937][  INFO][PID:930633] Filling build.info file with builder info
[2025-12-24 10:17:10,950][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:10,951][  INFO][PID:930633] Not re-notifying FE and msg buses for the new host.
[2025-12-24 10:17:10,951][  INFO][PID:930633] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9947656-epel-10-ppc64le --chroot epel-10-ppc64le --detached
[2025-12-24 10:17:11,898][  INFO][PID:930633] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18d' (ED25519) to the list of known hosts.


[2025-12-24 10:17:11,899][  INFO][PID:930633] Downloading the builder-live.log file, attempt 1
[2025-12-24 10:17:11,900][  INFO][PID:930633] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18d copr-rpmbuild-log
[2025-12-24 10:17:16,904][  INFO][PID:930633] Periodic builder liveness probe: alive
[2025-12-24 10:17:16,904][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:21,905][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:26,906][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:31,907][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:36,908][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:41,909][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:46,910][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:51,911][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:17:56,912][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:01,913][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:06,914][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:11,915][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:11,967][ ERROR][PID:930633] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-24 10:18:16,916][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:21,916][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:21,967][WARNING][PID:930633] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-24 10:18:26,917][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:31,918][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:31,968][  INFO][PID:930633] Downloading the builder-live.log file, attempt 2
[2025-12-24 10:18:31,969][  INFO][PID:930633] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18d copr-rpmbuild-log
[2025-12-24 10:18:36,919][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:41,920][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:46,921][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:51,922][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:18:56,923][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:01,924][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:06,925][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:11,926][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:16,927][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:21,928][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:26,929][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:31,930][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:32,027][ ERROR][PID:930633] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-24 10:19:36,931][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:41,932][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:42,035][WARNING][PID:930633] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-24 10:19:46,933][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:51,934][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:19:52,036][  INFO][PID:930633] Downloading the builder-live.log file, attempt 3
[2025-12-24 10:19:52,037][  INFO][PID:930633] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18d copr-rpmbuild-log
[2025-12-24 10:19:56,934][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:01,935][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:06,936][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:11,937][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:16,938][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:21,939][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:26,940][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:31,941][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:36,942][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:41,943][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:46,944][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:51,945][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:20:56,946][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:01,947][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:06,948][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:11,949][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:16,950][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:21,951][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:26,952][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:31,953][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:36,954][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:41,955][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:46,956][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:51,956][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:21:56,957][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:01,958][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:06,959][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:11,960][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:16,961][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:21,962][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:26,963][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:31,964][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:36,965][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:41,966][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:46,967][  INFO][PID:930633] Checking for cancel request
[2025-12-24 10:22:50,888][  INFO][PID:930633] Downloading results from builder
[2025-12-24 10:22:50,889][  INFO][PID:930633] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18d]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/09947656-dove-flatpak started
[2025-12-24 10:22:50,890][  INFO][PID:930633] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c18d]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/09947656-dove-flatpak/ &> /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/09947656-dove-flatpak/build-09947656.rsync.log
[2025-12-24 10:22:51,503][  INFO][PID:930633] rsyncing finished.
[2025-12-24 10:22:51,503][  INFO][PID:930633] Releasing VM back to pool
[2025-12-24 10:22:51,512][  INFO][PID:930633] Searching for 'success' file in resultdir
[2025-12-24 10:22:51,512][ ERROR][PID:930633] Build failed: Backend process error: No success file => build failure
[2025-12-24 10:22:51,513][  INFO][PID:930633] Finished build: id=9947656 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/celenity/copr chroot=epel-10-ppc64le 
[2025-12-24 10:22:51,514][ ERROR][PID:930633] Backend process error: No success file => build failure
[2025-12-24 10:22:51,515][  INFO][PID:930633] Worker failed build, took 340.5641520023346
[2025-12-24 10:22:51,515][  INFO][PID:930633] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "celenity",
            "project_name": "copr",
            "project_dirname": "copr",
            "submitter": "celenity",
            "ended_on": 1766571771.5152197,
            "started_on": 1766571430.9510677,
            "submitted_on": null,
            "status": 0,
            "chroot": "epel-10-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9947656-epel-10-ppc64le",
            "build_id": 9947656,
            "package_name": "dove-flatpak",
            "package_version": "202512231-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/celenity/copr/dove-flatpak",
            "git_hash": "08f842eed81447b3229424e40e9d95088380fbaa",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "celenity/copr--celenity",
            "results": null,
            "appstream": true,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/celenity/copr/epel-10-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le",
                "arch_power9"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/celenity/copr",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/celenity/copr",
            "result_dir": "09947656-dove-flatpak",
            "built_packages": "",
            "id": 9947656,
            "mockchain_macros": {
                "copr_username": "celenity",
                "copr_projectname": "copr",
                "vendor": "Fedora Project COPR (celenity/copr)"
            }
        }
    ]
}
[2025-12-24 10:22:51,660][  INFO][PID:930633] Sending fedora-messaging bus message in build.end
[2025-12-24 10:22:51,690][  INFO][PID:930633] Compressing /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/09947656-dove-flatpak/builder-live.log by gzip
[2025-12-24 10:22:51,691][  INFO][PID:930633] Running command 'gzip /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/09947656-dove-flatpak/builder-live.log' as PID 951830
[2025-12-24 10:22:51,698][  INFO][PID:930633] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/09947656-dove-flatpak/builder-live.log)
[2025-12-24 10:22:51,699][  INFO][PID:930633] Compressing /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/09947656-dove-flatpak/backend.log by gzip
[2025-12-24 10:22:51,700][  INFO][PID:930633] Running command 'gzip /var/lib/copr/public_html/results/celenity/copr/epel-10-ppc64le/09947656-dove-flatpak/backend.log' as PID 951833