[2026-01-26 20:29:35,929][  INFO][PID:3082225] Marking build as starting
[2026-01-26 20:29:35,993][  INFO][PID:3082225] Checking that /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/repodata/repomd.xml exists
[2026-01-26 20:29:36,381][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:29:36,381][  INFO][PID:3082225] VM allocation process starts
[2026-01-26 20:29:36,392][  INFO][PID:3082225] Trying to allocate VM: ResallocHost, ticket_id=1033261, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-26 20:29:39,404][  INFO][PID:3082225] Allocated host ResallocHost, ticket_id=1033261, hostname=2620:52:6:1161:dead:beef:cafe:c10c, name=vmhost_p09_02_prod_01616907_20260126_202445, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-26 20:29:39,770][  INFO][PID:3082225] Allocating ssh connection to builder
[2026-01-26 20:29:39,771][  INFO][PID:3082225] Checking that builder machine is OK
[2026-01-26 20:29:40,773][  INFO][PID:3082225] Running remote command: copr-builder-ready fedora-43-ppc64le
[2026-01-26 20:29:41,328][  INFO][PID:3082225] Red Hat subscription not needed for fedora-43-ppc64le
Builder is ready to be used

[2026-01-26 20:29:41,329][  INFO][PID:3082225] Filling build.info file with builder info
[2026-01-26 20:29:41,329][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:29:41,330][  INFO][PID:3082225] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "mjg",
            "project_name": "notmuch-git",
            "project_dirname": "notmuch-git",
            "submitter": "https://api.github.com/users/mjg",
            "ended_on": null,
            "started_on": 1769459381.3305163,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-43-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10064515-fedora-43-ppc64le",
            "build_id": 10064515,
            "package_name": "notmuch",
            "package_version": "0.40~rc1^4.g98590853-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/mjg/notmuch-git/notmuch",
            "git_hash": "08b3dfa4d8d594230ccb9a9239e2d162375572c1",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "mjg/notmuch-git--https://api.github.com/users/mjg",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/mjg/notmuch-git/fedora-43-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/mjg/notmuch-git",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/mjg/notmuch-git",
            "result_dir": "10064515-notmuch",
            "built_packages": "",
            "id": 10064515,
            "mockchain_macros": {
                "copr_username": "mjg",
                "copr_projectname": "notmuch-git",
                "vendor": "Fedora Project COPR (mjg/notmuch-git)"
            }
        }
    ]
}
[2026-01-26 20:29:41,453][  INFO][PID:3082225] Sending fedora-messaging bus message in build.start
[2026-01-26 20:29:41,946][  INFO][PID:3082225] Sending fedora-messaging bus message in chroot.start
[2026-01-26 20:29:41,971][  INFO][PID:3082225] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10064515-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached
[2026-01-26 20:29:42,934][  INFO][PID:3082225] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c10c' (ED25519) to the list of known hosts.


[2026-01-26 20:29:42,935][  INFO][PID:3082225] Downloading the builder-live.log file, attempt 1
[2026-01-26 20:29:42,936][  INFO][PID:3082225] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10c copr-rpmbuild-log
[2026-01-26 20:29:47,942][  INFO][PID:3082225] Periodic builder liveness probe: alive
[2026-01-26 20:29:47,943][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:29:52,944][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:29:57,945][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:02,946][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:07,948][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:12,949][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:17,950][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:22,951][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:27,952][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:32,953][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:37,954][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:42,955][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:47,958][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:52,959][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:30:57,960][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:02,961][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:07,962][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:12,963][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:17,964][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:22,965][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:27,967][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:32,971][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:37,972][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:42,973][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:47,974][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:52,975][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:31:57,976][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:02,977][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:07,978][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:12,979][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:17,982][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:22,983][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:27,984][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:32,985][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:37,986][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:42,987][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:47,988][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:53,012][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:32:58,013][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:03,014][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:08,015][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:13,016][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:18,017][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:23,018][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:28,019][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:33,020][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:38,021][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:43,022][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:48,023][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:53,024][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:33:58,025][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:03,026][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:08,029][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:13,030][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:18,031][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:23,034][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:28,036][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:33,037][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:38,038][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:43,039][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:48,041][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:53,044][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:34:58,045][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:03,046][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:08,047][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:13,048][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:18,049][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:23,050][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:28,051][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:33,052][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:38,053][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:43,054][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:48,055][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:48,370][ ERROR][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-26 20:35:53,056][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:58,058][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:35:58,371][WARNING][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-26 20:36:03,060][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:08,061][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:08,372][  INFO][PID:3082225] Downloading the builder-live.log file, attempt 2
[2026-01-26 20:36:08,373][  INFO][PID:3082225] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10c copr-rpmbuild-log
[2026-01-26 20:36:13,062][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:18,064][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:23,065][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:28,065][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:33,066][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:38,068][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:43,071][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:48,072][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:53,073][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:36:58,074][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:03,075][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:08,076][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:08,413][ ERROR][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-26 20:37:13,077][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:18,078][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:18,417][WARNING][PID:3082225] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-26 20:37:23,080][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:28,081][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:28,417][  INFO][PID:3082225] Downloading the builder-live.log file, attempt 3
[2026-01-26 20:37:28,419][  INFO][PID:3082225] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10c copr-rpmbuild-log
[2026-01-26 20:37:33,082][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:38,083][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:43,084][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:48,085][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:53,086][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:37:58,087][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:03,088][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:08,090][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:13,091][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:18,092][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:23,093][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:28,094][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:28,486][ ERROR][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-26 20:38:33,095][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:38,096][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:38,492][WARNING][PID:3082225] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-26 20:38:43,097][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:48,098][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:48,493][  INFO][PID:3082225] Downloading the builder-live.log file, attempt 4
[2026-01-26 20:38:48,494][  INFO][PID:3082225] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10c copr-rpmbuild-log
[2026-01-26 20:38:53,099][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:38:58,100][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:03,101][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:08,102][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:13,103][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:18,104][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:23,105][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:28,105][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:33,106][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:38,107][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:43,108][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:48,161][  INFO][PID:3082225] Periodic builder liveness probe: alive
[2026-01-26 20:39:48,162][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:48,561][ ERROR][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-26 20:39:53,162][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:58,163][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:58,577][ ERROR][PID:3082225] Giving up for unstable SSH, failures: 1769459758.3712144, 1769459838.417121, 1769459918.4929185, 1769459998.5776336
[2026-01-26 20:39:58,578][ ERROR][PID:3082225] Re-try request for task on 'ResallocHost, ticket_id=1033261, hostname=2620:52:6:1161:dead:beef:cafe:c10c, name=vmhost_p09_02_prod_01616907_20260126_202445, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-01-26 20:39:58,578][  INFO][PID:3082225] Releasing VM back to pool
[2026-01-26 20:39:58,587][  INFO][PID:3082225] Retry #1 (on other host)
[2026-01-26 20:39:58,587][  INFO][PID:3082225] Checking that /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/repodata/repomd.xml exists
[2026-01-26 20:39:58,588][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:39:58,588][  INFO][PID:3082225] VM allocation process starts
[2026-01-26 20:39:58,634][  INFO][PID:3082225] Trying to allocate VM: ResallocHost, ticket_id=1033465, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-26 20:40:01,683][  INFO][PID:3082225] Allocated host ResallocHost, ticket_id=1033465, hostname=2620:52:6:1161:dead:beef:cafe:c190, name=vmhost_p09_04_prod_01616908_20260126_202447, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-01-26 20:40:01,683][  INFO][PID:3082225] Allocating ssh connection to builder
[2026-01-26 20:40:01,684][  INFO][PID:3082225] Checking that builder machine is OK
[2026-01-26 20:40:02,910][  INFO][PID:3082225] Running remote command: copr-builder-ready fedora-43-ppc64le
[2026-01-26 20:40:03,494][  INFO][PID:3082225] Red Hat subscription not needed for fedora-43-ppc64le
Builder is ready to be used

[2026-01-26 20:40:03,498][  INFO][PID:3082225] Filling build.info file with builder info
[2026-01-26 20:40:03,508][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:03,510][  INFO][PID:3082225] Not re-notifying FE and msg buses for the new host.
[2026-01-26 20:40:03,510][  INFO][PID:3082225] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10064515-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached
[2026-01-26 20:40:04,882][  INFO][PID:3082225] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c190' (ED25519) to the list of known hosts.


[2026-01-26 20:40:04,883][  INFO][PID:3082225] Downloading the builder-live.log file, attempt 1
[2026-01-26 20:40:04,884][  INFO][PID:3082225] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c190 copr-rpmbuild-log
[2026-01-26 20:40:09,883][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:14,884][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:19,885][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:24,886][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:29,887][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:34,888][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:39,889][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:44,890][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:49,891][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:54,892][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:40:59,893][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:04,894][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:04,951][ ERROR][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-26 20:41:09,895][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:14,896][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:14,968][WARNING][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-26 20:41:19,897][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:24,898][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:24,969][  INFO][PID:3082225] Downloading the builder-live.log file, attempt 2
[2026-01-26 20:41:24,970][  INFO][PID:3082225] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c190 copr-rpmbuild-log
[2026-01-26 20:41:29,899][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:34,900][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:39,901][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:44,902][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:49,903][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:54,904][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:41:59,905][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:04,906][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:09,907][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:14,908][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:19,909][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:24,910][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:25,037][ ERROR][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-26 20:42:29,911][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:34,912][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:35,044][WARNING][PID:3082225] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-26 20:42:39,913][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:44,914][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:45,045][  INFO][PID:3082225] Downloading the builder-live.log file, attempt 3
[2026-01-26 20:42:45,047][  INFO][PID:3082225] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c190 copr-rpmbuild-log
[2026-01-26 20:42:49,915][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:54,916][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:42:59,917][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:04,918][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:09,919][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:14,920][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:19,921][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:24,922][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:29,923][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:34,924][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:39,924][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:44,925][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:45,114][ ERROR][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-26 20:43:49,927][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:54,927][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:43:55,125][WARNING][PID:3082225] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-26 20:43:59,928][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:04,929][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:05,126][  INFO][PID:3082225] Downloading the builder-live.log file, attempt 4
[2026-01-26 20:44:05,127][  INFO][PID:3082225] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c190 copr-rpmbuild-log
[2026-01-26 20:44:09,930][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:14,931][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:19,932][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:24,933][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:29,934][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:34,935][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:39,936][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:44,937][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:49,938][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:54,939][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:44:59,940][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:04,941][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:09,942][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:14,943][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:19,944][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:24,945][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:29,945][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:34,946][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:39,947][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:44,948][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:49,949][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:54,950][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:45:59,951][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:04,952][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:09,953][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:14,954][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:19,955][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:24,956][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:29,957][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:34,958][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:39,959][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:44,960][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:49,961][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:54,962][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:46:59,963][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:04,964][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:09,965][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:14,967][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:19,968][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:24,969][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:29,970][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:34,971][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:39,972][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:44,973][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:49,974][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:54,975][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:47:59,976][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:04,977][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:09,978][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:14,979][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:19,980][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:24,981][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:29,982][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:34,984][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:39,985][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:44,986][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:49,987][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:54,989][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:48:59,990][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:04,991][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:09,992][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:14,993][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:19,994][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:24,995][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:29,996][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:34,997][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:40,001][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:45,002][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:50,009][  INFO][PID:3082225] Periodic builder liveness probe: alive
[2026-01-26 20:49:50,010][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:49:55,011][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:00,012][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:05,013][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:10,014][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:15,015][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:20,016][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:25,017][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:30,018][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:35,019][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:40,020][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:45,021][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:50,022][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:50:55,023][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:00,023][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:05,024][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:10,025][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:15,026][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:20,027][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:25,028][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:30,029][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:35,030][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:40,031][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:45,032][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:50,033][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:51:55,034][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:00,035][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:05,036][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:10,037][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:15,038][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:20,039][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:25,040][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:30,041][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:35,042][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:40,042][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:45,043][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:50,044][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:52:55,045][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:00,046][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:05,047][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:10,048][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:15,051][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:20,052][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:25,053][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:30,054][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:35,055][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:40,056][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:45,057][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:50,058][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:53:55,059][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:00,060][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:05,061][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:10,062][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:15,063][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:20,064][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:25,065][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:30,066][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:35,067][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:40,068][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:45,069][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:50,070][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:54:55,071][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:00,072][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:05,073][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:10,074][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:15,075][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:20,076][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:25,078][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:30,079][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:35,080][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:40,081][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:45,082][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:50,083][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:55:55,084][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:00,085][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:05,086][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:10,087][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:15,088][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:20,089][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:25,091][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:30,092][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:35,093][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:40,094][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:45,095][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:50,096][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:56:55,097][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:00,101][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:05,103][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:10,105][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:15,107][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:20,108][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:25,109][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:30,110][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:35,111][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:40,112][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:45,113][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:50,117][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:57:55,119][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:00,120][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:05,121][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:10,122][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:15,124][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:20,126][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:25,127][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:30,128][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:35,129][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:40,130][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:45,131][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:50,132][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:58:55,794][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:00,796][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:05,797][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:10,798][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:15,799][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:20,800][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:25,801][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:30,802][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:35,803][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:40,805][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:45,807][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:50,813][  INFO][PID:3082225] Periodic builder liveness probe: alive
[2026-01-26 20:59:50,814][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 20:59:55,814][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:00,815][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:05,816][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:10,817][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:15,818][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:20,820][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:25,822][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:30,823][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:35,825][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:40,826][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:45,827][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:50,828][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:00:55,829][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:01:00,830][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:01:05,831][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:01:10,832][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:01:15,833][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:01:20,834][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:01:25,835][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:01:30,836][  INFO][PID:3082225] Checking for cancel request
[2026-01-26 21:01:33,616][  INFO][PID:3082225] Downloading results from builder
[2026-01-26 21:01:33,616][  INFO][PID:3082225] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c190]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch started
[2026-01-26 21:01:33,617][  INFO][PID:3082225] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c190]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/ &> /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/build-10064515.rsync.log
[2026-01-26 21:02:33,719][  INFO][PID:3082225] rsyncing finished.
[2026-01-26 21:02:33,720][ ERROR][PID:3082225] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-26 21:02:33,720][ ERROR][PID:3082225] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-26 21:02:43,721][  INFO][PID:3082225] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c190]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch started
[2026-01-26 21:02:43,722][  INFO][PID:3082225] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c190]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/ &> /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/build-10064515.rsync.log
[2026-01-26 21:03:43,804][  INFO][PID:3082225] rsyncing finished.
[2026-01-26 21:03:43,804][ ERROR][PID:3082225] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-26 21:03:43,805][ ERROR][PID:3082225] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-01-26 21:03:53,805][  INFO][PID:3082225] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c190]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch started
[2026-01-26 21:03:53,806][  INFO][PID:3082225] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c190]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/ &> /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/build-10064515.rsync.log
[2026-01-26 21:03:56,039][  INFO][PID:3082225] rsyncing finished.
[2026-01-26 21:03:56,866][  INFO][PID:3082225] Releasing VM back to pool
[2026-01-26 21:03:56,877][  INFO][PID:3082225] Searching for 'success' file in resultdir
[2026-01-26 21:03:56,878][  INFO][PID:3082225] Going to sign pkgs from source: 10064515-fedora-43-ppc64le in chroot: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le
[2026-01-26 21:03:56,902][  INFO][PID:3082225] Calling '/bin/sign -u mjg#notmuch-git@copr.fedorahosted.org -p' (attempt #1)
[2026-01-26 21:03:57,176][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-vim-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:03:57,894][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-vim-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:03:57,895][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/emacs-notmuch-0.40~rc1^4.g98590853-1.fc43.noarch.rpm' (attempt #1)
[2026-01-26 21:03:58,644][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/emacs-notmuch-0.40~rc1^4.g98590853-1.fc43.noarch.rpm
[2026-01-26 21:03:58,645][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/ruby-notmuch-debuginfo-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:03:59,361][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/ruby-notmuch-debuginfo-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:03:59,361][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/python3-notmuch2-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:04:00,097][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/python3-notmuch2-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:04:00,097][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-0.40~rc1^4.g98590853-1.fc43.src.rpm' (attempt #1)
[2026-01-26 21:04:00,889][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-0.40~rc1^4.g98590853-1.fc43.src.rpm
[2026-01-26 21:04:00,889][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-devel-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:04:01,789][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-devel-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:04:01,790][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/ruby-notmuch-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:04:02,510][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/ruby-notmuch-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:04:02,510][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:04:03,280][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:04:03,280][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-git-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:04:04,005][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-git-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:04:04,006][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-debugsource-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:04:04,750][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-debugsource-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:04:04,750][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-mutt-0.40~rc1^4.g98590853-1.fc43.noarch.rpm' (attempt #1)
[2026-01-26 21:04:05,465][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-mutt-0.40~rc1^4.g98590853-1.fc43.noarch.rpm
[2026-01-26 21:04:05,465][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-debuginfo-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:04:07,052][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/notmuch-debuginfo-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:04:07,052][  INFO][PID:3082225] Calling '/bin/sign -4 -h sha256 -u mjg#notmuch-git@copr.fedorahosted.org -r /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/python3-notmuch2-debuginfo-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm' (attempt #1)
[2026-01-26 21:04:07,688][  INFO][PID:3082225] signed rpm: /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/python3-notmuch2-debuginfo-0.40~rc1^4.g98590853-1.fc43.ppc64le.rpm
[2026-01-26 21:04:07,689][  INFO][PID:3082225] Sign done
[2026-01-26 21:04:07,690][  INFO][PID:3082225] Incremental createrepo run, adding 10064515-notmuch into https://download.copr.fedorainfracloud.org/results/mjg/notmuch-git/fedora-43-ppc64le, (auto-create-repo=True)
[2026-01-26 21:04:07,692][  INFO][PID:3082225] Running command 'copr-repo --batched /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le --add 10064515-notmuch --no-appstream-metadata' as PID 3240296
[2026-01-26 21:04:08,641][  INFO][PID:3082225] Finished after 0 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le --add 10064515-notmuch --no-appstream-metadata)
[2026-01-26 21:04:08,642][  INFO][PID:3082225] Getting build details
[2026-01-26 21:04:08,642][  INFO][PID:3082225] Listing built binary packages in /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch
[2026-01-26 21:04:08,643][  INFO][PID:3082225] Built packages:
['notmuch-mutt 0.40~rc1^4.g98590853', 'notmuch 0.40~rc1^4.g98590853', 'notmuch-debugsource 0.40~rc1^4.g98590853', 'python3-notmuch2-debuginfo 0.40~rc1^4.g98590853', 'python3-notmuch2 0.40~rc1^4.g98590853', 'notmuch-debuginfo 0.40~rc1^4.g98590853', 'notmuch-vim 0.40~rc1^4.g98590853', 'ruby-notmuch 0.40~rc1^4.g98590853', 'notmuch-devel 0.40~rc1^4.g98590853', 'ruby-notmuch-debuginfo 0.40~rc1^4.g98590853', 'notmuch-git 0.40~rc1^4.g98590853', 'emacs-notmuch 0.40~rc1^4.g98590853']
[2026-01-26 21:04:08,643][  INFO][PID:3082225] build details: {'built_packages': 'notmuch-mutt 0.40~rc1^4.g98590853\nnotmuch 0.40~rc1^4.g98590853\nnotmuch-debugsource 0.40~rc1^4.g98590853\npython3-notmuch2-debuginfo 0.40~rc1^4.g98590853\npython3-notmuch2 0.40~rc1^4.g98590853\nnotmuch-debuginfo 0.40~rc1^4.g98590853\nnotmuch-vim 0.40~rc1^4.g98590853\nruby-notmuch 0.40~rc1^4.g98590853\nnotmuch-devel 0.40~rc1^4.g98590853\nruby-notmuch-debuginfo 0.40~rc1^4.g98590853\nnotmuch-git 0.40~rc1^4.g98590853\nemacs-notmuch 0.40~rc1^4.g98590853'}
[2026-01-26 21:04:08,644][  INFO][PID:3082225] Retrieving pubkey
[2026-01-26 21:04:08,644][  INFO][PID:3082225] Calling '/bin/sign -u mjg#notmuch-git@copr.fedorahosted.org -p' (attempt #1)
[2026-01-26 21:04:08,828][  INFO][PID:3082225] Added pubkey for user mjg project notmuch-git into: /var/lib/copr/public_html/results/mjg/notmuch-git/pubkey.gpg
[2026-01-26 21:04:08,829][  INFO][PID:3082225] Finished build: id=10064515 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/mjg/notmuch-git chroot=fedora-43-ppc64le 
[2026-01-26 21:04:08,830][  INFO][PID:3082225] Worker succeeded build, took 1445.3206813335419
[2026-01-26 21:04:08,831][  INFO][PID:3082225] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "mjg",
            "project_name": "notmuch-git",
            "project_dirname": "notmuch-git",
            "submitter": "https://api.github.com/users/mjg",
            "ended_on": 1769461448.830677,
            "started_on": 1769460003.5099957,
            "submitted_on": null,
            "status": 1,
            "chroot": "fedora-43-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10064515-fedora-43-ppc64le",
            "build_id": 10064515,
            "package_name": "notmuch",
            "package_version": "0.40~rc1^4.g98590853-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/mjg/notmuch-git/notmuch",
            "git_hash": "08b3dfa4d8d594230ccb9a9239e2d162375572c1",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "mjg/notmuch-git--https://api.github.com/users/mjg",
            "results": {
                "packages": [
                    {
                        "name": "notmuch-mutt",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "noarch"
                    },
                    {
                        "name": "notmuch",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "notmuch-debugsource",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "python3-notmuch2-debuginfo",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "notmuch",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "src"
                    },
                    {
                        "name": "python3-notmuch2",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "notmuch-debuginfo",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "notmuch-vim",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "ruby-notmuch",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "notmuch-devel",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "ruby-notmuch-debuginfo",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "notmuch-git",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "ppc64le"
                    },
                    {
                        "name": "emacs-notmuch",
                        "epoch": null,
                        "version": "0.40~rc1^4.g98590853",
                        "release": "1.fc43",
                        "arch": "noarch"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/mjg/notmuch-git/fedora-43-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/mjg/notmuch-git",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/mjg/notmuch-git",
            "result_dir": "10064515-notmuch",
            "built_packages": "notmuch-mutt 0.40~rc1^4.g98590853\nnotmuch 0.40~rc1^4.g98590853\nnotmuch-debugsource 0.40~rc1^4.g98590853\npython3-notmuch2-debuginfo 0.40~rc1^4.g98590853\npython3-notmuch2 0.40~rc1^4.g98590853\nnotmuch-debuginfo 0.40~rc1^4.g98590853\nnotmuch-vim 0.40~rc1^4.g98590853\nruby-notmuch 0.40~rc1^4.g98590853\nnotmuch-devel 0.40~rc1^4.g98590853\nruby-notmuch-debuginfo 0.40~rc1^4.g98590853\nnotmuch-git 0.40~rc1^4.g98590853\nemacs-notmuch 0.40~rc1^4.g98590853",
            "id": 10064515,
            "mockchain_macros": {
                "copr_username": "mjg",
                "copr_projectname": "notmuch-git",
                "vendor": "Fedora Project COPR (mjg/notmuch-git)"
            }
        }
    ]
}
[2026-01-26 21:04:09,012][  INFO][PID:3082225] Sending fedora-messaging bus message in build.end
[2026-01-26 21:04:09,041][  INFO][PID:3082225] Compressing /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/builder-live.log by gzip
[2026-01-26 21:04:09,043][  INFO][PID:3082225] Running command 'gzip /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/builder-live.log' as PID 3240491
[2026-01-26 21:04:09,266][  INFO][PID:3082225] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/builder-live.log)
[2026-01-26 21:04:09,269][  INFO][PID:3082225] Compressing /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/backend.log by gzip
[2026-01-26 21:04:09,282][  INFO][PID:3082225] Running command 'gzip /var/lib/copr/public_html/results/mjg/notmuch-git/fedora-43-ppc64le/10064515-notmuch/backend.log' as PID 3240547