[2026-02-10 12:55:11,432][  INFO][PID:4014982] Marking build as starting
[2026-02-10 12:55:11,486][  INFO][PID:4014982] Checking that /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/repodata/repomd.xml exists
[2026-02-10 12:55:11,501][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:11,502][  INFO][PID:4014982] VM allocation process starts
[2026-02-10 12:55:11,514][  INFO][PID:4014982] Trying to allocate VM: ResallocHost, ticket_id=1255961, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 12:55:14,532][  INFO][PID:4014982] Allocated host ResallocHost, ticket_id=1255961, hostname=2620:52:6:1161:dead:beef:cafe:c192, name=vmhost_p09_04_prod_02124217_20260210_122115, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 12:55:14,533][  INFO][PID:4014982] Allocating ssh connection to builder
[2026-02-10 12:55:14,533][  INFO][PID:4014982] Checking that builder machine is OK
[2026-02-10 12:55:15,527][  INFO][PID:4014982] Running remote command: copr-builder-ready fedora-43-ppc64le
[2026-02-10 12:55:16,089][  INFO][PID:4014982] Red Hat subscription not needed for fedora-43-ppc64le
Builder is ready to be used

[2026-02-10 12:55:16,089][  INFO][PID:4014982] Filling build.info file with builder info
[2026-02-10 12:55:16,090][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:16,091][  INFO][PID:4014982] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "jonny",
            "project_name": "BitcoinCash",
            "project_dirname": "BitcoinCash",
            "submitter": "https://api.github.com/users/cculianu",
            "ended_on": null,
            "started_on": 1770728116.0911758,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-43-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10112128-fedora-43-ppc64le",
            "build_id": 10112128,
            "package_name": "fulcrum",
            "package_version": "2.1.0-0.0.git.1536.a517cbce",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/jonny/BitcoinCash/fulcrum",
            "git_hash": "ae4b0e076182f258549caaaf0b54c146708fcd6a",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "jonny/BitcoinCash--https://api.github.com/users/cculianu",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/jonny/BitcoinCash/fedora-43-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": true,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/jonny/BitcoinCash",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/jonny/BitcoinCash",
            "result_dir": "10112128-fulcrum",
            "built_packages": "",
            "id": 10112128,
            "mockchain_macros": {
                "copr_username": "jonny",
                "copr_projectname": "BitcoinCash",
                "vendor": "Fedora Project COPR (jonny/BitcoinCash)"
            }
        }
    ]
}
[2026-02-10 12:55:16,152][  INFO][PID:4014982] Sending fedora-messaging bus message in build.start
[2026-02-10 12:55:16,744][  INFO][PID:4014982] Sending fedora-messaging bus message in chroot.start
[2026-02-10 12:55:16,773][  INFO][PID:4014982] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10112128-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached
[2026-02-10 12:55:17,867][  INFO][PID:4014982] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c192' (ED25519) to the list of known hosts.


[2026-02-10 12:55:17,868][  INFO][PID:4014982] Downloading the builder-live.log file, attempt 1
[2026-02-10 12:55:17,869][  INFO][PID:4014982] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c192 copr-rpmbuild-log
[2026-02-10 12:55:22,875][  INFO][PID:4014982] Periodic builder liveness probe: alive
[2026-02-10 12:55:22,876][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:27,877][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:32,878][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:37,880][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:42,882][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:47,883][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:52,884][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:55:57,885][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:02,887][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:07,888][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:12,892][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:17,894][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:17,937][ ERROR][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 12:56:22,895][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:27,897][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:27,938][WARNING][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-02-10 12:56:32,899][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:37,901][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:37,940][  INFO][PID:4014982] Downloading the builder-live.log file, attempt 2
[2026-02-10 12:56:37,941][  INFO][PID:4014982] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c192 copr-rpmbuild-log
[2026-02-10 12:56:42,902][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:47,903][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:52,904][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:56:57,910][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:02,912][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:07,913][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:12,914][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:17,915][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:22,916][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:27,923][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:32,925][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:37,926][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:38,010][ ERROR][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 12:57:42,927][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:47,928][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:48,020][WARNING][PID:4014982] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-02-10 12:57:52,929][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:57,934][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:57:58,020][  INFO][PID:4014982] Downloading the builder-live.log file, attempt 3
[2026-02-10 12:57:58,022][  INFO][PID:4014982] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c192 copr-rpmbuild-log
[2026-02-10 12:58:02,935][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:07,939][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:12,940][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:17,943][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:22,944][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:27,945][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:32,946][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:37,947][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:42,948][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:47,949][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:52,951][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:58:57,952][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:02,953][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:07,954][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:12,955][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:17,956][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:22,957][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:27,958][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:32,960][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:37,962][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:42,963][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:47,978][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:52,980][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 12:59:57,982][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:02,987][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:07,988][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:12,990][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:17,991][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:22,992][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:27,993][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:32,994][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:37,995][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:42,996][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:47,997][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:52,999][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:00:58,000][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:03,001][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:08,002][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:13,003][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:18,004][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:23,006][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:28,006][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:33,007][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:38,008][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:43,009][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:48,011][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:53,012][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:01:58,013][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:03,014][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:08,015][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:13,018][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:18,020][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:23,022][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:28,023][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:33,024][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:38,045][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:43,046][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:48,071][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:53,072][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:02:58,073][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:03,074][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:08,075][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:13,077][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:18,078][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:23,079][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:28,081][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:33,082][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:38,102][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:43,103][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:48,105][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:53,106][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:03:58,107][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:03,108][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:08,110][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:13,111][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:18,112][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:23,112][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:28,114][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:33,115][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:38,116][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:43,117][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:48,118][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:53,119][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:04:58,120][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:03,140][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:08,141][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:13,142][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:18,143][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:23,148][  INFO][PID:4014982] Periodic builder liveness probe: alive
[2026-02-10 13:05:23,149][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:28,150][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:33,151][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:38,152][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:43,152][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:48,153][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:53,155][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:05:58,156][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:03,157][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:08,158][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:13,159][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:18,160][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:23,161][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:28,162][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:33,163][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:38,164][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:43,165][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:06:43,916][  INFO][PID:4014982] Downloading results from builder
[2026-02-10 13:06:43,917][  INFO][PID:4014982] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c192]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum started
[2026-02-10 13:06:43,918][  INFO][PID:4014982] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c192]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/ &> /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/build-10112128.rsync.log
[2026-02-10 13:07:44,060][  INFO][PID:4014982] rsyncing finished.
[2026-02-10 13:07:44,060][ ERROR][PID:4014982] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 13:07:44,061][ ERROR][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 13:07:54,062][  INFO][PID:4014982] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c192]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum started
[2026-02-10 13:07:54,063][  INFO][PID:4014982] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c192]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/ &> /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/build-10112128.rsync.log
[2026-02-10 13:08:54,161][  INFO][PID:4014982] rsyncing finished.
[2026-02-10 13:08:54,222][ ERROR][PID:4014982] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 13:08:54,226][ ERROR][PID:4014982] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 13:09:04,228][  INFO][PID:4014982] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c192]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum started
[2026-02-10 13:09:04,234][  INFO][PID:4014982] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c192]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/ &> /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/build-10112128.rsync.log
[2026-02-10 13:10:04,324][  INFO][PID:4014982] rsyncing finished.
[2026-02-10 13:10:04,367][ ERROR][PID:4014982] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 13:10:04,368][ ERROR][PID:4014982] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2026-02-10 13:10:14,369][ ERROR][PID:4014982] Re-try request for task on 'ResallocHost, ticket_id=1255961, hostname=2620:52:6:1161:dead:beef:cafe:c192, name=vmhost_p09_04_prod_02124217_20260210_122115, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 3 SSH attempts
[2026-02-10 13:10:14,370][  INFO][PID:4014982] Releasing VM back to pool
[2026-02-10 13:10:14,419][  INFO][PID:4014982] Retry #1 (on other host)
[2026-02-10 13:10:14,422][  INFO][PID:4014982] Checking that /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/repodata/repomd.xml exists
[2026-02-10 13:10:14,534][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:10:14,537][  INFO][PID:4014982] VM allocation process starts
[2026-02-10 13:10:14,580][  INFO][PID:4014982] Trying to allocate VM: ResallocHost, ticket_id=1256191, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 13:10:19,584][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:10:20,692][  INFO][PID:4014982] Allocated host ResallocHost, ticket_id=1256191, hostname=2620:52:6:1161:dead:beef:cafe:c18b, name=vmhost_p09_04_prod_02125305_20260210_123737, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 13:10:20,699][  INFO][PID:4014982] Allocating ssh connection to builder
[2026-02-10 13:10:20,701][  INFO][PID:4014982] Checking that builder machine is OK
[2026-02-10 13:11:20,800][ ERROR][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18b port 22: Connection timed out

[2026-02-10 13:11:30,891][ ERROR][PID:4014982] Re-try request for task on 'ResallocHost, ticket_id=1256191, hostname=2620:52:6:1161:dead:beef:cafe:c18b, name=vmhost_p09_04_prod_02125305_20260210_123737, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-02-10 13:11:30,933][  INFO][PID:4014982] Releasing VM back to pool
[2026-02-10 13:11:30,966][  INFO][PID:4014982] Retry #2 (on other host)
[2026-02-10 13:11:30,970][  INFO][PID:4014982] Checking that /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/repodata/repomd.xml exists
[2026-02-10 13:11:31,168][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:11:31,171][  INFO][PID:4014982] VM allocation process starts
[2026-02-10 13:11:31,253][  INFO][PID:4014982] Trying to allocate VM: ResallocHost, ticket_id=1256215, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 13:11:36,265][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:11:41,281][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:11:43,475][  INFO][PID:4014982] Allocated host ResallocHost, ticket_id=1256215, hostname=2620:52:6:1161:dead:beef:cafe:c18c, name=vmhost_p09_04_prod_02125527_20260210_124119, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 13:11:43,479][  INFO][PID:4014982] Allocating ssh connection to builder
[2026-02-10 13:11:43,489][  INFO][PID:4014982] Checking that builder machine is OK
[2026-02-10 13:11:44,568][  INFO][PID:4014982] Running remote command: copr-builder-ready fedora-43-ppc64le
[2026-02-10 13:11:45,142][  INFO][PID:4014982] Red Hat subscription not needed for fedora-43-ppc64le
Builder is ready to be used

[2026-02-10 13:11:45,143][  INFO][PID:4014982] Filling build.info file with builder info
[2026-02-10 13:11:45,313][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:11:45,315][  INFO][PID:4014982] Not re-notifying FE and msg buses for the new host.
[2026-02-10 13:11:45,316][  INFO][PID:4014982] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10112128-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached
[2026-02-10 13:11:46,645][  INFO][PID:4014982] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18c' (ED25519) to the list of known hosts.


[2026-02-10 13:11:46,647][  INFO][PID:4014982] Downloading the builder-live.log file, attempt 1
[2026-02-10 13:11:46,823][  INFO][PID:4014982] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log
[2026-02-10 13:11:51,648][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:11:56,652][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:01,654][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:06,663][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:11,672][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:16,679][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:21,685][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:26,687][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:31,689][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:36,690][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:41,697][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:46,710][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:46,959][ ERROR][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 13:12:51,726][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:56,737][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:12:56,975][WARNING][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-02-10 13:13:01,750][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:06,757][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:06,976][  INFO][PID:4014982] Downloading the builder-live.log file, attempt 2
[2026-02-10 13:13:06,981][  INFO][PID:4014982] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log
[2026-02-10 13:13:11,763][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:16,768][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:21,776][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:26,777][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:31,781][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:36,789][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:41,805][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:46,814][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:51,816][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:13:56,824][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:01,831][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:06,835][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:07,102][ ERROR][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 13:14:11,836][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:16,846][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:17,107][WARNING][PID:4014982] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-02-10 13:14:21,849][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:26,858][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:27,108][  INFO][PID:4014982] Downloading the builder-live.log file, attempt 3
[2026-02-10 13:14:27,121][  INFO][PID:4014982] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log
[2026-02-10 13:14:31,871][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:36,874][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:41,883][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:46,890][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:51,894][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:14:56,901][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:01,914][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:06,918][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:11,920][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:16,929][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:21,936][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:26,993][  INFO][PID:4014982] Periodic builder liveness probe: alive
[2026-02-10 13:15:26,994][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:27,197][ ERROR][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 13:15:32,002][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:37,005][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:37,270][WARNING][PID:4014982] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-02-10 13:15:42,006][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:47,020][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:47,274][  INFO][PID:4014982] Downloading the builder-live.log file, attempt 4
[2026-02-10 13:15:47,292][  INFO][PID:4014982] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log
[2026-02-10 13:15:52,022][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:15:57,040][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:02,044][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:07,045][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:12,048][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:17,051][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:22,055][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:27,062][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:32,065][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:37,069][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:42,076][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:47,081][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:47,408][ ERROR][PID:4014982] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-02-10 13:16:52,086][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:57,090][  INFO][PID:4014982] Checking for cancel request
[2026-02-10 13:16:57,433][ ERROR][PID:4014982] Giving up for unstable SSH, failures: 1770729176.9758353, 1770729257.1071854, 1770729337.2703907, 1770729417.4335449
[2026-02-10 13:16:57,435][ ERROR][PID:4014982] Re-try request for task on 'ResallocHost, ticket_id=1256215, hostname=2620:52:6:1161:dead:beef:cafe:c18c, name=vmhost_p09_04_prod_02125527_20260210_124119, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-02-10 13:16:57,439][  INFO][PID:4014982] Releasing VM back to pool
[2026-02-10 13:16:57,473][ ERROR][PID:4014982] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c18c', '2620:52:6:1161:dead:beef:cafe:c18b', '2620:52:6:1161:dead:beef:cafe:c192'}
[2026-02-10 13:16:57,478][WARNING][PID:4014982] Switching not-finished job state to 'failed'
[2026-02-10 13:16:57,481][  INFO][PID:4014982] Worker failed build, took 312.1624734401703
[2026-02-10 13:16:57,486][  INFO][PID:4014982] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "jonny",
            "project_name": "BitcoinCash",
            "project_dirname": "BitcoinCash",
            "submitter": "https://api.github.com/users/cculianu",
            "ended_on": 1770729417.4781053,
            "started_on": 1770729105.3156319,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-43-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10112128-fedora-43-ppc64le",
            "build_id": 10112128,
            "package_name": "fulcrum",
            "package_version": "2.1.0-0.0.git.1536.a517cbce",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/jonny/BitcoinCash/fulcrum",
            "git_hash": "ae4b0e076182f258549caaaf0b54c146708fcd6a",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "jonny/BitcoinCash--https://api.github.com/users/cculianu",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/jonny/BitcoinCash/fedora-43-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": true,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/jonny/BitcoinCash",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/jonny/BitcoinCash",
            "result_dir": "10112128-fulcrum",
            "built_packages": "",
            "id": 10112128,
            "mockchain_macros": {
                "copr_username": "jonny",
                "copr_projectname": "BitcoinCash",
                "vendor": "Fedora Project COPR (jonny/BitcoinCash)"
            }
        }
    ]
}
[2026-02-10 13:16:57,645][  INFO][PID:4014982] Sending fedora-messaging bus message in build.end
[2026-02-10 13:16:57,759][  INFO][PID:4014982] Compressing /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/builder-live.log by gzip
[2026-02-10 13:16:57,766][  INFO][PID:4014982] Running command 'gzip /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/builder-live.log' as PID 4176717
[2026-02-10 13:16:57,898][  INFO][PID:4014982] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/builder-live.log)
[2026-02-10 13:16:57,899][  INFO][PID:4014982] Compressing /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/backend.log by gzip
[2026-02-10 13:16:57,904][  INFO][PID:4014982] Running command 'gzip /var/lib/copr/public_html/results/jonny/BitcoinCash/fedora-43-ppc64le/10112128-fulcrum/backend.log' as PID 4176739