[2025-12-08 15:33:47,166][  INFO][PID:3366019] Marking build as starting
[2025-12-08 15:33:47,212][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:33:47,214][  INFO][PID:3366019] VM allocation process starts
[2025-12-08 15:33:47,229][  INFO][PID:3366019] Trying to allocate VM: ResallocHost, ticket_id=219303, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-08 15:33:52,230][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:33:53,266][  INFO][PID:3366019] Allocated host ResallocHost, ticket_id=219303, hostname=2620:52:6:1161:dead:beef:cafe:c10e, name=vmhost_p09_02_prod_00218047_20251208_152050, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-08 15:33:53,266][  INFO][PID:3366019] Allocating ssh connection to builder
[2025-12-08 15:33:53,267][  INFO][PID:3366019] Checking that builder machine is OK
[2025-12-08 15:34:53,327][ ERROR][PID:3366019] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c10e port 22: Connection timed out

[2025-12-08 15:35:03,405][ ERROR][PID:3366019] Re-try request for task on 'ResallocHost, ticket_id=219303, hostname=2620:52:6:1161:dead:beef:cafe:c10e, name=vmhost_p09_02_prod_00218047_20251208_152050, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2025-12-08 15:35:03,405][  INFO][PID:3366019] Releasing VM back to pool
[2025-12-08 15:35:03,416][  INFO][PID:3366019] Retry #1 (on other host)
[2025-12-08 15:35:03,417][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:35:03,418][  INFO][PID:3366019] VM allocation process starts
[2025-12-08 15:35:03,430][  INFO][PID:3366019] Trying to allocate VM: ResallocHost, ticket_id=219335, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-08 15:35:06,486][  INFO][PID:3366019] Allocated host ResallocHost, ticket_id=219335, hostname=2620:52:6:1161:dead:beef:cafe:c111, name=vmhost_p09_02_prod_00218231_20251208_152541, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-08 15:35:06,487][  INFO][PID:3366019] Allocating ssh connection to builder
[2025-12-08 15:35:06,488][  INFO][PID:3366019] Checking that builder machine is OK
[2025-12-08 15:36:06,559][ ERROR][PID:3366019] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c111 port 22: Connection timed out

[2025-12-08 15:36:16,560][ ERROR][PID:3366019] Re-try request for task on 'ResallocHost, ticket_id=219335, hostname=2620:52:6:1161:dead:beef:cafe:c111, name=vmhost_p09_02_prod_00218231_20251208_152541, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2025-12-08 15:36:16,561][  INFO][PID:3366019] Releasing VM back to pool
[2025-12-08 15:36:16,571][  INFO][PID:3366019] Retry #2 (on other host)
[2025-12-08 15:36:16,571][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:36:16,572][  INFO][PID:3366019] VM allocation process starts
[2025-12-08 15:36:16,585][  INFO][PID:3366019] Trying to allocate VM: ResallocHost, ticket_id=219372, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-08 15:36:21,587][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:36:22,616][  INFO][PID:3366019] Allocated host ResallocHost, ticket_id=219372, hostname=140.211.168.158, name=copr_osuosl_p09_prod_00217870_20251208_151550, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-08 15:36:22,617][  INFO][PID:3366019] Allocating ssh connection to builder
[2025-12-08 15:36:22,618][  INFO][PID:3366019] Checking that builder machine is OK
[2025-12-08 15:36:24,073][  INFO][PID:3366019] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-12-08 15:36:25,086][  INFO][PID:3366019] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-12-08 15:36:25,087][  INFO][PID:3366019] Filling build.info file with builder info
[2025-12-08 15:36:25,089][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:36:25,090][  INFO][PID:3366019] Not re-notifying FE and msg buses for the new host.
[2025-12-08 15:36:25,093][  INFO][PID:3366019] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9886848-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-12-08 15:36:26,557][  INFO][PID:3366019] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '140.211.168.158' (ED25519) to the list of known hosts.


[2025-12-08 15:36:26,560][  INFO][PID:3366019] Downloading the builder-live.log file, attempt 1
[2025-12-08 15:36:26,570][  INFO][PID:3366019] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.158 copr-rpmbuild-log
[2025-12-08 15:36:31,620][  INFO][PID:3366019] Periodic builder liveness probe: alive
[2025-12-08 15:36:31,621][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:36:36,623][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:36:41,625][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:36:46,627][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:36:51,629][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:36:56,630][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:01,632][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:06,634][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:11,635][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:16,636][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:21,637][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:26,682][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:31,683][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:36,684][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:41,685][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:46,729][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:51,732][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:37:56,734][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:01,741][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:06,742][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:11,743][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:16,744][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:21,752][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:26,853][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:31,902][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:36,904][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:42,175][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:47,496][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:52,499][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:38:57,500][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:02,503][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:07,504][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:12,508][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:17,511][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:22,515][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:27,519][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:32,526][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:37,527][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:42,528][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:47,529][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:52,530][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:39:57,531][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:02,532][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:07,533][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:12,534][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:17,537][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:22,539][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:27,540][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:32,543][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:37,544][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:42,746][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:47,751][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:52,755][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:40:57,758][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:02,764][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:07,766][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:12,768][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:17,769][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:22,775][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:27,778][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:32,779][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:37,780][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:42,782][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:47,784][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:52,785][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:41:57,787][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:02,789][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:07,790][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:12,793][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:17,794][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:22,795][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:27,796][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:32,797][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:37,798][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:42,799][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:47,800][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:52,802][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:42:57,803][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:43:02,804][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:43:07,805][  INFO][PID:3366019] Checking for cancel request
[2025-12-08 15:43:10,833][  INFO][PID:3366019] Downloading results from builder
[2025-12-08 15:43:10,834][  INFO][PID:3366019] rsyncing of mockbuilder@140.211.168.158:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/09886848-apbs started
[2025-12-08 15:43:10,835][  INFO][PID:3366019] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@140.211.168.158:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/09886848-apbs/ &> /var/lib/copr/public_html/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/09886848-apbs/build-09886848.rsync.log
[2025-12-08 15:43:13,904][  INFO][PID:3366019] rsyncing finished.
[2025-12-08 15:43:13,905][  INFO][PID:3366019] Releasing VM back to pool
[2025-12-08 15:43:13,916][  INFO][PID:3366019] Searching for 'success' file in resultdir
[2025-12-08 15:43:13,916][ ERROR][PID:3366019] Build failed: Backend process error: No success file => build failure
[2025-12-08 15:43:13,917][  INFO][PID:3366019] Finished build: id=9886848 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/jplesnik/swig-rebuild chroot=fedora-rawhide-ppc64le 
[2025-12-08 15:43:13,919][ ERROR][PID:3366019] Backend process error: No success file => build failure
[2025-12-08 15:43:13,919][  INFO][PID:3366019] Worker failed build, took 408.82950234413147
[2025-12-08 15:43:13,920][  INFO][PID:3366019] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "jplesnik",
            "project_name": "swig-rebuild",
            "project_dirname": "swig-rebuild",
            "submitter": "jplesnik",
            "ended_on": 1765208593.919665,
            "started_on": 1765208185.0901628,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9886848-fedora-rawhide-ppc64le",
            "build_id": 9886848,
            "package_name": "apbs",
            "package_version": "3.0.0-31",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/jplesnik/swig-rebuild/apbs",
            "git_hash": "f2255a329c32613dcced2babb8aed00c369234a8",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "jplesnik/swig-rebuild--jplesnik",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/jplesnik/swig-rebuild",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/jplesnik/swig-rebuild",
            "result_dir": "09886848-apbs",
            "built_packages": "",
            "id": 9886848,
            "mockchain_macros": {
                "copr_username": "jplesnik",
                "copr_projectname": "swig-rebuild",
                "vendor": "Fedora Project COPR (jplesnik/swig-rebuild)"
            }
        }
    ]
}
[2025-12-08 15:43:13,957][  INFO][PID:3366019] Sending fedora-messaging bus message in build.end
[2025-12-08 15:43:14,567][  INFO][PID:3366019] Compressing /var/lib/copr/public_html/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/09886848-apbs/builder-live.log by gzip
[2025-12-08 15:43:14,569][  INFO][PID:3366019] Running command 'gzip /var/lib/copr/public_html/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/09886848-apbs/builder-live.log' as PID 3424804
[2025-12-08 15:43:14,603][  INFO][PID:3366019] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/09886848-apbs/builder-live.log)
[2025-12-08 15:43:14,604][  INFO][PID:3366019] Compressing /var/lib/copr/public_html/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/09886848-apbs/backend.log by gzip
[2025-12-08 15:43:14,606][  INFO][PID:3366019] Running command 'gzip /var/lib/copr/public_html/results/jplesnik/swig-rebuild/fedora-rawhide-ppc64le/09886848-apbs/backend.log' as PID 3424807