[2025-12-21 06:56:23,646][  INFO][PID:944862] Marking build as starting
[2025-12-21 06:56:23,834][  INFO][PID:944862] Checking that /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/repodata/repomd.xml exists
[2025-12-21 06:56:23,835][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:56:23,836][  INFO][PID:944862] VM allocation process starts
[2025-12-21 06:56:23,845][  INFO][PID:944862] Trying to allocate VM: ResallocHost, ticket_id=462069, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-21 06:56:26,857][  INFO][PID:944862] Allocated host ResallocHost, ticket_id=462069, hostname=2620:52:6:1161:dead:beef:cafe:c14a, name=vmhost_p09_03_prod_00720165_20251221_065132, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-21 06:56:27,250][  INFO][PID:944862] Allocating ssh connection to builder
[2025-12-21 06:56:27,250][  INFO][PID:944862] Checking that builder machine is OK
[2025-12-21 06:56:28,139][  INFO][PID:944862] Running remote command: copr-builder-ready fedora-43-ppc64le
[2025-12-21 06:56:28,626][  INFO][PID:944862] Red Hat subscription not needed for fedora-43-ppc64le
Builder is ready to be used

[2025-12-21 06:56:28,626][  INFO][PID:944862] Filling build.info file with builder info
[2025-12-21 06:56:28,627][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:56:28,628][  INFO][PID:944862] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "python3-scipy-1.11.1",
            "project_dirname": "python3-scipy-1.11.1",
            "submitter": "https://src.fedoraproject.org/user/music",
            "ended_on": null,
            "started_on": 1766300188.6278782,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-43-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9939225-fedora-43-ppc64le",
            "build_id": 9939225,
            "package_name": "python-pynwb",
            "package_version": "3.1.3-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/python3-scipy-1.11.1/python-pynwb",
            "git_hash": "65bd2f77f9bb9ab26cf169eeba45a31883553bc9",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/python3-scipy-1.11.1--https://src.fedoraproject.org/user/music",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1",
            "result_dir": "09939225-python-pynwb",
            "built_packages": "",
            "id": 9939225,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "python3-scipy-1.11.1",
                "vendor": "Fedora Project COPR (psimovec/python3-scipy-1.11.1)"
            }
        }
    ]
}
[2025-12-21 06:56:28,681][  INFO][PID:944862] Sending fedora-messaging bus message in build.start
[2025-12-21 06:56:29,183][  INFO][PID:944862] Sending fedora-messaging bus message in chroot.start
[2025-12-21 06:56:29,211][  INFO][PID:944862] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9939225-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached
[2025-12-21 06:56:30,105][  INFO][PID:944862] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c14a' (ED25519) to the list of known hosts.


[2025-12-21 06:56:30,106][  INFO][PID:944862] Downloading the builder-live.log file, attempt 1
[2025-12-21 06:56:30,107][  INFO][PID:944862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14a copr-rpmbuild-log
[2025-12-21 06:56:35,114][  INFO][PID:944862] Periodic builder liveness probe: alive
[2025-12-21 06:56:35,115][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:56:40,115][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:56:45,116][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:56:50,117][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:56:55,118][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:00,119][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:05,120][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:10,121][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:15,122][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:20,123][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:25,124][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:30,125][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:35,126][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:40,127][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:45,128][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:50,129][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:57:55,130][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:00,131][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:05,132][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:10,133][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:15,134][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:20,135][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:25,136][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:30,137][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:35,138][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:40,139][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:45,140][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:50,141][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:58:55,142][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:00,144][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:05,145][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:10,147][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:15,147][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:20,148][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:25,149][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:30,150][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:35,151][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:40,152][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:45,153][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:50,154][  INFO][PID:944862] Checking for cancel request
[2025-12-21 06:59:55,155][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:00,156][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:05,158][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:10,159][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:15,161][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:20,162][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:25,163][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:30,164][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:35,165][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:40,166][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:45,167][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:50,168][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:00:55,169][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:00,170][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:05,171][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:10,172][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:15,173][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:20,174][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:25,175][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:30,176][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:35,177][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:40,177][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:45,179][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:50,180][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:01:55,180][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:00,182][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:05,182][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:10,183][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:15,184][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:20,185][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:21,350][ ERROR][PID:944862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-21 07:02:25,186][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:30,187][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:31,351][WARNING][PID:944862] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-21 07:02:35,188][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:40,189][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:41,352][  INFO][PID:944862] Downloading the builder-live.log file, attempt 2
[2025-12-21 07:02:41,353][  INFO][PID:944862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14a copr-rpmbuild-log
[2025-12-21 07:02:45,190][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:50,214][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:02:55,215][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:00,216][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:05,217][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:10,218][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:15,219][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:20,220][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:25,221][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:30,222][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:35,223][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:40,224][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:41,421][ ERROR][PID:944862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-21 07:03:45,225][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:50,226][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:03:51,438][WARNING][PID:944862] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-21 07:03:55,229][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:00,230][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:01,439][  INFO][PID:944862] Downloading the builder-live.log file, attempt 3
[2025-12-21 07:04:01,440][  INFO][PID:944862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14a copr-rpmbuild-log
[2025-12-21 07:04:05,231][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:10,232][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:15,233][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:20,234][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:25,235][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:30,245][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:35,246][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:40,258][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:45,259][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:50,260][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:04:55,261][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:00,262][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:01,508][ ERROR][PID:944862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-21 07:05:05,263][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:10,264][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:11,548][WARNING][PID:944862] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-21 07:05:15,265][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:20,266][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:21,549][  INFO][PID:944862] Downloading the builder-live.log file, attempt 4
[2025-12-21 07:05:21,550][  INFO][PID:944862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c14a copr-rpmbuild-log
[2025-12-21 07:05:25,267][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:30,268][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:35,269][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:40,270][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:45,271][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:50,272][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:05:55,272][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:00,274][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:05,275][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:10,276][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:15,277][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:20,278][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:21,616][ ERROR][PID:944862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-21 07:06:25,279][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:30,280][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:31,627][ ERROR][PID:944862] Giving up for unstable SSH, failures: 1766300551.351689, 1766300631.4383547, 1766300711.5482306, 1766300791.6277564
[2025-12-21 07:06:31,628][ ERROR][PID:944862] Re-try request for task on 'ResallocHost, ticket_id=462069, hostname=2620:52:6:1161:dead:beef:cafe:c14a, name=vmhost_p09_03_prod_00720165_20251221_065132, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-12-21 07:06:31,629][  INFO][PID:944862] Releasing VM back to pool
[2025-12-21 07:06:31,639][  INFO][PID:944862] Retry #1 (on other host)
[2025-12-21 07:06:31,640][  INFO][PID:944862] Checking that /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/repodata/repomd.xml exists
[2025-12-21 07:06:31,640][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:06:31,640][  INFO][PID:944862] VM allocation process starts
[2025-12-21 07:06:31,652][  INFO][PID:944862] Trying to allocate VM: ResallocHost, ticket_id=462140, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-21 07:06:34,667][  INFO][PID:944862] Allocated host ResallocHost, ticket_id=462140, hostname=2620:52:6:1161:dead:beef:cafe:c14c, name=vmhost_p09_03_prod_00720172_20251221_065220, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-21 07:06:34,667][  INFO][PID:944862] Allocating ssh connection to builder
[2025-12-21 07:06:34,668][  INFO][PID:944862] Checking that builder machine is OK
[2025-12-21 07:07:34,678][ ERROR][PID:944862] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c14c port 22: Connection timed out

[2025-12-21 07:07:44,679][ ERROR][PID:944862] Re-try request for task on 'ResallocHost, ticket_id=462140, hostname=2620:52:6:1161:dead:beef:cafe:c14c, name=vmhost_p09_03_prod_00720172_20251221_065220, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2025-12-21 07:07:44,679][  INFO][PID:944862] Releasing VM back to pool
[2025-12-21 07:07:44,690][  INFO][PID:944862] Retry #2 (on other host)
[2025-12-21 07:07:44,690][  INFO][PID:944862] Checking that /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/repodata/repomd.xml exists
[2025-12-21 07:07:44,690][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:07:44,691][  INFO][PID:944862] VM allocation process starts
[2025-12-21 07:07:44,703][  INFO][PID:944862] Trying to allocate VM: ResallocHost, ticket_id=462169, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-21 07:07:47,720][  INFO][PID:944862] Allocated host ResallocHost, ticket_id=462169, hostname=2620:52:6:1161:dead:beef:cafe:c150, name=vmhost_p09_03_prod_00720250_20251221_070016, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-21 07:07:47,722][  INFO][PID:944862] Allocating ssh connection to builder
[2025-12-21 07:07:47,722][  INFO][PID:944862] Checking that builder machine is OK
[2025-12-21 07:07:48,676][  INFO][PID:944862] Running remote command: copr-builder-ready fedora-43-ppc64le
[2025-12-21 07:07:49,167][  INFO][PID:944862] Red Hat subscription not needed for fedora-43-ppc64le
Builder is ready to be used

[2025-12-21 07:07:49,168][  INFO][PID:944862] Filling build.info file with builder info
[2025-12-21 07:07:49,188][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:07:49,188][  INFO][PID:944862] Not re-notifying FE and msg buses for the new host.
[2025-12-21 07:07:49,189][  INFO][PID:944862] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9939225-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached
[2025-12-21 07:07:50,097][  INFO][PID:944862] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c150' (ED25519) to the list of known hosts.


[2025-12-21 07:07:50,098][  INFO][PID:944862] Downloading the builder-live.log file, attempt 1
[2025-12-21 07:07:50,099][  INFO][PID:944862] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c150 copr-rpmbuild-log
[2025-12-21 07:07:55,108][  INFO][PID:944862] Periodic builder liveness probe: alive
[2025-12-21 07:07:55,109][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:00,110][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:05,111][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:10,112][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:15,113][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:20,114][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:25,115][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:30,116][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:35,117][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:40,118][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:45,119][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:50,120][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:08:55,120][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:00,121][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:05,123][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:10,123][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:15,124][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:20,125][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:25,126][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:30,127][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:35,128][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:40,129][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:45,130][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:50,131][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:09:55,132][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:00,135][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:05,136][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:10,137][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:15,137][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:20,138][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:25,139][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:30,140][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:35,141][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:40,142][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:45,143][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:50,144][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:10:55,145][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:00,146][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:05,147][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:10,148][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:15,149][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:20,150][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:25,151][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:30,151][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:35,152][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:40,171][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:45,172][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:50,173][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:11:55,174][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:00,176][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:05,177][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:10,178][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:15,179][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:20,180][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:25,181][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:30,182][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:35,183][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:40,183][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:45,184][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:50,185][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:12:55,186][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:00,291][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:05,294][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:10,296][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:15,344][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:20,351][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:25,353][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:30,361][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:35,362][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:40,363][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:45,363][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:50,364][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:13:55,367][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:00,368][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:05,370][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:10,420][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:15,684][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:20,770][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:25,771][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:30,773][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:35,774][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:40,775][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:45,777][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:50,778][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:14:55,779][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:00,780][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:05,781][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:10,782][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:15,783][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:20,784][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:25,784][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:30,786][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:35,790][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:40,809][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:46,066][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:51,068][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:15:56,069][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:01,071][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:06,073][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:11,075][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:16,076][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:21,077][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:26,079][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:31,079][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:36,081][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:41,082][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:46,083][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:51,084][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:16:56,142][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:01,145][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:06,147][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:11,148][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:16,149][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:21,150][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:26,152][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:31,218][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:36,218][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:41,219][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:46,220][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:51,221][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:17:56,291][  INFO][PID:944862] Periodic builder liveness probe: alive
[2025-12-21 07:17:56,438][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:01,440][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:06,441][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:11,443][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:16,444][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:21,460][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:26,461][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:31,462][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:36,463][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:41,464][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:46,815][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:51,816][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:18:56,817][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:01,818][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:06,819][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:11,819][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:16,820][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:21,821][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:26,822][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:31,833][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:36,937][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:41,939][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:46,941][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:51,943][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:19:56,944][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:01,945][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:06,991][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:11,992][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:16,993][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:22,001][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:27,106][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:32,107][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:37,108][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:42,109][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:47,166][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:52,168][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:20:57,227][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:02,229][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:07,230][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:12,302][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:17,303][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:22,304][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:27,305][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:32,306][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:37,377][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:42,379][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:47,433][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:52,581][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:21:57,582][  INFO][PID:944862] Checking for cancel request
[2025-12-21 07:22:02,220][  INFO][PID:944862] Downloading results from builder
[2025-12-21 07:22:02,221][  INFO][PID:944862] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c150]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb started
[2025-12-21 07:22:02,222][  INFO][PID:944862] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c150]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/ &> /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/build-09939225.rsync.log
[2025-12-21 07:22:03,342][  INFO][PID:944862] rsyncing finished.
[2025-12-21 07:22:03,343][  INFO][PID:944862] Releasing VM back to pool
[2025-12-21 07:22:03,351][  INFO][PID:944862] Searching for 'success' file in resultdir
[2025-12-21 07:22:03,352][  INFO][PID:944862] Going to sign pkgs from source: 9939225-fedora-43-ppc64le in chroot: /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le
[2025-12-21 07:22:03,352][  INFO][PID:944862] Calling '/bin/sign -u psimovec#python3-scipy-1.11.1@copr.fedorahosted.org -p' (attempt #1)
[2025-12-21 07:22:03,619][  INFO][PID:944862] Calling '/bin/sign -4 -h sha256 -u psimovec#python3-scipy-1.11.1@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/python3-pynwb-3.1.3-1.fc44.noarch.rpm' (attempt #1)
[2025-12-21 07:22:04,339][  INFO][PID:944862] signed rpm: /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/python3-pynwb-3.1.3-1.fc44.noarch.rpm
[2025-12-21 07:22:04,339][  INFO][PID:944862] Calling '/bin/sign -4 -h sha256 -u psimovec#python3-scipy-1.11.1@copr.fedorahosted.org -r /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/python-pynwb-3.1.3-1.fc44.src.rpm' (attempt #1)
[2025-12-21 07:22:05,848][  INFO][PID:944862] signed rpm: /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/python-pynwb-3.1.3-1.fc44.src.rpm
[2025-12-21 07:22:05,849][  INFO][PID:944862] Sign done
[2025-12-21 07:22:05,849][  INFO][PID:944862] Incremental createrepo run, adding 09939225-python-pynwb into https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le, (auto-create-repo=True)
[2025-12-21 07:22:05,850][  INFO][PID:944862] Running command 'copr-repo --batched /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le --add 09939225-python-pynwb --no-appstream-metadata' as PID 1044413
[2025-12-21 07:22:07,857][  INFO][PID:944862] Finished after 2 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le --add 09939225-python-pynwb --no-appstream-metadata)
[2025-12-21 07:22:07,858][  INFO][PID:944862] Getting build details
[2025-12-21 07:22:07,858][  INFO][PID:944862] Listing built binary packages in /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb
[2025-12-21 07:22:07,859][  INFO][PID:944862] Built packages:
['python3-pynwb 3.1.3']
[2025-12-21 07:22:07,859][  INFO][PID:944862] build details: {'built_packages': 'python3-pynwb 3.1.3'}
[2025-12-21 07:22:07,859][  INFO][PID:944862] Retrieving pubkey
[2025-12-21 07:22:07,860][  INFO][PID:944862] Calling '/bin/sign -u psimovec#python3-scipy-1.11.1@copr.fedorahosted.org -p' (attempt #1)
[2025-12-21 07:22:08,053][  INFO][PID:944862] Added pubkey for user psimovec project python3-scipy-1.11.1 into: /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/pubkey.gpg
[2025-12-21 07:22:08,054][  INFO][PID:944862] Finished build: id=9939225 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1 chroot=fedora-43-ppc64le 
[2025-12-21 07:22:08,055][  INFO][PID:944862] Worker succeeded build, took 858.8669621944427
[2025-12-21 07:22:08,056][  INFO][PID:944862] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "psimovec",
            "project_name": "python3-scipy-1.11.1",
            "project_dirname": "python3-scipy-1.11.1",
            "submitter": "https://src.fedoraproject.org/user/music",
            "ended_on": 1766301728.0559049,
            "started_on": 1766300869.1889427,
            "submitted_on": null,
            "status": 1,
            "chroot": "fedora-43-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9939225-fedora-43-ppc64le",
            "build_id": 9939225,
            "package_name": "python-pynwb",
            "package_version": "3.1.3-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/psimovec/python3-scipy-1.11.1/python-pynwb",
            "git_hash": "65bd2f77f9bb9ab26cf169eeba45a31883553bc9",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "psimovec/python3-scipy-1.11.1--https://src.fedoraproject.org/user/music",
            "results": {
                "packages": [
                    {
                        "name": "python3-pynwb",
                        "epoch": null,
                        "version": "3.1.3",
                        "release": "1.fc44",
                        "arch": "noarch"
                    },
                    {
                        "name": "python-pynwb",
                        "epoch": null,
                        "version": "3.1.3",
                        "release": "1.fc44",
                        "arch": "src"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/",
                    "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch",
                    "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/psimovec/python3-scipy-1.11.1",
            "result_dir": "09939225-python-pynwb",
            "built_packages": "python3-pynwb 3.1.3",
            "id": 9939225,
            "mockchain_macros": {
                "copr_username": "psimovec",
                "copr_projectname": "python3-scipy-1.11.1",
                "vendor": "Fedora Project COPR (psimovec/python3-scipy-1.11.1)"
            }
        }
    ]
}
[2025-12-21 07:22:08,130][  INFO][PID:944862] Sending fedora-messaging bus message in build.end
[2025-12-21 07:22:08,158][  INFO][PID:944862] Compressing /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/builder-live.log by gzip
[2025-12-21 07:22:08,159][  INFO][PID:944862] Running command 'gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/builder-live.log' as PID 1044583
[2025-12-21 07:22:08,220][  INFO][PID:944862] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/builder-live.log)
[2025-12-21 07:22:08,220][  INFO][PID:944862] Compressing /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/backend.log by gzip
[2025-12-21 07:22:08,221][  INFO][PID:944862] Running command 'gzip /var/lib/copr/public_html/results/psimovec/python3-scipy-1.11.1/fedora-43-ppc64le/09939225-python-pynwb/backend.log' as PID 1044597