[2026-02-10 17:12:13,453][  INFO][PID:2093837] Marking build as starting
[2026-02-10 17:12:13,554][  INFO][PID:2093837] Checking that /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/repodata/repomd.xml exists
[2026-02-10 17:12:13,778][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:12:13,779][  INFO][PID:2093837] VM allocation process starts
[2026-02-10 17:12:13,793][  INFO][PID:2093837] Trying to allocate VM: ResallocHost, ticket_id=1260309, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 17:12:16,812][  INFO][PID:2093837] Allocated host ResallocHost, ticket_id=1260309, hostname=2620:52:6:1161:dead:beef:cafe:c18f, name=vmhost_p09_04_prod_02145551_20260210_165749, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 17:12:16,814][  INFO][PID:2093837] Allocating ssh connection to builder
[2026-02-10 17:12:16,817][  INFO][PID:2093837] Checking that builder machine is OK
[2026-02-10 17:13:16,894][ ERROR][PID:2093837] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18f port 22: Connection timed out

[2026-02-10 17:13:26,922][ ERROR][PID:2093837] Re-try request for task on 'ResallocHost, ticket_id=1260309, hostname=2620:52:6:1161:dead:beef:cafe:c18f, name=vmhost_p09_04_prod_02145551_20260210_165749, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-02-10 17:13:26,922][  INFO][PID:2093837] Releasing VM back to pool
[2026-02-10 17:13:26,932][  INFO][PID:2093837] Retry #1 (on other host)
[2026-02-10 17:13:26,933][  INFO][PID:2093837] Checking that /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/repodata/repomd.xml exists
[2026-02-10 17:13:26,933][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:13:26,934][  INFO][PID:2093837] VM allocation process starts
[2026-02-10 17:13:26,944][  INFO][PID:2093837] Trying to allocate VM: ResallocHost, ticket_id=1260326, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 17:13:29,960][  INFO][PID:2093837] Allocated host ResallocHost, ticket_id=1260326, hostname=2620:52:6:1161:dead:beef:cafe:c114, name=vmhost_p09_02_prod_02145836_20260210_170337, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 17:13:29,961][  INFO][PID:2093837] Allocating ssh connection to builder
[2026-02-10 17:13:29,961][  INFO][PID:2093837] Checking that builder machine is OK
[2026-02-10 17:14:30,033][ ERROR][PID:2093837] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c114 port 22: Connection timed out

[2026-02-10 17:14:40,034][ ERROR][PID:2093837] Re-try request for task on 'ResallocHost, ticket_id=1260326, hostname=2620:52:6:1161:dead:beef:cafe:c114, name=vmhost_p09_02_prod_02145836_20260210_170337, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2026-02-10 17:14:40,035][  INFO][PID:2093837] Releasing VM back to pool
[2026-02-10 17:14:40,116][  INFO][PID:2093837] Retry #2 (on other host)
[2026-02-10 17:14:40,117][  INFO][PID:2093837] Checking that /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/repodata/repomd.xml exists
[2026-02-10 17:14:40,118][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:14:40,118][  INFO][PID:2093837] VM allocation process starts
[2026-02-10 17:14:40,132][  INFO][PID:2093837] Trying to allocate VM: ResallocHost, ticket_id=1260337, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 17:14:43,148][  INFO][PID:2093837] Allocated host ResallocHost, ticket_id=1260337, hostname=2620:52:6:1161:dead:beef:cafe:c10f, name=vmhost_p09_02_prod_02145962_20260210_170609, requested_tags=['copr_builder', 'arch_ppc64le']
[2026-02-10 17:14:43,148][  INFO][PID:2093837] Allocating ssh connection to builder
[2026-02-10 17:14:43,149][  INFO][PID:2093837] Checking that builder machine is OK
[2026-02-10 17:14:44,095][  INFO][PID:2093837] Running remote command: copr-builder-ready fedora-44-ppc64le
[2026-02-10 17:14:44,619][  INFO][PID:2093837] Red Hat subscription not needed for fedora-44-ppc64le
Builder is ready to be used

[2026-02-10 17:14:44,620][  INFO][PID:2093837] Filling build.info file with builder info
[2026-02-10 17:14:44,620][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:14:44,621][  INFO][PID:2093837] Not re-notifying FE and msg buses for the new host.
[2026-02-10 17:14:44,622][  INFO][PID:2093837] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10113564-fedora-44-ppc64le --chroot fedora-44-ppc64le --detached
[2026-02-10 17:14:45,551][  INFO][PID:2093837] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c10f' (ED25519) to the list of known hosts.


[2026-02-10 17:14:45,552][  INFO][PID:2093837] Downloading the builder-live.log file, attempt 1
[2026-02-10 17:14:45,553][  INFO][PID:2093837] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c10f copr-rpmbuild-log
[2026-02-10 17:14:50,563][  INFO][PID:2093837] Periodic builder liveness probe: alive
[2026-02-10 17:14:50,564][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:14:55,565][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:00,566][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:05,567][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:10,568][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:15,569][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:20,570][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:25,571][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:30,572][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:35,577][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:40,578][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:45,579][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:50,581][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:15:55,582][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:00,583][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:05,594][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:10,598][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:15,599][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:20,600][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:25,603][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:30,606][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:35,607][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:40,608][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:45,609][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:50,610][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:16:55,612][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:00,648][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:05,649][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:10,649][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:15,650][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:20,655][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:25,656][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:30,657][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:35,657][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:40,659][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:45,661][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:50,663][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:17:55,664][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:00,665][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:05,666][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:10,667][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:15,668][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:20,671][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:25,677][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:30,678][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:35,680][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:40,682][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:45,683][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:50,687][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:18:55,689][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:00,690][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:05,691][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:10,692][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:15,694][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:20,695][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:25,696][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:30,697][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:35,699][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:40,700][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:45,701][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:50,701][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:19:55,703][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:00,706][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:05,708][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:10,708][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:15,709][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:20,711][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:25,712][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:30,713][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:35,714][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:40,716][  INFO][PID:2093837] Checking for cancel request
[2026-02-10 17:20:42,941][  INFO][PID:2093837] Downloading results from builder
[2026-02-10 17:20:42,942][  INFO][PID:2093837] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10f]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/10113564-freeipa started
[2026-02-10 17:20:42,943][  INFO][PID:2093837] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c10f]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/10113564-freeipa/ &> /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/10113564-freeipa/build-10113564.rsync.log
[2026-02-10 17:20:44,068][  INFO][PID:2093837] rsyncing finished.
[2026-02-10 17:20:44,069][  INFO][PID:2093837] Releasing VM back to pool
[2026-02-10 17:20:44,081][  INFO][PID:2093837] Searching for 'success' file in resultdir
[2026-02-10 17:20:44,082][ ERROR][PID:2093837] Build failed: Backend process error: No success file => build failure
[2026-02-10 17:20:44,082][  INFO][PID:2093837] Finished build: id=10113564 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly chroot=fedora-44-ppc64le 
[2026-02-10 17:20:44,084][ ERROR][PID:2093837] Backend process error: No success file => build failure
[2026-02-10 17:20:44,084][  INFO][PID:2093837] Worker failed build, took 359.4629566669464
[2026-02-10 17:20:44,085][  INFO][PID:2093837] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "@freeipa",
            "project_name": "freeipa-4.13-nightly",
            "project_dirname": "freeipa-4.13-nightly",
            "submitter": "https://api.github.com/users/abbra",
            "ended_on": 1770744044.084523,
            "started_on": 1770743684.6215663,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-44-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "10113564-fedora-44-ppc64le",
            "build_id": 10113564,
            "package_name": "freeipa",
            "package_version": "4.13.2.dev202602101706+git2e24bfc03-0",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@freeipa/freeipa-4.13-nightly/freeipa",
            "git_hash": "bcb8a49506d1b77a801f38dea49671c248a54228",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@freeipa/freeipa-4.13-nightly--https://api.github.com/users/abbra",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@freeipa/freeipa-4.13-nightly",
            "result_dir": "10113564-freeipa",
            "built_packages": "",
            "id": 10113564,
            "mockchain_macros": {
                "copr_username": "@freeipa",
                "copr_projectname": "freeipa-4.13-nightly",
                "vendor": "Fedora Project COPR (@freeipa/freeipa-4.13-nightly)"
            }
        }
    ]
}
[2026-02-10 17:20:44,138][  INFO][PID:2093837] Sending fedora-messaging bus message in build.end
[2026-02-10 17:20:44,712][  INFO][PID:2093837] Compressing /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/10113564-freeipa/builder-live.log by gzip
[2026-02-10 17:20:44,715][  INFO][PID:2093837] Running command 'gzip /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/10113564-freeipa/builder-live.log' as PID 2156439
[2026-02-10 17:20:44,808][  INFO][PID:2093837] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/10113564-freeipa/builder-live.log)
[2026-02-10 17:20:44,808][  INFO][PID:2093837] Compressing /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/10113564-freeipa/backend.log by gzip
[2026-02-10 17:20:44,810][  INFO][PID:2093837] Running command 'gzip /var/lib/copr/public_html/results/@freeipa/freeipa-4.13-nightly/fedora-44-ppc64le/10113564-freeipa/backend.log' as PID 2156469