[2025-12-20 12:20:05,469][  INFO][PID:921139] Marking build as starting
[2025-12-20 12:20:05,566][  INFO][PID:921139] Checking that /var/lib/copr/public_html/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2025-12-20 12:20:05,651][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:20:05,657][  INFO][PID:921139] VM allocation process starts
[2025-12-20 12:20:05,670][  INFO][PID:921139] Trying to allocate VM: ResallocHost, ticket_id=454958, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-20 12:20:08,702][  INFO][PID:921139] Allocated host ResallocHost, ticket_id=454958, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_00713792_20251220_120503, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-20 12:20:08,703][  INFO][PID:921139] Allocating ssh connection to builder
[2025-12-20 12:20:08,703][  INFO][PID:921139] Checking that builder machine is OK
[2025-12-20 12:21:08,771][ ERROR][PID:921139] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c14d port 22: Connection timed out

[2025-12-20 12:21:18,772][ ERROR][PID:921139] Re-try request for task on 'ResallocHost, ticket_id=454958, hostname=2620:52:6:1161:dead:beef:cafe:c14d, name=vmhost_p09_03_prod_00713792_20251220_120503, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2025-12-20 12:21:18,773][  INFO][PID:921139] Releasing VM back to pool
[2025-12-20 12:21:18,788][  INFO][PID:921139] Retry #1 (on other host)
[2025-12-20 12:21:18,789][  INFO][PID:921139] Checking that /var/lib/copr/public_html/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2025-12-20 12:21:18,789][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:21:18,790][  INFO][PID:921139] VM allocation process starts
[2025-12-20 12:21:18,850][  INFO][PID:921139] Trying to allocate VM: ResallocHost, ticket_id=454963, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-20 12:21:21,890][  INFO][PID:921139] Allocated host ResallocHost, ticket_id=454963, hostname=2620:52:6:1161:dead:beef:cafe:c152, name=vmhost_p09_03_prod_00713813_20251220_120652, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-20 12:21:21,890][  INFO][PID:921139] Allocating ssh connection to builder
[2025-12-20 12:21:21,891][  INFO][PID:921139] Checking that builder machine is OK
[2025-12-20 12:21:23,512][  INFO][PID:921139] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-12-20 12:21:23,988][  INFO][PID:921139] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-12-20 12:21:24,355][  INFO][PID:921139] Filling build.info file with builder info
[2025-12-20 12:21:24,389][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:21:24,390][  INFO][PID:921139] Not re-notifying FE and msg buses for the new host.
[2025-12-20 12:21:24,390][  INFO][PID:921139] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9937908-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-12-20 12:21:28,712][  INFO][PID:921139] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c152' (ED25519) to the list of known hosts.


[2025-12-20 12:21:28,713][  INFO][PID:921139] Downloading the builder-live.log file, attempt 1
[2025-12-20 12:21:28,714][  INFO][PID:921139] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c152 copr-rpmbuild-log
[2025-12-20 12:21:33,718][  INFO][PID:921139] Periodic builder liveness probe: alive
[2025-12-20 12:21:33,718][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:21:38,719][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:21:43,744][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:21:48,744][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:21:53,745][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:21:58,746][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:03,747][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:08,748][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:13,749][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:18,750][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:23,751][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:28,752][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:33,753][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:38,754][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:43,755][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:48,756][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:53,757][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:22:58,758][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:03,759][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:08,831][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:13,832][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:18,833][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:23,833][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:28,834][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:33,835][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:38,836][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:43,837][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:48,838][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:53,839][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:23:58,892][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:03,893][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:08,894][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:13,895][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:18,896][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:23,897][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:28,898][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:33,899][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:38,900][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:43,901][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:48,902][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:53,903][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:24:58,904][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:03,905][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:08,905][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:13,906][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:18,907][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:23,908][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:28,925][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:33,926][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:38,927][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:43,928][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:48,929][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:53,930][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:25:58,931][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:03,932][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:09,144][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:14,154][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:19,172][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:24,173][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:29,174][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:34,175][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:39,176][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:44,177][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:49,550][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:54,563][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:26:59,564][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:04,565][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:09,566][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:14,567][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:19,569][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:24,570][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:29,830][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:34,831][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:39,832][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:44,833][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:50,227][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:27:55,228][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:00,229][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:05,230][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:10,231][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:15,285][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:20,329][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:25,368][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:30,599][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:35,701][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:40,702][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:45,703][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:51,035][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:28:56,036][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:01,037][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:06,038][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:11,040][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:16,108][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:21,128][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:26,140][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:31,245][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:36,340][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:41,417][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:46,488][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:51,631][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:29:56,772][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:01,860][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:06,983][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:11,984][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:17,024][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:22,088][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:27,107][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:32,296][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:37,297][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:42,311][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:47,312][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:52,401][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:30:57,439][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:02,483][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:07,485][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:12,681][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:17,682][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:22,893][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:27,895][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:33,019][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:38,059][  INFO][PID:921139] Periodic builder liveness probe: alive
[2025-12-20 12:31:38,104][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:43,105][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:48,106][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:53,241][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:31:58,308][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:03,379][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:08,491][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:13,555][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:18,726][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:23,783][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:28,858][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:34,180][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:39,181][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:44,183][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:49,275][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:54,576][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:32:59,582][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:04,943][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:09,944][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:14,945][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:19,946][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:24,947][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:30,104][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:35,264][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:40,295][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:45,419][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:50,441][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:33:55,556][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:00,629][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:05,632][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:10,769][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:15,903][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:20,917][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:25,980][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:30,997][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:36,163][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:41,182][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:46,339][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:51,629][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:34:56,813][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:01,834][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:07,065][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:12,128][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:17,129][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:22,177][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:27,178][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:32,262][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:37,452][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:42,453][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:47,467][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:52,469][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:35:57,517][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:02,615][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:07,616][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:12,617][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:17,618][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:22,634][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:27,651][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:32,659][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:37,775][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:42,828][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:47,926][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:52,927][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:36:57,928][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:02,967][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:07,968][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:12,969][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:17,970][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:22,971][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:28,077][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:33,078][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:38,079][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:43,080][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:48,081][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:53,082][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:37:58,083][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:03,102][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:08,103][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:13,104][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:18,105][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:23,106][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:28,107][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:33,108][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:38,109][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:43,110][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:48,111][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:53,112][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:38:58,113][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:03,130][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:08,131][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:13,132][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:18,133][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:23,134][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:28,135][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:33,136][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:38,137][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:43,138][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:48,139][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:53,140][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:39:58,156][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:03,158][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:08,159][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:13,160][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:18,161][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:23,162][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:28,238][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:33,239][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:38,240][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:43,241][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:48,242][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:53,243][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:40:58,244][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:03,264][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:08,265][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:13,266][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:18,267][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:23,268][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:28,269][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:33,270][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:38,984][  INFO][PID:921139] Periodic builder liveness probe: alive
[2025-12-20 12:41:39,357][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:44,364][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:50,833][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:41:55,993][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:42:02,009][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:42:11,533][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:42:17,764][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:42:26,982][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:42:33,889][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:42:39,851][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:42:48,893][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:42:54,471][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:43:00,775][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:43:07,223][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:43:13,920][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:43:24,147][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:43:31,965][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:43:37,326][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:43:43,627][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:43:54,077][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:04,041][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:09,820][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:16,528][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:24,226][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:29,921][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:36,447][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:43,314][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:50,332][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:44:56,930][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:02,329][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:07,768][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:15,668][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:22,363][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:30,267][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:35,468][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:41,952][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:48,081][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:53,926][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:45:59,031][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:05,356][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:15,235][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:20,637][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:26,180][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:34,057][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:39,568][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:45,869][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:51,993][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:46:57,804][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:47:04,611][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:47:12,637][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:47:23,621][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:47:29,028][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:47:34,463][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:47:36,370][ ERROR][PID:921139] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-20 12:47:40,971][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:47:46,372][WARNING][PID:921139] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-20 12:47:47,745][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:47:56,372][  INFO][PID:921139] Downloading the builder-live.log file, attempt 2
[2025-12-20 12:47:56,378][  INFO][PID:921139] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c152 copr-rpmbuild-log
[2025-12-20 12:47:56,843][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:02,545][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:07,559][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:12,645][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:17,855][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:23,088][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:28,089][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:33,090][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:38,091][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:43,092][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:48,093][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:53,295][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:48:56,445][ ERROR][PID:921139] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-20 12:48:58,354][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:03,361][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:06,457][WARNING][PID:921139] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-20 12:49:08,369][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:13,433][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:16,459][  INFO][PID:921139] Downloading the builder-live.log file, attempt 3
[2025-12-20 12:49:16,462][  INFO][PID:921139] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c152 copr-rpmbuild-log
[2025-12-20 12:49:18,507][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:23,508][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:28,509][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:33,511][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:38,564][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:43,579][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:48,744][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:53,773][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:49:58,776][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:03,778][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:08,781][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:13,782][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:16,533][ ERROR][PID:921139] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-20 12:50:18,784][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:23,950][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:26,534][WARNING][PID:921139] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-12-20 12:50:28,988][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:33,990][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:36,535][  INFO][PID:921139] Downloading the builder-live.log file, attempt 4
[2025-12-20 12:50:36,536][  INFO][PID:921139] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c152 copr-rpmbuild-log
[2025-12-20 12:50:38,991][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:43,993][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:48,994][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:53,995][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:50:58,996][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:03,996][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:09,020][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:14,101][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:19,102][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:24,103][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:29,104][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:34,105][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:36,603][ ERROR][PID:921139] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-12-20 12:51:39,106][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:51:44,120][  INFO][PID:921139] Periodic builder liveness probe: dead
[2025-12-20 12:51:44,121][  INFO][PID:921139] calling cancel callback, and waiting
[2025-12-20 12:51:46,616][ ERROR][PID:921139] Giving up for unstable SSH, failures: 1766234866.372073, 1766234946.4578323, 1766235026.5342429, 1766235106.6159568
[2025-12-20 12:52:44,191][ ERROR][PID:921139] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c152 port 22: Connection timed out

[2025-12-20 12:53:54,249][ ERROR][PID:921139] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c152 port 22: Connection timed out

[2025-12-20 12:54:05,334][ ERROR][PID:921139] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c152 port 22: Permission denied

[2025-12-20 12:54:16,406][ ERROR][PID:921139] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c152 port 22: Permission denied

[2025-12-20 12:54:26,407][ ERROR][PID:921139] Can't ssh to cancel build.
[2025-12-20 12:54:26,408][ ERROR][PID:921139] Re-try request for task on 'ResallocHost, ticket_id=454963, hostname=2620:52:6:1161:dead:beef:cafe:c152, name=vmhost_p09_03_prod_00713813_20251220_120652, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-12-20 12:54:26,408][  INFO][PID:921139] Releasing VM back to pool
[2025-12-20 12:54:26,416][  INFO][PID:921139] Retry #2 (on other host)
[2025-12-20 12:54:26,417][  INFO][PID:921139] Checking that /var/lib/copr/public_html/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/repodata/repomd.xml exists
[2025-12-20 12:54:26,417][  INFO][PID:921139] Checking for cancel request
[2025-12-20 12:54:26,418][  INFO][PID:921139] VM allocation process starts
[2025-12-20 12:54:26,452][  INFO][PID:921139] Trying to allocate VM: ResallocHost, ticket_id=455229, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-20 12:54:29,494][  INFO][PID:921139] Allocated host ResallocHost, ticket_id=455229, hostname=2620:52:6:1161:dead:beef:cafe:c148, name=vmhost_p09_03_prod_00714054_20251220_124340, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-12-20 12:54:29,495][  INFO][PID:921139] Allocating ssh connection to builder
[2025-12-20 12:54:29,495][  INFO][PID:921139] Checking that builder machine is OK
[2025-12-20 12:55:29,544][ ERROR][PID:921139] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c148 port 22: Connection timed out

[2025-12-20 12:55:39,545][ ERROR][PID:921139] Re-try request for task on 'ResallocHost, ticket_id=455229, hostname=2620:52:6:1161:dead:beef:cafe:c148, name=vmhost_p09_03_prod_00714054_20251220_124340, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts
[2025-12-20 12:55:39,545][  INFO][PID:921139] Releasing VM back to pool
[2025-12-20 12:55:39,554][ ERROR][PID:921139] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c148', '2620:52:6:1161:dead:beef:cafe:c152', '2620:52:6:1161:dead:beef:cafe:c14d'}
[2025-12-20 12:55:39,555][WARNING][PID:921139] Switching not-finished job state to 'failed'
[2025-12-20 12:55:39,555][  INFO][PID:921139] Worker failed build, took 2055.1651837825775
[2025-12-20 12:55:39,556][  INFO][PID:921139] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "mtasaka",
            "project_name": "ruby-40-01",
            "project_dirname": "ruby-40-01",
            "submitter": "mtasaka",
            "ended_on": 1766235339.5552554,
            "started_on": 1766233284.3900716,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "9937908-fedora-rawhide-ppc64le",
            "build_id": 9937908,
            "package_name": "ruby",
            "package_version": "4.0.0~20251220.1216gita57039196f-29.fc44.gita57039196f",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/mtasaka/ruby-40-01/ruby",
            "git_hash": "00b868d5ad587dc00b2bb432a1eb11f7cb2e93e3",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "mtasaka/ruby-40-01--mtasaka",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/mtasaka/ruby-40-01",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/mtasaka/ruby-40-01",
            "result_dir": "09937908-ruby",
            "built_packages": "",
            "id": 9937908,
            "mockchain_macros": {
                "copr_username": "mtasaka",
                "copr_projectname": "ruby-40-01",
                "vendor": "Fedora Project COPR (mtasaka/ruby-40-01)"
            }
        }
    ]
}
[2025-12-20 12:55:39,601][  INFO][PID:921139] Sending fedora-messaging bus message in build.end
[2025-12-20 12:55:40,125][  INFO][PID:921139] Compressing /var/lib/copr/public_html/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/09937908-ruby/builder-live.log by gzip
[2025-12-20 12:55:40,126][  INFO][PID:921139] Running command 'gzip /var/lib/copr/public_html/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/09937908-ruby/builder-live.log' as PID 1066378
[2025-12-20 12:55:40,234][  INFO][PID:921139] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/09937908-ruby/builder-live.log)
[2025-12-20 12:55:40,235][  INFO][PID:921139] Compressing /var/lib/copr/public_html/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/09937908-ruby/backend.log by gzip
[2025-12-20 12:55:40,235][  INFO][PID:921139] Running command 'gzip /var/lib/copr/public_html/results/mtasaka/ruby-40-01/fedora-rawhide-ppc64le/09937908-ruby/backend.log' as PID 1066383