[2025-12-23 03:04:50,263][ INFO][PID:2054485] Marking build as starting [2025-12-23 03:04:50,349][ INFO][PID:2054485] Checking that /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/repodata/repomd.xml exists [2025-12-23 03:04:50,360][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:04:50,361][ INFO][PID:2054485] VM allocation process starts [2025-12-23 03:04:50,371][ INFO][PID:2054485] Trying to allocate VM: ResallocHost, ticket_id=478805, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-23 03:04:53,383][ INFO][PID:2054485] Allocated host ResallocHost, ticket_id=478805, hostname=2620:52:6:1161:dead:beef:cafe:c150, name=vmhost_p09_03_prod_00735773_20251223_024421, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-23 03:04:53,383][ INFO][PID:2054485] Allocating ssh connection to builder [2025-12-23 03:04:53,384][ INFO][PID:2054485] Checking that builder machine is OK [2025-12-23 03:05:53,453][ ERROR][PID:2054485] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c150 port 22: Connection timed out [2025-12-23 03:06:03,454][ ERROR][PID:2054485] Re-try request for task on 'ResallocHost, ticket_id=478805, hostname=2620:52:6:1161:dead:beef:cafe:c150, name=vmhost_p09_03_prod_00735773_20251223_024421, requested_tags=['copr_builder', 'arch_ppc64le']': Unable to finish after 1 SSH attempts [2025-12-23 03:06:03,454][ INFO][PID:2054485] Releasing VM back to pool [2025-12-23 03:06:03,463][ INFO][PID:2054485] Retry #1 (on other host) [2025-12-23 03:06:03,464][ INFO][PID:2054485] Checking that /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/repodata/repomd.xml exists [2025-12-23 03:06:03,464][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:03,465][ INFO][PID:2054485] VM allocation process starts [2025-12-23 03:06:03,510][ INFO][PID:2054485] Trying to allocate VM: ResallocHost, ticket_id=478812, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-23 03:06:06,571][ INFO][PID:2054485] Allocated host ResallocHost, ticket_id=478812, hostname=2620:52:6:1161:dead:beef:cafe:c18c, name=vmhost_p09_04_prod_00735807_20251223_024818, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-23 03:06:06,572][ INFO][PID:2054485] Allocating ssh connection to builder [2025-12-23 03:06:06,572][ INFO][PID:2054485] Checking that builder machine is OK [2025-12-23 03:06:07,472][ INFO][PID:2054485] Running remote command: copr-builder-ready fedora-43-ppc64le [2025-12-23 03:06:07,952][ INFO][PID:2054485] Red Hat subscription not needed for fedora-43-ppc64le Builder is ready to be used [2025-12-23 03:06:07,952][ INFO][PID:2054485] Filling build.info file with builder info [2025-12-23 03:06:07,953][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:07,953][ INFO][PID:2054485] Not re-notifying FE and msg buses for the new host. [2025-12-23 03:06:07,954][ INFO][PID:2054485] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9942974-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached [2025-12-23 03:06:08,859][ INFO][PID:2054485] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18c' (ED25519) to the list of known hosts. [2025-12-23 03:06:08,860][ INFO][PID:2054485] Downloading the builder-live.log file, attempt 1 [2025-12-23 03:06:08,861][ INFO][PID:2054485] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log [2025-12-23 03:06:13,877][ INFO][PID:2054485] Periodic builder liveness probe: alive [2025-12-23 03:06:13,878][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:18,883][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:23,885][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:28,886][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:33,887][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:38,888][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:43,889][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:48,890][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:53,892][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:06:58,892][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:03,893][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:08,894][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:13,895][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:18,896][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:23,898][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:28,899][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:33,900][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:38,901][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:43,902][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:48,903][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:53,904][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:07:58,905][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:03,906][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:08,907][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:13,952][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:18,953][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:23,955][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:28,957][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:33,958][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:38,959][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:43,960][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:48,961][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:53,963][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:08:58,964][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:03,965][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:08,966][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:13,967][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:18,969][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:23,979][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:28,980][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:33,983][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:38,985][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:43,986][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:48,987][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:53,988][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:09:58,990][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:03,992][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:08,994][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:13,997][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:18,998][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:24,000][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:29,002][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:34,008][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:39,010][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:44,011][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:49,012][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:54,013][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:10:59,014][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:04,015][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:09,015][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:14,016][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:19,018][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:24,019][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:29,021][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:34,023][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:39,024][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:44,024][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:49,025][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:54,026][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:11:59,027][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:04,028][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:09,029][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:14,030][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:19,031][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:24,032][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:29,033][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:34,034][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:39,035][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:44,036][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:49,037][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:54,038][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:12:59,039][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:04,040][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:09,041][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:14,042][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:19,043][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:24,044][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:29,045][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:34,046][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:39,047][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:44,047][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:49,048][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:54,049][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:13:59,050][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:04,051][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:09,052][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:14,053][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:19,054][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:24,055][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:29,056][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:34,057][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:39,058][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:44,059][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:49,060][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:54,061][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:14:59,065][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:04,066][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:09,111][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:14,269][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:19,388][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:24,449][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:29,536][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:34,596][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:39,646][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:44,758][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:49,846][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:15:54,981][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:00,183][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:05,188][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:10,201][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:15,211][ INFO][PID:2054485] Periodic builder liveness probe: alive [2025-12-23 03:16:15,213][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:20,219][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:25,231][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:30,233][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:35,236][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:40,236][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:45,237][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:50,238][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:16:55,239][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:00,240][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:05,290][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:10,291][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:15,292][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:20,923][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:25,926][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:30,927][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:35,928][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:40,929][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:45,930][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:50,931][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:17:55,932][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:00,933][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:05,934][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:10,935][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:15,936][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:21,032][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:26,033][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:31,034][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:36,035][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:41,036][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:46,037][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:51,038][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:18:56,039][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:01,040][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:06,041][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:11,042][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:16,043][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:21,402][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:26,404][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:31,405][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:36,406][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:41,407][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:46,408][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:51,409][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:19:56,410][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:01,412][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:06,413][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:11,414][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:16,417][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:21,419][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:26,420][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:31,421][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:36,422][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:41,423][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:46,423][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:51,425][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:20:56,426][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:01,427][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:06,428][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:11,429][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:16,667][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:21,669][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:26,671][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:31,672][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:36,674][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:41,676][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:46,677][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:51,678][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:21:56,679][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:01,680][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:06,681][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:11,682][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:16,684][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:21,686][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:26,687][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:31,688][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:36,689][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:41,690][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:46,691][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:51,692][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:22:56,693][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:01,694][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:06,695][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:11,696][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:16,697][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:21,698][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:26,700][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:31,701][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:36,702][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:41,703][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:46,704][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:51,705][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:23:56,706][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:01,711][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:06,713][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:11,714][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:16,715][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:21,717][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:26,718][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:31,767][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:36,768][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:41,770][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:46,771][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:51,772][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:24:56,773][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:01,774][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:06,775][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:11,776][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:16,777][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:21,778][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:26,872][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:31,873][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:36,875][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:41,875][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:46,877][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:51,877][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:25:56,878][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:01,879][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:06,881][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:11,882][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:16,887][ INFO][PID:2054485] Periodic builder liveness probe: alive [2025-12-23 03:26:16,888][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:21,889][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:26,889][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:31,891][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:36,892][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:42,029][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:47,031][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:52,032][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:26:57,033][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:02,099][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:07,101][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:12,137][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:17,138][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:22,139][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:27,140][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:32,141][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:37,142][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:42,143][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:47,144][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:52,145][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:27:57,146][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:02,147][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:07,148][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:12,149][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:17,150][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:22,151][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:27,152][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:32,153][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:37,154][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:42,155][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:47,155][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:52,156][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:28:57,338][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:02,343][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:07,344][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:12,345][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:17,346][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:22,347][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:27,348][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:32,349][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:37,351][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:42,352][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:47,353][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:52,354][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:29:57,355][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:02,356][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:07,357][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:12,358][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:17,359][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:22,361][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:27,361][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:32,362][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:37,363][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:42,593][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:47,595][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:53,067][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:30:58,071][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:03,391][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:08,490][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:13,493][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:18,496][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:23,498][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:28,501][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:33,503][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:38,503][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:44,018][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:49,019][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:54,020][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:31:59,022][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:04,023][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:09,024][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:14,026][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:19,027][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:24,481][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:29,524][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:34,579][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:39,662][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:44,668][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:49,830][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:54,913][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:32:59,915][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:04,918][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:10,155][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:15,157][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:20,243][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:25,244][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:30,245][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:35,246][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:40,247][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:45,248][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:50,249][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:33:55,250][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:00,251][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:03,356][ ERROR][PID:2054485] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-23 03:34:05,252][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:10,253][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:13,357][WARNING][PID:2054485] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-23 03:34:15,254][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:20,255][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:23,358][ INFO][PID:2054485] Downloading the builder-live.log file, attempt 2 [2025-12-23 03:34:23,552][ INFO][PID:2054485] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log [2025-12-23 03:34:25,256][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:30,257][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:35,258][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:40,293][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:45,294][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:50,295][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:34:55,659][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:00,660][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:05,660][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:10,661][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:15,663][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:20,664][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:23,619][ ERROR][PID:2054485] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-23 03:35:25,665][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:30,666][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:33,865][WARNING][PID:2054485] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-23 03:35:35,667][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:40,668][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:43,865][ INFO][PID:2054485] Downloading the builder-live.log file, attempt 3 [2025-12-23 03:35:43,867][ INFO][PID:2054485] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log [2025-12-23 03:35:45,669][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:50,670][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:35:55,671][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:36:00,672][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:36:05,673][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:36:10,674][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:36:15,676][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:36:20,682][ INFO][PID:2054485] Periodic builder liveness probe: dead [2025-12-23 03:36:20,683][ INFO][PID:2054485] calling cancel callback, and waiting [2025-12-23 03:36:43,932][ ERROR][PID:2054485] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-23 03:36:53,964][WARNING][PID:2054485] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2025-12-23 03:37:03,967][ INFO][PID:2054485] Downloading the builder-live.log file, attempt 4 [2025-12-23 03:37:03,969][ INFO][PID:2054485] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c18c copr-rpmbuild-log [2025-12-23 03:37:20,750][ ERROR][PID:2054485] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke: OUT: ERR: ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c18c port 22: Connection timed out [2025-12-23 03:38:04,036][ ERROR][PID:2054485] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2025-12-23 03:38:08,012][WARNING][PID:2054485] Can't cancel build out: err: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c18c' (ED25519) to the list of known hosts. Traceback (most recent call last): File "/usr/bin/copr-rpmbuild-cancel", line 85, in _main() ~~~~~^^ File "/usr/bin/copr-rpmbuild-cancel", line 69, in _main with open(pidfile, "r") as pidfd: ~~~~^^^^^^^^^^^^^^ FileNotFoundError: [Errno 2] No such file or directory: '/var/lib/copr-rpmbuild/pid' [2025-12-23 03:38:14,050][ ERROR][PID:2054485] Giving up for unstable SSH, failures: 1766460853.3574703, 1766460933.8651752, 1766461013.9644544, 1766461094.050235 [2025-12-23 03:38:14,068][ ERROR][PID:2054485] Re-try request for task on 'ResallocHost, ticket_id=478812, hostname=2620:52:6:1161:dead:beef:cafe:c18c, name=vmhost_p09_04_prod_00735807_20251223_024818, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED [2025-12-23 03:38:14,069][ INFO][PID:2054485] Releasing VM back to pool [2025-12-23 03:38:14,079][ INFO][PID:2054485] Retry #2 (on other host) [2025-12-23 03:38:14,079][ INFO][PID:2054485] Checking that /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/repodata/repomd.xml exists [2025-12-23 03:38:14,080][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:38:14,081][ INFO][PID:2054485] VM allocation process starts [2025-12-23 03:38:14,091][ INFO][PID:2054485] Trying to allocate VM: ResallocHost, ticket_id=479141, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-23 03:38:17,108][ INFO][PID:2054485] Allocated host ResallocHost, ticket_id=479141, hostname=2620:52:6:1161:dead:beef:cafe:c189, name=vmhost_p09_04_prod_00736049_20251223_033259, requested_tags=['copr_builder', 'arch_ppc64le'] [2025-12-23 03:38:17,109][ INFO][PID:2054485] Allocating ssh connection to builder [2025-12-23 03:38:17,110][ INFO][PID:2054485] Checking that builder machine is OK [2025-12-23 03:38:17,989][ INFO][PID:2054485] Running remote command: copr-builder-ready fedora-43-ppc64le [2025-12-23 03:38:18,473][ INFO][PID:2054485] Red Hat subscription not needed for fedora-43-ppc64le Builder is ready to be used [2025-12-23 03:38:18,473][ INFO][PID:2054485] Filling build.info file with builder info [2025-12-23 03:38:18,493][ INFO][PID:2054485] Checking for cancel request [2025-12-23 03:38:18,494][ INFO][PID:2054485] Not re-notifying FE and msg buses for the new host. [2025-12-23 03:38:18,495][ INFO][PID:2054485] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9942974-fedora-43-ppc64le --chroot fedora-43-ppc64le --detached [2025-12-23 03:38:19,506][ INFO][PID:2054485] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c189' (ED25519) to the list of known hosts. [2025-12-23 03:38:19,507][ INFO][PID:2054485] Downloading the builder-live.log file, attempt 1 [2025-12-23 03:38:19,508][ INFO][PID:2054485] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:6:1161:dead:beef:cafe:c189 copr-rpmbuild-log [2025-12-23 03:38:24,507][ INFO][PID:2054485] calling cancel callback, and waiting [2025-12-23 03:38:28,211][ INFO][PID:2054485] Cancel request succeeded out: success err: Warning: Permanently added '2620:52:6:1161:dead:beef:cafe:c189' (ED25519) to the list of known hosts. [2025-12-23 03:38:28,212][ INFO][PID:2054485] Downloading results from builder [2025-12-23 03:38:28,212][ INFO][PID:2054485] rsyncing of mockbuilder@[2620:52:6:1161:dead:beef:cafe:c189]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/09942974-biglybt started [2025-12-23 03:38:28,213][ INFO][PID:2054485] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:6:1161:dead:beef:cafe:c189]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/09942974-biglybt/ &> /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/09942974-biglybt/build-09942974.rsync.log [2025-12-23 03:38:28,839][ INFO][PID:2054485] rsyncing finished. [2025-12-23 03:38:28,840][ INFO][PID:2054485] Releasing VM back to pool [2025-12-23 03:38:28,848][ INFO][PID:2054485] Searching for 'success' file in resultdir [2025-12-23 03:38:28,849][ ERROR][PID:2054485] Build failed: Backend process error: No success file => build failure [2025-12-23 03:38:28,849][ INFO][PID:2054485] Finished build: id=9942974 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases chroot=fedora-43-ppc64le [2025-12-23 03:38:28,851][ ERROR][PID:2054485] Backend process error: No success file => build failure [2025-12-23 03:38:28,852][ INFO][PID:2054485] Worker failed build, took 10.357377529144287 [2025-12-23 03:38:28,852][ INFO][PID:2054485] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "sergiomb", "project_name": "builds_for_Stable_Releases", "project_dirname": "builds_for_Stable_Releases", "submitter": "sergiomb", "ended_on": 1766461108.8520164, "started_on": 1766461098.494639, "submitted_on": null, "status": 0, "chroot": "fedora-43-ppc64le", "arch": "ppc64le", "buildroot_pkgs": [], "task_id": "9942974-fedora-43-ppc64le", "build_id": 9942974, "package_name": "biglybt", "package_version": "4.0.0.0-1", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/sergiomb/builds_for_Stable_Releases/biglybt", "git_hash": "508916355849ed7ece64a981a3ca02a863195f69", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "sergiomb/builds_for_Stable_Releases--sergiomb", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 0, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": false, "fedora_review": true, "isolation": "default", "repo_priority": null, "tags": [ "arch_ppc64le" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/sergiomb/builds_for_Stable_Releases", "result_dir": "09942974-biglybt", "built_packages": "", "id": 9942974, "mockchain_macros": { "copr_username": "sergiomb", "copr_projectname": "builds_for_Stable_Releases", "vendor": "Fedora Project COPR (sergiomb/builds_for_Stable_Releases)" } } ] } [2025-12-23 03:38:28,909][ INFO][PID:2054485] Sending fedora-messaging bus message in build.end [2025-12-23 03:38:29,460][ INFO][PID:2054485] Compressing /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/09942974-biglybt/builder-live.log by gzip [2025-12-23 03:38:29,461][ INFO][PID:2054485] Running command 'gzip /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/09942974-biglybt/builder-live.log' as PID 2208282 [2025-12-23 03:38:29,463][ INFO][PID:2054485] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/09942974-biglybt/builder-live.log) [2025-12-23 03:38:29,463][ INFO][PID:2054485] Compressing /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/09942974-biglybt/backend.log by gzip [2025-12-23 03:38:29,464][ INFO][PID:2054485] Running command 'gzip /var/lib/copr/public_html/results/sergiomb/builds_for_Stable_Releases/fedora-43-ppc64le/09942974-biglybt/backend.log' as PID 2208283