[2025-08-21 14:11:49,217][  INFO][PID:2563449] Marking build as starting
[2025-08-21 14:11:49,267][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:11:49,270][  INFO][PID:2563449] VM allocation process starts
[2025-08-21 14:11:49,313][  INFO][PID:2563449] Trying to allocate VM: ResallocHost, ticket_id=4651668, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-08-21 14:11:52,389][  INFO][PID:2563449] Allocated host ResallocHost, ticket_id=4651668, hostname=140.211.168.61, name=copr_osuosl_p09_prod_06364390_20250821_140246, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-08-21 14:11:52,390][  INFO][PID:2563449] Allocating ssh connection to builder
[2025-08-21 14:11:52,392][  INFO][PID:2563449] Checking that builder machine is OK
[2025-08-21 14:11:53,752][  INFO][PID:2563449] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-08-21 14:11:54,833][  INFO][PID:2563449] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-08-21 14:11:54,834][  INFO][PID:2563449] Filling build.info file with builder info
[2025-08-21 14:11:54,837][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:11:54,838][  INFO][PID:2563449] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 90000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@fedora-llvm-team",
            "project_name": "clang-monthly-fedora-rebuild",
            "project_dirname": "clang-monthly-fedora-rebuild",
            "submitter": "tstellar",
            "ended_on": null,
            "started_on": 1755785514.837963,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [
                "fedora-clang-default-cc"
            ],
            "task_id": "9453123-fedora-rawhide-ppc64le",
            "build_id": 9453123,
            "package_name": "bubblewrap",
            "package_version": "0.11.0-2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/clang-monthly-fedora-rebuild/bubblewrap",
            "git_hash": "f1d786222f0711e09ffa03e898530c0058091cb2",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@fedora-llvm-team/clang-monthly-fedora-rebuild--tstellar",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/tstellar/fedora-clang-default-cc/fedora-rawhide-ppc64le/",
                    "id": "copr_tstellar_fedora_clang_default_cc",
                    "name": "Additional repo copr_tstellar_fedora_clang_default_cc"
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20250813/fedora-rawhide-ppc64le/",
                    "id": "copr_fedora_llvm_team_llvm_snapshots_big_merge_20250813",
                    "module_hotfixes": true,
                    "name": "Additional repo copr_fedora_llvm_team_llvm_snapshots_big_merge_20250813"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [
                "toolchain_clang",
                "clang_lto"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/clang-monthly-fedora-rebuild",
            "result_dir": "09453123-bubblewrap",
            "built_packages": "",
            "id": 9453123,
            "mockchain_macros": {
                "copr_username": "@fedora-llvm-team",
                "copr_projectname": "clang-monthly-fedora-rebuild",
                "vendor": "Fedora Project COPR (@fedora-llvm-team/clang-monthly-fedora-rebuild)"
            }
        }
    ]
}
[2025-08-21 14:11:54,865][  INFO][PID:2563449] Sending fedora-messaging bus message in build.start
[2025-08-21 14:11:55,845][  INFO][PID:2563449] Sending fedora-messaging bus message in chroot.start
[2025-08-21 14:11:55,878][  INFO][PID:2563449] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9453123-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-08-21 14:11:57,173][  INFO][PID:2563449] Downloading the builder-live.log file, attempt 1
[2025-08-21 14:11:57,177][  INFO][PID:2563449] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-08-21 14:12:02,188][  INFO][PID:2563449] Periodic builder liveness probe: alive
[2025-08-21 14:12:02,194][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:07,196][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:12,199][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:17,202][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:22,205][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:27,206][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:32,209][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:37,217][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:42,224][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:47,230][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:52,234][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:12:57,236][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:02,239][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:07,240][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:12,243][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:17,248][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:22,255][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:27,257][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:32,258][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:37,260][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:42,262][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:47,265][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:52,271][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:13:57,272][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:02,273][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:07,278][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:12,279][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:17,280][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:22,285][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:27,288][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:32,292][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:37,295][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:42,297][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:47,304][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:52,309][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:14:57,311][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:02,314][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:07,318][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:12,320][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:17,325][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:22,327][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:27,332][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:32,337][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:37,352][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:42,354][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:47,356][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:52,366][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:15:57,370][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:02,372][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:07,373][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:12,375][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:17,376][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:22,378][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:27,380][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:32,381][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:37,383][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:42,384][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:47,385][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:52,387][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:16:57,388][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:02,389][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:07,390][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:12,391][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:17,392][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:22,393][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:27,395][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:32,396][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:37,397][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:42,398][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:47,399][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:52,400][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:17:57,402][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:02,403][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:07,404][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:12,405][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:17,406][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:22,407][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:27,408][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:32,409][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:37,410][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:42,411][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:47,412][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:52,413][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:18:57,414][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:02,415][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:07,417][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:12,418][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:17,419][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:22,420][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:27,421][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:32,422][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:37,423][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:42,424][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:47,425][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:52,426][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:19:57,428][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:02,429][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:07,430][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:12,431][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:17,432][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:22,433][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:27,434][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:32,435][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:37,437][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:42,438][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:47,439][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:52,440][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:20:57,442][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:02,443][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:07,444][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:12,445][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:17,446][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:22,447][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:27,451][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:32,452][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:37,454][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:42,455][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:47,456][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:52,458][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:21:57,460][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:02,465][  INFO][PID:2563449] Periodic builder liveness probe: alive
[2025-08-21 14:22:02,466][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:07,467][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:12,468][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:17,469][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:22,470][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:27,471][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:32,472][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:37,473][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:42,474][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:47,475][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:52,476][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:22:57,478][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:02,479][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:07,480][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:12,481][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:17,482][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:22,483][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:27,484][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:32,485][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:37,486][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:42,487][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:47,488][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:52,489][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:23:57,491][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:02,492][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:07,494][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:12,495][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:17,496][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:22,497][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:27,498][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:32,499][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:37,501][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:42,502][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:47,503][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:52,504][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:24:57,505][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:02,506][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:07,512][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:12,514][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:17,517][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:22,518][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:27,519][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:32,521][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:37,522][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:42,523][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:47,525][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:52,526][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:25:57,527][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:02,528][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:07,529][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:12,530][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:17,531][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:22,532][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:27,533][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:32,534][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:37,535][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:42,536][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:47,538][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:52,549][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:26:57,550][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:02,552][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:07,553][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:12,554][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:17,555][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:22,556][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:27,560][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:32,561][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:37,568][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:42,572][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:47,575][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:52,581][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:27:57,585][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:02,587][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:07,588][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:12,592][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:17,594][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:22,595][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:27,596][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:32,597][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:37,599][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:42,600][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:47,601][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:52,607][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:28:57,608][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:02,609][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:07,610][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:12,612][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:17,613][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:22,614][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:27,615][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:32,616][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:37,617][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:42,619][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:47,620][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:52,621][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:29:57,624][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:02,625][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:07,626][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:12,627][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:17,628][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:22,629][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:27,630][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:32,632][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:37,633][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:42,634][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:47,635][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:52,636][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:30:57,637][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:02,638][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:07,640][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:12,640][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:17,642][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:22,643][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:27,644][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:32,645][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:37,646][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:42,647][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:47,648][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:52,649][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:31:57,650][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:02,658][  INFO][PID:2563449] Periodic builder liveness probe: alive
[2025-08-21 14:32:02,659][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:07,660][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:12,661][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:17,662][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:22,663][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:27,667][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:32,668][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:37,669][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:42,670][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:47,672][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:52,673][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:32:57,674][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:02,675][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:07,676][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:12,679][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:17,682][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:22,684][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:27,685][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:32,686][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:37,687][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:42,688][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:47,689][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:52,690][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:33:57,691][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:02,692][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:07,693][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:12,694][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:17,695][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:22,697][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:27,698][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:32,699][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:37,700][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:42,701][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:47,702][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:52,703][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:34:57,704][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:02,705][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:07,706][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:12,707][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:17,708][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:22,710][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:27,711][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:32,712][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:37,713][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:42,714][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:47,715][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:52,716][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:35:57,717][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:02,718][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:07,720][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:12,721][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:17,722][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:22,723][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:27,724][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:32,725][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:37,726][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:42,728][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:47,729][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:52,730][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:36:57,731][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:02,732][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:07,734][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:12,735][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:17,736][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:22,737][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:27,738][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:32,741][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:37,743][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:42,744][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:47,745][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:52,746][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:37:57,748][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:02,749][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:07,750][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:12,753][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:17,754][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:22,755][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:27,756][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:32,757][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:37,762][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:42,763][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:47,764][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:52,765][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:38:57,767][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:02,768][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:07,769][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:12,771][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:17,772][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:22,773][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:27,775][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:32,777][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:37,779][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:42,780][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:47,781][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:52,783][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:39:57,784][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:02,785][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:07,786][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:12,788][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:17,789][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:22,790][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:27,792][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:32,793][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:37,794][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:42,796][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:47,797][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:52,798][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:40:57,799][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:02,800][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:07,802][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:12,803][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:17,804][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:22,805][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:27,806][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:32,807][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:37,808][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:42,810][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:47,811][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:52,812][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:41:57,813][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:02,868][  INFO][PID:2563449] Periodic builder liveness probe: alive
[2025-08-21 14:42:02,869][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:07,870][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:12,871][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:17,872][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:22,874][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:27,875][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:32,876][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:37,878][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:42,879][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:47,880][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:52,882][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:42:57,883][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:02,884][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:07,885][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:12,886][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:17,887][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:22,888][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:27,890][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:32,891][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:37,892][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:42,893][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:47,894][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:52,895][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:43:57,896][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:02,898][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:07,905][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:12,907][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:17,908][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:22,910][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:27,911][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:32,912][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:37,913][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:42,914][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:47,921][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:52,928][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:44:57,929][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:02,930][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:07,931][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:12,932][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:17,933][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:22,934][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:27,936][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:32,937][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:37,938][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:42,940][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:47,941][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:52,943][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:45:57,944][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:02,947][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:07,948][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:12,949][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:17,952][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:22,954][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:27,955][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:32,957][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:37,960][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:42,961][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:47,962][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:52,964][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:46:57,966][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:02,969][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:07,972][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:12,975][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:17,977][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:22,980][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:27,982][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:32,985][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:37,988][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:42,990][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:47,995][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:52,997][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:47:57,998][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:03,000][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:08,001][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:13,002][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:18,004][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:23,005][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:24,139][ ERROR][PID:2563449] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-21 14:48:28,006][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:33,011][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:34,140][WARNING][PID:2563449] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-21 14:48:38,012][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:43,014][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:44,141][  INFO][PID:2563449] Downloading the builder-live.log file, attempt 2
[2025-08-21 14:48:44,143][  INFO][PID:2563449] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-08-21 14:48:48,015][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:53,018][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:48:58,034][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:03,035][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:08,036][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:13,038][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:18,039][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:23,040][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:28,042][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:33,043][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:38,044][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:43,046][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:44,215][ ERROR][PID:2563449] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-21 14:49:48,047][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:53,048][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:49:54,220][WARNING][PID:2563449] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-21 14:49:58,050][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:03,051][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:04,221][  INFO][PID:2563449] Downloading the builder-live.log file, attempt 3
[2025-08-21 14:50:04,222][  INFO][PID:2563449] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-08-21 14:50:08,052][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:13,053][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:18,054][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:23,055][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:28,056][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:33,057][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:38,059][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:43,060][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:48,061][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:53,062][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:50:58,063][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:03,064][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:04,290][ ERROR][PID:2563449] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-21 14:51:08,065][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:13,066][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:14,292][WARNING][PID:2563449] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-08-21 14:51:18,067][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:23,068][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:24,293][  INFO][PID:2563449] Downloading the builder-live.log file, attempt 4
[2025-08-21 14:51:24,294][  INFO][PID:2563449] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@140.211.168.61 copr-rpmbuild-log
[2025-08-21 14:51:28,069][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:33,070][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:38,071][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:43,072][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:48,073][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:53,074][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:51:58,075][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:52:03,082][  INFO][PID:2563449] Periodic builder liveness probe: dead
[2025-08-21 14:52:24,358][ ERROR][PID:2563449] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-08-21 14:52:34,359][ ERROR][PID:2563449] Giving up for unstable SSH, failures: 1755787714.1406426, 1755787794.2203329, 1755787874.292665, 1755787954.3590612
[2025-08-21 14:53:03,147][ ERROR][PID:2563449] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-08-21 14:54:13,216][ ERROR][PID:2563449] SSH connection lost on #2 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-08-21 14:55:23,286][ ERROR][PID:2563449] SSH connection lost on #3 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-08-21 14:56:33,356][ ERROR][PID:2563449] SSH connection lost on #4 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 140.211.168.61 port 22: Connection timed out

[2025-08-21 14:56:43,357][ ERROR][PID:2563449] Can't ssh to cancel build.
[2025-08-21 14:56:43,358][ ERROR][PID:2563449] Re-try request for task on 'ResallocHost, ticket_id=4651668, hostname=140.211.168.61, name=copr_osuosl_p09_prod_06364390_20250821_140246, requested_tags=['copr_builder', 'arch_ppc64le']': SSH problems when downloading live log: Resalloc ticket FAILED
[2025-08-21 14:56:43,358][  INFO][PID:2563449] Releasing VM back to pool
[2025-08-21 14:56:43,367][  INFO][PID:2563449] Retry #1 (on other host)
[2025-08-21 14:56:43,368][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:56:43,368][  INFO][PID:2563449] VM allocation process starts
[2025-08-21 14:56:43,412][  INFO][PID:2563449] Trying to allocate VM: ResallocHost, ticket_id=4652291, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-08-21 14:56:46,452][  INFO][PID:2563449] Allocated host ResallocHost, ticket_id=4652291, hostname=2620:52:3:1:dead:beef:cafe:c28f, name=copr_p09_01_prod_06365064_20250821_144654, requested_tags=['copr_builder', 'arch_ppc64le']
[2025-08-21 14:56:46,452][  INFO][PID:2563449] Allocating ssh connection to builder
[2025-08-21 14:56:46,453][  INFO][PID:2563449] Checking that builder machine is OK
[2025-08-21 14:56:47,438][  INFO][PID:2563449] Running remote command: copr-builder-ready fedora-rawhide-ppc64le
[2025-08-21 14:56:48,009][  INFO][PID:2563449] Red Hat subscription not needed for fedora-rawhide-ppc64le
Builder is ready to be used

[2025-08-21 14:56:48,010][  INFO][PID:2563449] Filling build.info file with builder info
[2025-08-21 14:56:48,014][  INFO][PID:2563449] Checking for cancel request
[2025-08-21 14:56:48,015][  INFO][PID:2563449] Not re-notifying FE and msg buses for the new host.
[2025-08-21 14:56:48,016][  INFO][PID:2563449] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9453123-fedora-rawhide-ppc64le --chroot fedora-rawhide-ppc64le --detached
[2025-08-21 14:56:49,018][  INFO][PID:2563449] Downloading the builder-live.log file, attempt 1
[2025-08-21 14:56:49,019][  INFO][PID:2563449] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c28f copr-rpmbuild-log
[2025-08-21 14:56:57,886][  INFO][PID:2563449] Cancel request succeeded
out:
success
err:
Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c28f' (ED25519) to the list of known hosts.

[2025-08-21 14:56:57,886][  INFO][PID:2563449] Downloading results from builder
[2025-08-21 14:56:57,887][  INFO][PID:2563449] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c28f]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/09453123-bubblewrap started
[2025-08-21 14:56:57,888][  INFO][PID:2563449] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c28f]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/09453123-bubblewrap/ &> /var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/09453123-bubblewrap/build-09453123.rsync.log
[2025-08-21 14:56:58,381][  INFO][PID:2563449] rsyncing finished.
[2025-08-21 14:56:58,381][  INFO][PID:2563449] Releasing VM back to pool
[2025-08-21 14:56:58,391][  INFO][PID:2563449] Searching for 'success' file in resultdir
[2025-08-21 14:56:58,391][ ERROR][PID:2563449] Build failed: Backend process error: No success file => build failure
[2025-08-21 14:56:58,392][  INFO][PID:2563449] Finished build: id=9453123 failed=True timeout=90000 destdir=/var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild chroot=fedora-rawhide-ppc64le 
[2025-08-21 14:56:58,394][ ERROR][PID:2563449] Backend process error: No success file => build failure
[2025-08-21 14:56:58,394][  INFO][PID:2563449] Worker failed build, took 10.379120826721191
[2025-08-21 14:56:58,395][  INFO][PID:2563449] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 90000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "@fedora-llvm-team",
            "project_name": "clang-monthly-fedora-rebuild",
            "project_dirname": "clang-monthly-fedora-rebuild",
            "submitter": "tstellar",
            "ended_on": 1755788218.3944504,
            "started_on": 1755788208.0153296,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [
                "fedora-clang-default-cc"
            ],
            "task_id": "9453123-fedora-rawhide-ppc64le",
            "build_id": 9453123,
            "package_name": "bubblewrap",
            "package_version": "0.11.0-2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/clang-monthly-fedora-rebuild/bubblewrap",
            "git_hash": "f1d786222f0711e09ffa03e898530c0058091cb2",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@fedora-llvm-team/clang-monthly-fedora-rebuild--tstellar",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/tstellar/fedora-clang-default-cc/fedora-rawhide-ppc64le/",
                    "id": "copr_tstellar_fedora_clang_default_cc",
                    "name": "Additional repo copr_tstellar_fedora_clang_default_cc"
                },
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-big-merge-20250813/fedora-rawhide-ppc64le/",
                    "id": "copr_fedora_llvm_team_llvm_snapshots_big_merge_20250813",
                    "module_hotfixes": true,
                    "name": "Additional repo copr_fedora_llvm_team_llvm_snapshots_big_merge_20250813"
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [
                "toolchain_clang",
                "clang_lto"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/clang-monthly-fedora-rebuild",
            "result_dir": "09453123-bubblewrap",
            "built_packages": "",
            "id": 9453123,
            "mockchain_macros": {
                "copr_username": "@fedora-llvm-team",
                "copr_projectname": "clang-monthly-fedora-rebuild",
                "vendor": "Fedora Project COPR (@fedora-llvm-team/clang-monthly-fedora-rebuild)"
            }
        }
    ]
}
[2025-08-21 14:56:58,421][  INFO][PID:2563449] Sending fedora-messaging bus message in build.end
[2025-08-21 14:56:58,445][  INFO][PID:2563449] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/09453123-bubblewrap/builder-live.log by gzip
[2025-08-21 14:56:58,447][  INFO][PID:2563449] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/09453123-bubblewrap/builder-live.log' as PID 2850486
[2025-08-21 14:56:58,448][  INFO][PID:2563449] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/09453123-bubblewrap/builder-live.log)
[2025-08-21 14:56:58,449][  INFO][PID:2563449] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/09453123-bubblewrap/backend.log by gzip
[2025-08-21 14:56:58,450][  INFO][PID:2563449] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/clang-monthly-fedora-rebuild/fedora-rawhide-ppc64le/09453123-bubblewrap/backend.log' as PID 2850487