[2024-10-14 15:25:42,260][ INFO][PID:99737] Marking build as starting [2024-10-14 15:25:42,373][ INFO][PID:99737] Trying to allocate VM [2024-10-14 15:33:00,606][ INFO][PID:99737] Allocated host ResallocHost, ticket_id=5382642, hostname=34.230.22.184 [2024-10-14 15:33:00,608][ INFO][PID:99737] Allocating ssh connection to builder [2024-10-14 15:33:00,609][ INFO][PID:99737] Checking that builder machine is OK [2024-10-14 15:33:00,943][ INFO][PID:99737] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.230.22.184 '/usr/bin/test -f /etc/mock/fedora-41-i386.cfg' [2024-10-14 15:33:01,167][ INFO][PID:99737] Filling build.info file with builder info [2024-10-14 15:33:01,168][ INFO][PID:99737] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "xxmitsu", "project_name": "llvm", "project_dirname": "llvm", "submitter": null, "ended_on": null, "started_on": 1728919981.1682608, "submitted_on": null, "status": 3, "chroot": "fedora-41-i386", "arch": "i386", "buildroot_pkgs": [ "git" ], "task_id": "8135373-fedora-41-i386", "build_id": 8135373, "package_name": "libomp", "package_version": "19.0.0-0.1.20240309.git0d6f9bf", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/xxmitsu/llvm/libomp", "git_hash": "e155a1177a62c7ec887c1e6756a58c17d95b1cde", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "xxmitsu/llvm--c7f3a6cc-9c63-477e-a399-292a3e5f96ac", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": true, "bootstrap": "off", "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/xxmitsu/llvm/fedora-41-i386/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_i386" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/xxmitsu/llvm", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/xxmitsu/llvm", "result_dir": "08135373-libomp", "built_packages": "", "id": 8135373, "mockchain_macros": { "copr_username": "xxmitsu", "copr_projectname": "llvm", "vendor": "Fedora Project COPR (xxmitsu/llvm)" } } ] } [2024-10-14 15:33:01,292][ INFO][PID:99737] Sending fedora-messaging bus message in build.start [2024-10-14 15:33:02,049][ INFO][PID:99737] Sending fedora-messaging bus message in chroot.start [2024-10-14 15:33:02,071][ INFO][PID:99737] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8135373-fedora-41-i386 --chroot fedora-41-i386 --detached [2024-10-14 15:33:02,426][ INFO][PID:99737] Downloading the builder-live.log file, attempt 1 [2024-10-14 15:33:02,428][ INFO][PID:99737] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.230.22.184 copr-rpmbuild-log [2024-10-14 15:33:07,434][ INFO][PID:99737] Periodic builder liveness probe: alive [2024-10-14 15:34:01,912][ ERROR][PID:99737] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-10-14 15:34:11,913][WARNING][PID:99737] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-10-14 15:34:21,914][ INFO][PID:99737] Downloading the builder-live.log file, attempt 2 [2024-10-14 15:34:21,915][ INFO][PID:99737] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.230.22.184 copr-rpmbuild-log [2024-10-14 15:35:21,983][ ERROR][PID:99737] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-10-14 15:35:31,987][WARNING][PID:99737] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-10-14 15:35:41,988][ INFO][PID:99737] Downloading the builder-live.log file, attempt 3 [2024-10-14 15:35:41,989][ INFO][PID:99737] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.230.22.184 copr-rpmbuild-log [2024-10-14 15:36:42,057][ ERROR][PID:99737] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-10-14 15:36:52,059][WARNING][PID:99737] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-10-14 15:37:02,060][ INFO][PID:99737] Downloading the builder-live.log file, attempt 4 [2024-10-14 15:37:02,062][ INFO][PID:99737] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@34.230.22.184 copr-rpmbuild-log [2024-10-14 15:38:02,130][ ERROR][PID:99737] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-10-14 15:38:12,143][ ERROR][PID:99737] Giving up for unstable SSH, failures: 1728920051.9133961, 1728920131.987199, 1728920212.0596476, 1728920292.14311 [2024-10-14 15:38:12,145][ ERROR][PID:99737] Re-try request for task on 'ResallocHost, ticket_id=5382642, hostname=34.230.22.184': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2024-10-14 15:38:12,145][ INFO][PID:99737] Releasing VM back to pool [2024-10-14 15:38:12,156][ INFO][PID:99737] Retry #1 (on other host) [2024-10-14 15:38:12,157][ INFO][PID:99737] Trying to allocate VM [2024-10-14 15:45:30,438][ INFO][PID:99737] Allocated host ResallocHost, ticket_id=5382769, hostname=44.211.240.96 [2024-10-14 15:45:30,439][ INFO][PID:99737] Allocating ssh connection to builder [2024-10-14 15:45:30,440][ INFO][PID:99737] Checking that builder machine is OK [2024-10-14 15:45:30,860][ INFO][PID:99737] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.211.240.96 '/usr/bin/test -f /etc/mock/fedora-41-i386.cfg' [2024-10-14 15:45:31,095][ INFO][PID:99737] Filling build.info file with builder info [2024-10-14 15:45:31,101][ INFO][PID:99737] Not re-notifying FE and msg buses for the new host. [2024-10-14 15:45:31,102][ INFO][PID:99737] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8135373-fedora-41-i386 --chroot fedora-41-i386 --detached [2024-10-14 15:45:31,548][ INFO][PID:99737] Downloading the builder-live.log file, attempt 1 [2024-10-14 15:45:31,551][ INFO][PID:99737] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.211.240.96 copr-rpmbuild-log [2024-10-14 15:45:36,555][ INFO][PID:99737] Periodic builder liveness probe: alive [2024-10-14 15:46:21,792][ INFO][PID:99737] Downloading results from builder [2024-10-14 15:46:21,793][ INFO][PID:99737] rsyncing of mockbuilder@44.211.240.96:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/xxmitsu/llvm/fedora-41-i386/08135373-libomp started [2024-10-14 15:46:21,795][ INFO][PID:99737] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@44.211.240.96:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/xxmitsu/llvm/fedora-41-i386/08135373-libomp/ &> /var/lib/copr/public_html/results/xxmitsu/llvm/fedora-41-i386/08135373-libomp/build-08135373.rsync.log [2024-10-14 15:46:23,287][ INFO][PID:99737] rsyncing finished. [2024-10-14 15:46:23,288][ INFO][PID:99737] Releasing VM back to pool [2024-10-14 15:46:23,305][ INFO][PID:99737] Searching for 'success' file in resultdir [2024-10-14 15:46:23,305][ ERROR][PID:99737] Build failed: Backend process error: No success file => build failure [2024-10-14 15:46:23,306][ INFO][PID:99737] Finished build: id=8135373 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/xxmitsu/llvm chroot=fedora-41-i386 [2024-10-14 15:46:23,310][ ERROR][PID:99737] Backend process error: No success file => build failure [2024-10-14 15:46:23,311][ INFO][PID:99737] Worker failed build, took 52.20942950248718 [2024-10-14 15:46:23,311][ INFO][PID:99737] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": true, "project_owner": "xxmitsu", "project_name": "llvm", "project_dirname": "llvm", "submitter": null, "ended_on": 1728920783.3112087, "started_on": 1728920731.1017792, "submitted_on": null, "status": 0, "chroot": "fedora-41-i386", "arch": "i386", "buildroot_pkgs": [ "git" ], "task_id": "8135373-fedora-41-i386", "build_id": 8135373, "package_name": "libomp", "package_version": "19.0.0-0.1.20240309.git0d6f9bf", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/xxmitsu/llvm/libomp", "git_hash": "e155a1177a62c7ec887c1e6756a58c17d95b1cde", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "xxmitsu/llvm--c7f3a6cc-9c63-477e-a399-292a3e5f96ac", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": true, "bootstrap": "off", "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/xxmitsu/llvm/fedora-41-i386/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "tags": [ "arch_i386" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/xxmitsu/llvm", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/xxmitsu/llvm", "result_dir": "08135373-libomp", "built_packages": "", "id": 8135373, "mockchain_macros": { "copr_username": "xxmitsu", "copr_projectname": "llvm", "vendor": "Fedora Project COPR (xxmitsu/llvm)" } } ] } [2024-10-14 15:46:23,378][ INFO][PID:99737] Sending fedora-messaging bus message in build.end [2024-10-14 15:46:23,404][ INFO][PID:99737] Compressing /var/lib/copr/public_html/results/xxmitsu/llvm/fedora-41-i386/08135373-libomp/builder-live.log by gzip [2024-10-14 15:46:23,408][ INFO][PID:99737] Running command 'gzip /var/lib/copr/public_html/results/xxmitsu/llvm/fedora-41-i386/08135373-libomp/builder-live.log' as PID 220024 [2024-10-14 15:46:23,416][ INFO][PID:99737] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/xxmitsu/llvm/fedora-41-i386/08135373-libomp/builder-live.log) [2024-10-14 15:46:23,417][ INFO][PID:99737] Compressing /var/lib/copr/public_html/results/xxmitsu/llvm/fedora-41-i386/08135373-libomp/backend.log by gzip [2024-10-14 15:46:23,419][ INFO][PID:99737] Running command 'gzip /var/lib/copr/public_html/results/xxmitsu/llvm/fedora-41-i386/08135373-libomp/backend.log' as PID 220026