[2025-11-17 15:54:51,718][ INFO][PID:4144534] Marking build as starting [2025-11-17 15:54:51,759][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:54:51,760][ INFO][PID:4144534] VM allocation process starts [2025-11-17 15:54:51,782][ INFO][PID:4144534] Trying to allocate VM: ResallocHost, ticket_id=6219709, requested_tags=['copr_builder', 'arch_x86_64'] [2025-11-17 15:54:56,783][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:01,784][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:06,785][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:11,786][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:16,788][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:21,789][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:26,790][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:31,791][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:36,792][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:41,793][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:46,794][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:51,795][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:55:56,797][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:01,798][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:06,800][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:11,801][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:16,802][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:21,805][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:26,808][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:31,809][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:36,810][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:41,813][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:46,815][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:51,818][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:56:56,819][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:01,820][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:06,821][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:11,823][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:16,825][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:21,826][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:26,827][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:31,828][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:36,829][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:41,830][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:46,832][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:51,833][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:57:56,834][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:01,836][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:06,837][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:11,838][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:16,839][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:21,840][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:26,841][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:31,842][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:36,843][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:41,844][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:46,845][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:51,846][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:58:56,847][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:01,849][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:06,851][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:11,852][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:16,852][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:21,854][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:26,854][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:31,856][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:36,857][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:41,859][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:46,860][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:51,861][ INFO][PID:4144534] Checking for cancel request [2025-11-17 15:59:56,862][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:01,864][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:06,866][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:11,868][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:16,869][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:21,871][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:26,873][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:31,874][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:36,882][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:41,883][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:46,885][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:51,887][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:00:56,888][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:01,890][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:06,892][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:11,893][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:16,896][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:21,905][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:26,906][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:31,909][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:36,910][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:41,911][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:46,914][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:51,915][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:01:56,916][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:02:01,918][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:02:06,919][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:02:09,948][ INFO][PID:4144534] Allocated host ResallocHost, ticket_id=6219709, hostname=2620:52:3:1:dead:beef:cafe:c1d9, name=copr_hv_x86_64_04_prod_08057848_20251117_155940, requested_tags=['copr_builder', 'arch_x86_64'] [2025-11-17 16:02:09,948][ INFO][PID:4144534] Allocating ssh connection to builder [2025-11-17 16:02:09,949][ INFO][PID:4144534] Checking that builder machine is OK [2025-11-17 16:02:10,516][ INFO][PID:4144534] Running remote command: copr-builder-ready fedora-41-x86_64 [2025-11-17 16:02:10,849][ INFO][PID:4144534] Red Hat subscription not needed for fedora-41-x86_64 Builder is ready to be used [2025-11-17 16:02:10,850][ INFO][PID:4144534] Filling build.info file with builder info [2025-11-17 16:02:10,851][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:02:10,852][ INFO][PID:4144534] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "networkmanager", "project_name": "NetworkManager-CI-1.28-git", "project_dirname": "NetworkManager-CI-1.28-git", "submitter": null, "ended_on": null, "started_on": 1763395330.8525567, "submitted_on": null, "status": 3, "chroot": "fedora-41-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9805523-fedora-41-x86_64", "build_id": 9805523, "package_name": "NetworkManager", "package_version": "1:1.28.3-27036.copr.ec4f7d540d", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/networkmanager/NetworkManager-CI-1.28-git/NetworkManager", "git_hash": "1e29869df24d893f188b941ef49535d7b3a9781d", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "networkmanager/NetworkManager-CI-1.28-git--326f2e28-08a4-46b3-9331-f074edd845cc", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "bootstrap": "off", "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-CI-1.28-git", "result_dir": "09805523-NetworkManager", "built_packages": "", "id": 9805523, "mockchain_macros": { "copr_username": "networkmanager", "copr_projectname": "NetworkManager-CI-1.28-git", "vendor": "Fedora Project COPR (networkmanager/NetworkManager-CI-1.28-git)" } } ] } [2025-11-17 16:02:10,909][ INFO][PID:4144534] Sending fedora-messaging bus message in build.start [2025-11-17 16:02:32,236][ ERROR][PID:4144534] Attempt 1 to publish a message failed (in /usr/lib/python3.13/site-packages/fedora_messaging/twisted/factory.py:188) [2025-11-17 16:02:57,217][ INFO][PID:4144534] Sending fedora-messaging bus message in chroot.start [2025-11-17 16:02:57,340][ INFO][PID:4144534] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9805523-fedora-41-x86_64 --chroot fedora-41-x86_64 --detached [2025-11-17 16:02:58,339][ INFO][PID:4144534] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c1d9' (ED25519) to the list of known hosts. [2025-11-17 16:02:58,343][ INFO][PID:4144534] Downloading the builder-live.log file, attempt 1 [2025-11-17 16:02:58,350][ INFO][PID:4144534] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c1d9 copr-rpmbuild-log [2025-11-17 16:03:03,362][ INFO][PID:4144534] Periodic builder liveness probe: alive [2025-11-17 16:03:03,362][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:08,363][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:13,365][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:18,366][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:23,367][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:28,369][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:33,371][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:38,372][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:43,374][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:48,375][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:53,377][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:03:58,378][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:03,379][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:08,380][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:13,381][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:18,383][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:23,386][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:28,387][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:33,388][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:38,390][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:43,391][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:48,394][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:53,395][ INFO][PID:4144534] Checking for cancel request [2025-11-17 16:04:53,583][ INFO][PID:4144534] Downloading results from builder [2025-11-17 16:04:53,584][ INFO][PID:4144534] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c1d9]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/09805523-NetworkManager started [2025-11-17 16:04:53,585][ INFO][PID:4144534] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c1d9]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/09805523-NetworkManager/ &> /var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/09805523-NetworkManager/build-09805523.rsync.log [2025-11-17 16:04:54,063][ INFO][PID:4144534] rsyncing finished. [2025-11-17 16:04:54,063][ INFO][PID:4144534] Releasing VM back to pool [2025-11-17 16:04:54,076][ INFO][PID:4144534] Searching for 'success' file in resultdir [2025-11-17 16:04:54,076][ ERROR][PID:4144534] Build failed: Backend process error: No success file => build failure [2025-11-17 16:04:54,077][ INFO][PID:4144534] Finished build: id=9805523 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git chroot=fedora-41-x86_64 [2025-11-17 16:04:54,078][ ERROR][PID:4144534] Backend process error: No success file => build failure [2025-11-17 16:04:54,079][ INFO][PID:4144534] Worker failed build, took 163.22682905197144 [2025-11-17 16:04:54,079][ INFO][PID:4144534] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "networkmanager", "project_name": "NetworkManager-CI-1.28-git", "project_dirname": "NetworkManager-CI-1.28-git", "submitter": null, "ended_on": 1763395494.0793858, "started_on": 1763395330.8525567, "submitted_on": null, "status": 0, "chroot": "fedora-41-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "9805523-fedora-41-x86_64", "build_id": 9805523, "package_name": "NetworkManager", "package_version": "1:1.28.3-27036.copr.ec4f7d540d", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/networkmanager/NetworkManager-CI-1.28-git/NetworkManager", "git_hash": "1e29869df24d893f188b941ef49535d7b3a9781d", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "networkmanager/NetworkManager-CI-1.28-git--326f2e28-08a4-46b3-9331-f074edd845cc", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "bootstrap": "off", "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/networkmanager/NetworkManager-CI-1.28-git", "result_dir": "09805523-NetworkManager", "built_packages": "", "id": 9805523, "mockchain_macros": { "copr_username": "networkmanager", "copr_projectname": "NetworkManager-CI-1.28-git", "vendor": "Fedora Project COPR (networkmanager/NetworkManager-CI-1.28-git)" } } ] } [2025-11-17 16:04:54,147][ INFO][PID:4144534] Sending fedora-messaging bus message in build.end [2025-11-17 16:05:24,153][ ERROR][PID:4144534] Attempt 1 to publish a message failed (in /usr/lib/python3.13/site-packages/fedora_messaging/api.py:346) [2025-11-17 16:05:54,157][ ERROR][PID:4144534] Attempt 2 to publish a message failed (in /usr/lib/python3.13/site-packages/fedora_messaging/api.py:346) [2025-11-17 16:06:24,162][ ERROR][PID:4144534] Attempt 3 to publish a message failed (in /usr/lib/python3.13/site-packages/fedora_messaging/api.py:346) [2025-11-17 16:06:54,166][ ERROR][PID:4144534] Attempt 4 to publish a message failed (in /usr/lib/python3.13/site-packages/fedora_messaging/api.py:346) [2025-11-17 16:07:24,173][ ERROR][PID:4144534] Attempt 5 to publish a message failed (in /usr/lib/python3.13/site-packages/fedora_messaging/api.py:346) [2025-11-17 16:07:24,186][ INFO][PID:4144534] Compressing /var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/09805523-NetworkManager/builder-live.log by gzip [2025-11-17 16:07:24,188][ INFO][PID:4144534] Running command 'gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/09805523-NetworkManager/builder-live.log' as PID 48079 [2025-11-17 16:07:24,204][ INFO][PID:4144534] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/09805523-NetworkManager/builder-live.log) [2025-11-17 16:07:24,204][ INFO][PID:4144534] Compressing /var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/09805523-NetworkManager/backend.log by gzip [2025-11-17 16:07:24,206][ INFO][PID:4144534] Running command 'gzip /var/lib/copr/public_html/results/networkmanager/NetworkManager-CI-1.28-git/fedora-41-x86_64/09805523-NetworkManager/backend.log' as PID 48082