[2024-11-22 19:24:01,851][ INFO][PID:14626] Marking build as starting [2024-11-22 19:24:01,959][ INFO][PID:14626] Trying to allocate VM [2024-11-22 19:25:10,074][ INFO][PID:14626] Allocated host ResallocHost, ticket_id=6024701, hostname=52.23.181.109 [2024-11-22 19:25:10,075][ INFO][PID:14626] Allocating ssh connection to builder [2024-11-22 19:25:10,076][ INFO][PID:14626] Checking that builder machine is OK [2024-11-22 19:25:10,415][ INFO][PID:14626] Running remote command: copr-builder-ready fedora-39-x86_64 [2024-11-22 19:25:10,639][ INFO][PID:14626] Red Hat subscription not needed for fedora-39-x86_64 Builder is ready to be used [2024-11-22 19:25:10,640][ INFO][PID:14626] Filling build.info file with builder info [2024-11-22 19:25:10,642][ INFO][PID:14626] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@python", "project_name": "python3.12", "project_dirname": "python3.12:pr:12", "submitter": "https://src.fedoraproject.org/user/churchyard", "ended_on": null, "started_on": 1732303510.6415982, "submitted_on": null, "status": 3, "chroot": "fedora-39-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "8302640-fedora-39-x86_64", "build_id": 8302640, "package_name": "python-pexpect", "package_version": "4.9.0-7", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@python/python3.12:pr:12/python-pexpect", "git_hash": "8b38b6e8b1b550dd94b98f230da779ea1ed656f0", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@python/python3.12--https://src.fedoraproject.org/user/churchyard", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": true, "bootstrap": "off", "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@python/python3.12/fedora-39-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@python/python3.12:pr:12", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@python/python3.12:pr:12", "result_dir": "08302640-python-pexpect", "built_packages": "", "id": 8302640, "mockchain_macros": { "copr_username": "@python", "copr_projectname": "python3.12", "vendor": "Fedora Project COPR (@python/python3.12)" } } ] } [2024-11-22 19:25:10,827][ INFO][PID:14626] Sending fedora-messaging bus message in build.start [2024-11-22 19:25:11,792][ INFO][PID:14626] Sending fedora-messaging bus message in chroot.start [2024-11-22 19:25:11,822][ INFO][PID:14626] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8302640-fedora-39-x86_64 --chroot fedora-39-x86_64 --detached [2024-11-22 19:25:12,166][ INFO][PID:14626] Downloading the builder-live.log file, attempt 1 [2024-11-22 19:25:12,168][ INFO][PID:14626] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.23.181.109 copr-rpmbuild-log [2024-11-22 19:25:17,177][ INFO][PID:14626] Periodic builder liveness probe: alive [2024-11-22 19:34:07,658][ ERROR][PID:14626] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-11-22 19:34:17,658][WARNING][PID:14626] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-11-22 19:34:27,662][ INFO][PID:14626] Downloading the builder-live.log file, attempt 2 [2024-11-22 19:34:27,665][ INFO][PID:14626] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.23.181.109 copr-rpmbuild-log [2024-11-22 19:35:17,282][ INFO][PID:14626] Periodic builder liveness probe: alive [2024-11-22 19:35:27,733][ ERROR][PID:14626] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-11-22 19:35:37,735][WARNING][PID:14626] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-11-22 19:35:47,736][ INFO][PID:14626] Downloading the builder-live.log file, attempt 3 [2024-11-22 19:35:47,737][ INFO][PID:14626] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.23.181.109 copr-rpmbuild-log [2024-11-22 19:36:47,804][ ERROR][PID:14626] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-11-22 19:36:57,816][WARNING][PID:14626] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts [2024-11-22 19:37:07,816][ INFO][PID:14626] Downloading the builder-live.log file, attempt 4 [2024-11-22 19:37:07,818][ INFO][PID:14626] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.23.181.109 copr-rpmbuild-log [2024-11-22 19:38:07,880][ ERROR][PID:14626] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke. [2024-11-22 19:38:17,884][ ERROR][PID:14626] Giving up for unstable SSH, failures: 1732304057.6589036, 1732304137.735528, 1732304217.8159726, 1732304297.8844323 [2024-11-22 19:38:17,885][ ERROR][PID:14626] Re-try request for task on 'ResallocHost, ticket_id=6024701, hostname=52.23.181.109': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts [2024-11-22 19:38:17,886][ INFO][PID:14626] Releasing VM back to pool [2024-11-22 19:38:17,896][ INFO][PID:14626] Retry #1 (on other host) [2024-11-22 19:38:17,897][ INFO][PID:14626] Trying to allocate VM [2024-11-22 19:38:20,920][ INFO][PID:14626] Allocated host ResallocHost, ticket_id=6024828, hostname=44.211.23.23 [2024-11-22 19:38:20,920][ INFO][PID:14626] Allocating ssh connection to builder [2024-11-22 19:38:20,921][ INFO][PID:14626] Checking that builder machine is OK [2024-11-22 19:38:21,246][ INFO][PID:14626] Running remote command: copr-builder-ready fedora-39-x86_64 [2024-11-22 19:38:21,446][ INFO][PID:14626] Red Hat subscription not needed for fedora-39-x86_64 Builder is ready to be used [2024-11-22 19:38:21,447][ INFO][PID:14626] Filling build.info file with builder info [2024-11-22 19:38:21,463][ INFO][PID:14626] Not re-notifying FE and msg buses for the new host. [2024-11-22 19:38:21,464][ INFO][PID:14626] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8302640-fedora-39-x86_64 --chroot fedora-39-x86_64 --detached [2024-11-22 19:38:21,806][ INFO][PID:14626] Downloading the builder-live.log file, attempt 1 [2024-11-22 19:38:21,808][ INFO][PID:14626] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.211.23.23 copr-rpmbuild-log [2024-11-22 19:42:56,085][ INFO][PID:14626] Downloading results from builder [2024-11-22 19:42:56,096][ INFO][PID:14626] rsyncing of mockbuilder@44.211.23.23:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect started [2024-11-22 19:42:56,108][ INFO][PID:14626] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@44.211.23.23:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/ &> /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/build-08302640.rsync.log [2024-11-22 19:42:56,451][ INFO][PID:14626] rsyncing finished. [2024-11-22 19:42:56,452][ INFO][PID:14626] Releasing VM back to pool [2024-11-22 19:42:56,481][ INFO][PID:14626] Searching for 'success' file in resultdir [2024-11-22 19:42:56,485][ INFO][PID:14626] Going to sign pkgs from source: 8302640-fedora-39-x86_64 in chroot: /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64 [2024-11-22 19:42:56,488][ INFO][PID:14626] Calling '/bin/sign -u @python#python3.12@copr.fedorahosted.org -p' (attempt #1) [2024-11-22 19:42:56,814][ INFO][PID:14626] Calling '/bin/sign -4 -h sha256 -u @python#python3.12@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/python3-pexpect-4.9.0-7.fc42.noarch.rpm' (attempt #1) [2024-11-22 19:42:57,699][ INFO][PID:14626] signed rpm: /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/python3-pexpect-4.9.0-7.fc42.noarch.rpm [2024-11-22 19:42:57,702][ INFO][PID:14626] Calling '/bin/sign -4 -h sha256 -u @python#python3.12@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/python-pexpect-4.9.0-7.fc42.src.rpm' (attempt #1) [2024-11-22 19:42:58,584][ INFO][PID:14626] signed rpm: /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/python-pexpect-4.9.0-7.fc42.src.rpm [2024-11-22 19:42:58,585][ INFO][PID:14626] Sign done [2024-11-22 19:42:58,585][ INFO][PID:14626] Incremental createrepo run, adding 08302640-python-pexpect into https://download.copr.fedorainfracloud.org/results/@python/python3.12/fedora-39-x86_64, (auto-create-repo=True) [2024-11-22 19:42:58,587][ INFO][PID:14626] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64 --add 08302640-python-pexpect --no-appstream-metadata' as PID 116071 [2024-11-22 19:43:00,014][ INFO][PID:14626] Finished after 1 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64 --add 08302640-python-pexpect --no-appstream-metadata) [2024-11-22 19:43:00,017][ INFO][PID:14626] Getting build details [2024-11-22 19:43:00,019][ INFO][PID:14626] Listing built binary packages in /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect [2024-11-22 19:43:00,022][ INFO][PID:14626] Built packages: ['python3-pexpect 4.9.0'] [2024-11-22 19:43:00,022][ INFO][PID:14626] build details: {'built_packages': 'python3-pexpect 4.9.0'} [2024-11-22 19:43:00,024][ INFO][PID:14626] Retrieving pubkey [2024-11-22 19:43:00,029][ INFO][PID:14626] Calling '/bin/sign -u @python#python3.12@copr.fedorahosted.org -p' (attempt #1) [2024-11-22 19:43:00,353][ INFO][PID:14626] Added pubkey for user @python project python3.12 into: /var/lib/copr/public_html/results/@python/python3.12:pr:12/pubkey.gpg [2024-11-22 19:43:00,355][ INFO][PID:14626] Finished build: id=8302640 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@python/python3.12:pr:12 chroot=fedora-39-x86_64 [2024-11-22 19:43:00,361][ INFO][PID:14626] Worker succeeded build, took 278.897652387619 [2024-11-22 19:43:00,362][ INFO][PID:14626] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "@python", "project_name": "python3.12", "project_dirname": "python3.12:pr:12", "submitter": "https://src.fedoraproject.org/user/churchyard", "ended_on": 1732304580.361472, "started_on": 1732304301.4638195, "submitted_on": null, "status": 1, "chroot": "fedora-39-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "8302640-fedora-39-x86_64", "build_id": 8302640, "package_name": "python-pexpect", "package_version": "4.9.0-7", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@python/python3.12:pr:12/python-pexpect", "git_hash": "8b38b6e8b1b550dd94b98f230da779ea1ed656f0", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "@python/python3.12--https://src.fedoraproject.org/user/churchyard", "results": { "packages": [ { "name": "python-pexpect", "epoch": null, "version": "4.9.0", "release": "7.fc42", "arch": "src" }, { "name": "python3-pexpect", "epoch": null, "version": "4.9.0", "release": "7.fc42", "arch": "noarch" } ] }, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": null, "background": true, "bootstrap": "off", "fedora_review": false, "isolation": "default", "repo_priority": null, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/@python/python3.12/fedora-39-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null }, { "baseurl": "http://kojipkgs.fedoraproject.org/repos/rawhide/latest/$basearch/", "id": "http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch", "name": "Additional repo http_kojipkgs_fedoraproject_org_repos_rawhide_latest_basearch" } ], "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/@python/python3.12:pr:12", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@python/python3.12:pr:12", "result_dir": "08302640-python-pexpect", "built_packages": "python3-pexpect 4.9.0", "id": 8302640, "mockchain_macros": { "copr_username": "@python", "copr_projectname": "python3.12", "vendor": "Fedora Project COPR (@python/python3.12)" } } ] } [2024-11-22 19:43:00,541][ INFO][PID:14626] Sending fedora-messaging bus message in build.end [2024-11-22 19:43:00,574][ INFO][PID:14626] Compressing /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/builder-live.log by gzip [2024-11-22 19:43:00,583][ INFO][PID:14626] Running command 'gzip /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/builder-live.log' as PID 116429 [2024-11-22 19:43:00,589][ INFO][PID:14626] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/builder-live.log) [2024-11-22 19:43:00,590][ INFO][PID:14626] Compressing /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/backend.log by gzip [2024-11-22 19:43:00,594][ INFO][PID:14626] Running command 'gzip /var/lib/copr/public_html/results/@python/python3.12:pr:12/fedora-39-x86_64/08302640-python-pexpect/backend.log' as PID 116430