[2025-01-14 18:15:00,536][ INFO][PID:1135345] Marking build as starting
[2025-01-14 18:15:00,581][ INFO][PID:1135345] Trying to allocate VM
[2025-01-14 18:15:03,673][ INFO][PID:1135345] Allocated host ResallocHost, ticket_id=673725, hostname=98.80.140.66, name=aws_x86_64_spot_prod_00551487_20250114_173009
[2025-01-14 18:15:03,674][ INFO][PID:1135345] Allocating ssh connection to builder
[2025-01-14 18:15:03,675][ INFO][PID:1135345] Checking that builder machine is OK
[2025-01-14 18:15:04,135][ INFO][PID:1135345] Running remote command: copr-builder-ready fedora-rawhide-x86_64
[2025-01-14 18:15:04,406][ INFO][PID:1135345] Red Hat subscription not needed for fedora-rawhide-x86_64
Builder is ready to be used
[2025-01-14 18:15:04,413][ INFO][PID:1135345] Filling build.info file with builder info
[2025-01-14 18:15:04,415][ INFO][PID:1135345] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "@copr",
"project_name": "PyPI",
"project_dirname": "PyPI",
"submitter": "ksurma",
"ended_on": null,
"started_on": 1736878504.4149113,
"submitted_on": null,
"status": 3,
"chroot": "fedora-rawhide-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "8513152-fedora-rawhide-x86_64",
"build_id": 8513152,
"package_name": "python-distgen",
"package_version": "2.1-1",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/PyPI/python-distgen",
"git_hash": "2baa76c2623fc4dcbaeed4813ee23569b94b7355",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "@copr/PyPI--ksurma",
"results": null,
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"storage": null,
"background": true,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI/fedora-rawhide-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/@copr/PyPI",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI",
"result_dir": "08513152-python-distgen",
"built_packages": "",
"id": 8513152,
"mockchain_macros": {
"copr_username": "@copr",
"copr_projectname": "PyPI",
"vendor": "Fedora Project COPR (@copr/PyPI)"
}
}
]
}
[2025-01-14 18:15:04,464][ INFO][PID:1135345] Sending fedora-messaging bus message in build.start
[2025-01-14 18:15:06,051][ INFO][PID:1135345] Sending fedora-messaging bus message in chroot.start
[2025-01-14 18:15:06,094][ INFO][PID:1135345] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8513152-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached
[2025-01-14 18:15:06,510][ INFO][PID:1135345] Downloading the builder-live.log file, attempt 1
[2025-01-14 18:15:06,525][ INFO][PID:1135345] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.80.140.66 copr-rpmbuild-log
[2025-01-14 18:15:11,529][ INFO][PID:1135345] Periodic builder liveness probe: alive
[2025-01-14 18:15:23,494][ ERROR][PID:1135345] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-14 18:15:33,495][WARNING][PID:1135345] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-14 18:15:43,496][ INFO][PID:1135345] Downloading the builder-live.log file, attempt 2
[2025-01-14 18:15:43,498][ INFO][PID:1135345] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.80.140.66 copr-rpmbuild-log
[2025-01-14 18:16:43,568][ ERROR][PID:1135345] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-14 18:16:53,574][WARNING][PID:1135345] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-14 18:17:03,575][ INFO][PID:1135345] Downloading the builder-live.log file, attempt 3
[2025-01-14 18:17:03,577][ INFO][PID:1135345] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.80.140.66 copr-rpmbuild-log
[2025-01-14 18:18:03,647][ ERROR][PID:1135345] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-14 18:18:13,663][WARNING][PID:1135345] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-14 18:18:23,664][ INFO][PID:1135345] Downloading the builder-live.log file, attempt 4
[2025-01-14 18:18:23,665][ INFO][PID:1135345] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.80.140.66 copr-rpmbuild-log
[2025-01-14 18:19:23,738][ ERROR][PID:1135345] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-14 18:19:33,747][ ERROR][PID:1135345] Giving up for unstable SSH, failures: 1736878533.4959342, 1736878613.574333, 1736878693.663281, 1736878773.7472377
[2025-01-14 18:19:33,748][ ERROR][PID:1135345] Re-try request for task on 'ResallocHost, ticket_id=673725, hostname=98.80.140.66, name=aws_x86_64_spot_prod_00551487_20250114_173009': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-01-14 18:19:33,748][ INFO][PID:1135345] Releasing VM back to pool
[2025-01-14 18:19:33,774][ INFO][PID:1135345] Retry #1 (on other host)
[2025-01-14 18:19:33,775][ INFO][PID:1135345] Trying to allocate VM
[2025-01-14 18:19:36,805][ INFO][PID:1135345] Allocated host ResallocHost, ticket_id=673820, hostname=98.81.179.101, name=aws_x86_64_normalreserved_prod_00552013_20250114_181450
[2025-01-14 18:19:36,806][ INFO][PID:1135345] Allocating ssh connection to builder
[2025-01-14 18:19:36,806][ INFO][PID:1135345] Checking that builder machine is OK
[2025-01-14 18:19:37,156][ INFO][PID:1135345] Running remote command: copr-builder-ready fedora-rawhide-x86_64
[2025-01-14 18:19:37,382][ INFO][PID:1135345] Red Hat subscription not needed for fedora-rawhide-x86_64
Builder is ready to be used
[2025-01-14 18:19:37,383][ INFO][PID:1135345] Filling build.info file with builder info
[2025-01-14 18:19:37,384][ INFO][PID:1135345] Not re-notifying FE and msg buses for the new host.
[2025-01-14 18:19:37,384][ INFO][PID:1135345] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8513152-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached
[2025-01-14 18:19:37,743][ INFO][PID:1135345] Downloading the builder-live.log file, attempt 1
[2025-01-14 18:19:37,745][ INFO][PID:1135345] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.81.179.101 copr-rpmbuild-log
[2025-01-14 18:20:17,414][ INFO][PID:1135345] Downloading results from builder
[2025-01-14 18:20:17,415][ INFO][PID:1135345] rsyncing of mockbuilder@98.81.179.101:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen started
[2025-01-14 18:20:17,416][ INFO][PID:1135345] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@98.81.179.101:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/ &> /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/build-08513152.rsync.log
[2025-01-14 18:20:17,650][ INFO][PID:1135345] rsyncing finished.
[2025-01-14 18:20:17,651][ INFO][PID:1135345] Releasing VM back to pool
[2025-01-14 18:20:17,660][ INFO][PID:1135345] Searching for 'success' file in resultdir
[2025-01-14 18:20:17,661][ INFO][PID:1135345] Going to sign pkgs from source: 8513152-fedora-rawhide-x86_64 in chroot: /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64
[2025-01-14 18:20:17,662][ INFO][PID:1135345] Calling '/bin/sign -u @copr#PyPI@copr.fedorahosted.org -p' (attempt #1)
[2025-01-14 18:20:17,960][ INFO][PID:1135345] Calling '/bin/sign -4 -h sha256 -u @copr#PyPI@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/python-distgen-2.1-1.fc42.src.rpm' (attempt #1)
[2025-01-14 18:20:18,752][ INFO][PID:1135345] signed rpm: /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/python-distgen-2.1-1.fc42.src.rpm
[2025-01-14 18:20:18,753][ INFO][PID:1135345] Calling '/bin/sign -4 -h sha256 -u @copr#PyPI@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/python3-distgen-2.1-1.fc42.noarch.rpm' (attempt #1)
[2025-01-14 18:20:19,544][ INFO][PID:1135345] signed rpm: /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/python3-distgen-2.1-1.fc42.noarch.rpm
[2025-01-14 18:20:19,545][ INFO][PID:1135345] Sign done
[2025-01-14 18:20:19,546][ INFO][PID:1135345] Incremental createrepo run, adding 08513152-python-distgen into https://download.copr.fedorainfracloud.org/results/@copr/PyPI/fedora-rawhide-x86_64, (auto-create-repo=True)
[2025-01-14 18:20:19,547][ INFO][PID:1135345] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64 --add 08513152-python-distgen --no-appstream-metadata' as PID 1168916
[2025-01-14 18:21:25,099][ INFO][PID:1135345] Finished after 65 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64 --add 08513152-python-distgen --no-appstream-metadata)
[2025-01-14 18:21:25,100][ INFO][PID:1135345] Getting build details
[2025-01-14 18:21:25,101][ INFO][PID:1135345] Listing built binary packages in /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen
[2025-01-14 18:21:25,102][ INFO][PID:1135345] Built packages:
['python3-distgen 2.1']
[2025-01-14 18:21:25,107][ INFO][PID:1135345] build details: {'built_packages': 'python3-distgen 2.1'}
[2025-01-14 18:21:25,111][ INFO][PID:1135345] Retrieving pubkey
[2025-01-14 18:21:25,118][ INFO][PID:1135345] Calling '/bin/sign -u @copr#PyPI@copr.fedorahosted.org -p' (attempt #1)
[2025-01-14 18:21:25,430][ INFO][PID:1135345] Added pubkey for user @copr project PyPI into: /var/lib/copr/public_html/results/@copr/PyPI/pubkey.gpg
[2025-01-14 18:21:25,431][ INFO][PID:1135345] Finished build: id=8513152 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@copr/PyPI chroot=fedora-rawhide-x86_64
[2025-01-14 18:21:25,434][ INFO][PID:1135345] Worker succeeded build, took 108.05057740211487
[2025-01-14 18:21:25,436][ INFO][PID:1135345] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "@copr",
"project_name": "PyPI",
"project_dirname": "PyPI",
"submitter": "ksurma",
"ended_on": 1736878885.4348686,
"started_on": 1736878777.3842912,
"submitted_on": null,
"status": 1,
"chroot": "fedora-rawhide-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "8513152-fedora-rawhide-x86_64",
"build_id": 8513152,
"package_name": "python-distgen",
"package_version": "2.1-1",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/PyPI/python-distgen",
"git_hash": "2baa76c2623fc4dcbaeed4813ee23569b94b7355",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "@copr/PyPI--ksurma",
"results": {
"packages": [
{
"name": "python3-distgen",
"epoch": null,
"version": "2.1",
"release": "1.fc42",
"arch": "noarch"
},
{
"name": "python-distgen",
"epoch": null,
"version": "2.1",
"release": "1.fc42",
"arch": "src"
}
]
},
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"storage": null,
"background": true,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI/fedora-rawhide-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/@copr/PyPI",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI",
"result_dir": "08513152-python-distgen",
"built_packages": "python3-distgen 2.1",
"id": 8513152,
"mockchain_macros": {
"copr_username": "@copr",
"copr_projectname": "PyPI",
"vendor": "Fedora Project COPR (@copr/PyPI)"
}
}
]
}
[2025-01-14 18:21:25,478][ INFO][PID:1135345] Sending fedora-messaging bus message in build.end
[2025-01-14 18:21:25,505][ INFO][PID:1135345] Compressing /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/builder-live.log by gzip
[2025-01-14 18:21:25,508][ INFO][PID:1135345] Running command 'gzip /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/builder-live.log' as PID 1177449
[2025-01-14 18:21:25,515][ INFO][PID:1135345] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/builder-live.log)
[2025-01-14 18:21:25,516][ INFO][PID:1135345] Compressing /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08513152-python-distgen/backend.log by gzip