[2025-01-31 15:08:57,589][ INFO][PID:2381191] Marking build as starting
[2025-01-31 15:08:57,623][ INFO][PID:2381191] Trying to allocate VM
[2025-01-31 15:10:25,862][ INFO][PID:2381191] Allocated host ResallocHost, ticket_id=1008132, hostname=3.95.251.60, name=aws_x86_64_spot_prod_00963690_20250131_150622
[2025-01-31 15:10:25,874][ INFO][PID:2381191] Allocating ssh connection to builder
[2025-01-31 15:10:25,874][ INFO][PID:2381191] Checking that builder machine is OK
[2025-01-31 15:10:26,223][ INFO][PID:2381191] Running remote command: copr-builder-ready fedora-rawhide-x86_64
[2025-01-31 15:10:26,445][ INFO][PID:2381191] Red Hat subscription not needed for fedora-rawhide-x86_64
Builder is ready to be used
[2025-01-31 15:10:26,446][ INFO][PID:2381191] Filling build.info file with builder info
[2025-01-31 15:10:26,448][ INFO][PID:2381191] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "@copr",
"project_name": "PyPI",
"project_dirname": "PyPI",
"submitter": "ksurma",
"ended_on": null,
"started_on": 1738336226.4483392,
"submitted_on": null,
"status": 3,
"chroot": "fedora-rawhide-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "8590144-fedora-rawhide-x86_64",
"build_id": 8590144,
"package_name": "python-deid",
"package_version": "0.4.0-1",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/PyPI/python-deid",
"git_hash": "6f6b0f0aca67331f1788cb970a6da9c02e96c498",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "@copr/PyPI--ksurma",
"results": null,
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"storage": null,
"background": true,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI/fedora-rawhide-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/@copr/PyPI",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI",
"result_dir": "08590144-python-deid",
"built_packages": "",
"id": 8590144,
"mockchain_macros": {
"copr_username": "@copr",
"copr_projectname": "PyPI",
"vendor": "Fedora Project COPR (@copr/PyPI)"
}
}
]
}
[2025-01-31 15:10:26,489][ INFO][PID:2381191] Sending fedora-messaging bus message in build.start
[2025-01-31 15:10:27,268][ INFO][PID:2381191] Sending fedora-messaging bus message in chroot.start
[2025-01-31 15:10:27,298][ INFO][PID:2381191] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8590144-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached
[2025-01-31 15:10:27,661][ INFO][PID:2381191] Downloading the builder-live.log file, attempt 1
[2025-01-31 15:10:27,666][ INFO][PID:2381191] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.95.251.60 copr-rpmbuild-log
[2025-01-31 15:10:32,699][ INFO][PID:2381191] Periodic builder liveness probe: alive
[2025-01-31 15:10:43,454][ ERROR][PID:2381191] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-31 15:10:53,458][WARNING][PID:2381191] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-31 15:11:03,459][ INFO][PID:2381191] Downloading the builder-live.log file, attempt 2
[2025-01-31 15:11:03,461][ INFO][PID:2381191] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.95.251.60 copr-rpmbuild-log
[2025-01-31 15:12:03,532][ ERROR][PID:2381191] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-31 15:12:13,543][WARNING][PID:2381191] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-31 15:12:23,544][ INFO][PID:2381191] Downloading the builder-live.log file, attempt 3
[2025-01-31 15:12:23,552][ INFO][PID:2381191] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.95.251.60 copr-rpmbuild-log
[2025-01-31 15:13:23,620][ ERROR][PID:2381191] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-31 15:13:33,629][WARNING][PID:2381191] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-01-31 15:13:43,633][ INFO][PID:2381191] Downloading the builder-live.log file, attempt 4
[2025-01-31 15:13:43,637][ INFO][PID:2381191] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.95.251.60 copr-rpmbuild-log
[2025-01-31 15:14:43,706][ ERROR][PID:2381191] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-01-31 15:14:53,730][ ERROR][PID:2381191] Giving up for unstable SSH, failures: 1738336253.4582796, 1738336333.5435405, 1738336413.629315, 1738336493.7293706
[2025-01-31 15:14:53,732][ ERROR][PID:2381191] Re-try request for task on 'ResallocHost, ticket_id=1008132, hostname=3.95.251.60, name=aws_x86_64_spot_prod_00963690_20250131_150622': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-01-31 15:14:53,733][ INFO][PID:2381191] Releasing VM back to pool
[2025-01-31 15:14:53,994][ INFO][PID:2381191] Retry #1 (on other host)
[2025-01-31 15:14:53,998][ INFO][PID:2381191] Trying to allocate VM
[2025-01-31 15:15:42,838][ INFO][PID:2381191] Allocated host ResallocHost, ticket_id=1008339, hostname=44.203.212.249, name=aws_x86_64_spot_prod_00963617_20250131_150211
[2025-01-31 15:15:42,843][ INFO][PID:2381191] Allocating ssh connection to builder
[2025-01-31 15:15:42,844][ INFO][PID:2381191] Checking that builder machine is OK
[2025-01-31 15:15:43,231][ INFO][PID:2381191] Running remote command: copr-builder-ready fedora-rawhide-x86_64
[2025-01-31 15:15:43,477][ INFO][PID:2381191] Red Hat subscription not needed for fedora-rawhide-x86_64
Builder is ready to be used
[2025-01-31 15:15:43,479][ INFO][PID:2381191] Filling build.info file with builder info
[2025-01-31 15:15:43,492][ INFO][PID:2381191] Not re-notifying FE and msg buses for the new host.
[2025-01-31 15:15:43,492][ INFO][PID:2381191] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8590144-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached
[2025-01-31 15:15:43,876][ INFO][PID:2381191] Downloading the builder-live.log file, attempt 1
[2025-01-31 15:15:43,880][ INFO][PID:2381191] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.203.212.249 copr-rpmbuild-log
[2025-01-31 15:16:12,034][ INFO][PID:2381191] Downloading results from builder
[2025-01-31 15:16:12,038][ INFO][PID:2381191] rsyncing of mockbuilder@44.203.212.249:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid started
[2025-01-31 15:16:12,040][ INFO][PID:2381191] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@44.203.212.249:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/ &> /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/build-08590144.rsync.log
[2025-01-31 15:16:12,348][ INFO][PID:2381191] rsyncing finished.
[2025-01-31 15:16:12,349][ INFO][PID:2381191] Releasing VM back to pool
[2025-01-31 15:16:12,437][ INFO][PID:2381191] Searching for 'success' file in resultdir
[2025-01-31 15:16:12,438][ INFO][PID:2381191] Going to sign pkgs from source: 8590144-fedora-rawhide-x86_64 in chroot: /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64
[2025-01-31 15:16:12,439][ INFO][PID:2381191] Calling '/bin/sign -u @copr#PyPI@copr.fedorahosted.org -p' (attempt #1)
[2025-01-31 15:16:12,751][ INFO][PID:2381191] Calling '/bin/sign -4 -h sha256 -u @copr#PyPI@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/python3-deid-0.4.0-1.fc42.noarch.rpm' (attempt #1)
[2025-01-31 15:16:13,320][ INFO][PID:2381191] signed rpm: /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/python3-deid-0.4.0-1.fc42.noarch.rpm
[2025-01-31 15:16:13,321][ INFO][PID:2381191] Calling '/bin/sign -4 -h sha256 -u @copr#PyPI@copr.fedorahosted.org -r /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/python-deid-0.4.0-1.fc42.src.rpm' (attempt #1)
[2025-01-31 15:16:14,006][ INFO][PID:2381191] signed rpm: /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/python-deid-0.4.0-1.fc42.src.rpm
[2025-01-31 15:16:14,006][ INFO][PID:2381191] Sign done
[2025-01-31 15:16:14,007][ INFO][PID:2381191] Incremental createrepo run, adding 08590144-python-deid into https://download.copr.fedorainfracloud.org/results/@copr/PyPI/fedora-rawhide-x86_64, (auto-create-repo=True)
[2025-01-31 15:16:14,008][ INFO][PID:2381191] Running command 'copr-repo --batched /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64 --add 08590144-python-deid --no-appstream-metadata' as PID 2440150
[2025-01-31 15:18:43,727][ INFO][PID:2381191] Finished after 149 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64 --add 08590144-python-deid --no-appstream-metadata)
[2025-01-31 15:18:43,741][ INFO][PID:2381191] Getting build details
[2025-01-31 15:18:43,742][ INFO][PID:2381191] Listing built binary packages in /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid
[2025-01-31 15:18:43,743][ INFO][PID:2381191] Built packages:
['python3-deid 0.4.0']
[2025-01-31 15:18:43,743][ INFO][PID:2381191] build details: {'built_packages': 'python3-deid 0.4.0'}
[2025-01-31 15:18:43,744][ INFO][PID:2381191] Retrieving pubkey
[2025-01-31 15:18:43,744][ INFO][PID:2381191] Calling '/bin/sign -u @copr#PyPI@copr.fedorahosted.org -p' (attempt #1)
[2025-01-31 15:18:43,972][ INFO][PID:2381191] Added pubkey for user @copr project PyPI into: /var/lib/copr/public_html/results/@copr/PyPI/pubkey.gpg
[2025-01-31 15:18:43,973][ INFO][PID:2381191] Finished build: id=8590144 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/@copr/PyPI chroot=fedora-rawhide-x86_64
[2025-01-31 15:18:43,976][ INFO][PID:2381191] Worker succeeded build, took 180.48410296440125
[2025-01-31 15:18:43,978][ INFO][PID:2381191] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "@copr",
"project_name": "PyPI",
"project_dirname": "PyPI",
"submitter": "ksurma",
"ended_on": 1738336723.9763129,
"started_on": 1738336543.49221,
"submitted_on": null,
"status": 1,
"chroot": "fedora-rawhide-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "8590144-fedora-rawhide-x86_64",
"build_id": 8590144,
"package_name": "python-deid",
"package_version": "0.4.0-1",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@copr/PyPI/python-deid",
"git_hash": "6f6b0f0aca67331f1788cb970a6da9c02e96c498",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "@copr/PyPI--ksurma",
"results": {
"packages": [
{
"name": "python3-deid",
"epoch": null,
"version": "0.4.0",
"release": "1.fc42",
"arch": "noarch"
},
{
"name": "python-deid",
"epoch": null,
"version": "0.4.0",
"release": "1.fc42",
"arch": "src"
}
]
},
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"storage": null,
"background": true,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI/fedora-rawhide-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/@copr/PyPI",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/@copr/PyPI",
"result_dir": "08590144-python-deid",
"built_packages": "python3-deid 0.4.0",
"id": 8590144,
"mockchain_macros": {
"copr_username": "@copr",
"copr_projectname": "PyPI",
"vendor": "Fedora Project COPR (@copr/PyPI)"
}
}
]
}
[2025-01-31 15:18:44,014][ INFO][PID:2381191] Sending fedora-messaging bus message in build.end
[2025-01-31 15:18:44,036][ INFO][PID:2381191] Compressing /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/builder-live.log by gzip
[2025-01-31 15:18:44,037][ INFO][PID:2381191] Running command 'gzip /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/builder-live.log' as PID 2463745
[2025-01-31 15:18:44,063][ INFO][PID:2381191] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/builder-live.log)
[2025-01-31 15:18:44,064][ INFO][PID:2381191] Compressing /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/backend.log by gzip
[2025-01-31 15:18:44,066][ INFO][PID:2381191] Running command 'gzip /var/lib/copr/public_html/results/@copr/PyPI/fedora-rawhide-x86_64/08590144-python-deid/backend.log' as PID 2463750