[2025-03-11 17:48:59,843][ INFO][PID:3842494] Marking build as starting
[2025-03-11 17:48:59,904][ INFO][PID:3842494] Trying to allocate VM
[2025-03-11 17:50:08,225][ INFO][PID:3842494] Allocated host ResallocHost, ticket_id=1750498, hostname=54.83.169.102, name=aws_x86_64_spot_prod_01867003_20250311_171815
[2025-03-11 17:50:08,226][ INFO][PID:3842494] Allocating ssh connection to builder
[2025-03-11 17:50:08,226][ INFO][PID:3842494] Checking that builder machine is OK
[2025-03-11 17:50:08,578][ INFO][PID:3842494] Running remote command: copr-builder-ready fedora-42-x86_64
[2025-03-11 17:50:08,800][ INFO][PID:3842494] Red Hat subscription not needed for fedora-42-x86_64
Builder is ready to be used
[2025-03-11 17:50:08,802][ INFO][PID:3842494] Filling build.info file with builder info
[2025-03-11 17:50:08,805][ INFO][PID:3842494] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "farchord",
"project_name": "kde-test",
"project_dirname": "kde-test",
"submitter": "farchord",
"ended_on": null,
"started_on": 1741715408.804741,
"submitted_on": null,
"status": 3,
"chroot": "fedora-42-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "8751286-fedora-42-x86_64",
"build_id": 8751286,
"package_name": "spectacle",
"package_version": "1:6.3.3-1",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/farchord/kde-test/spectacle",
"git_hash": "a4aa7414a762240acfb370b98a0cb0f72e5f3f2a",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "farchord/kde-test--farchord",
"results": null,
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"storage": null,
"background": false,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/farchord/kde-test/fedora-42-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/farchord/kde-test",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/farchord/kde-test",
"result_dir": "08751286-spectacle",
"built_packages": "",
"id": 8751286,
"mockchain_macros": {
"copr_username": "farchord",
"copr_projectname": "kde-test",
"vendor": "Fedora Project COPR (farchord/kde-test)"
}
}
]
}
[2025-03-11 17:50:08,840][ INFO][PID:3842494] Sending fedora-messaging bus message in build.start
[2025-03-11 17:50:09,644][ INFO][PID:3842494] Sending fedora-messaging bus message in chroot.start
[2025-03-11 17:50:09,669][ INFO][PID:3842494] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8751286-fedora-42-x86_64 --chroot fedora-42-x86_64 --detached
[2025-03-11 17:50:10,085][ INFO][PID:3842494] Downloading the builder-live.log file, attempt 1
[2025-03-11 17:50:10,086][ INFO][PID:3842494] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.83.169.102 copr-rpmbuild-log
[2025-03-11 17:50:15,095][ INFO][PID:3842494] Periodic builder liveness probe: alive
[2025-03-11 17:53:46,655][ ERROR][PID:3842494] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-03-11 17:53:56,656][WARNING][PID:3842494] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-03-11 17:54:06,656][ INFO][PID:3842494] Downloading the builder-live.log file, attempt 2
[2025-03-11 17:54:06,659][ INFO][PID:3842494] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.83.169.102 copr-rpmbuild-log
[2025-03-11 17:55:06,727][ ERROR][PID:3842494] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-03-11 17:55:16,761][WARNING][PID:3842494] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-03-11 17:55:26,761][ INFO][PID:3842494] Downloading the builder-live.log file, attempt 3
[2025-03-11 17:55:26,762][ INFO][PID:3842494] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.83.169.102 copr-rpmbuild-log
[2025-03-11 17:56:26,831][ ERROR][PID:3842494] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-03-11 17:56:36,846][WARNING][PID:3842494] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-03-11 17:56:46,847][ INFO][PID:3842494] Downloading the builder-live.log file, attempt 4
[2025-03-11 17:56:46,848][ INFO][PID:3842494] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.83.169.102 copr-rpmbuild-log
[2025-03-11 17:57:46,918][ ERROR][PID:3842494] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-03-11 17:57:56,930][ ERROR][PID:3842494] Giving up for unstable SSH, failures: 1741715636.656069, 1741715716.760983, 1741715796.8465672, 1741715876.9297473
[2025-03-11 17:57:56,931][ ERROR][PID:3842494] Re-try request for task on 'ResallocHost, ticket_id=1750498, hostname=54.83.169.102, name=aws_x86_64_spot_prod_01867003_20250311_171815': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-03-11 17:57:56,931][ INFO][PID:3842494] Releasing VM back to pool
[2025-03-11 17:57:56,957][ INFO][PID:3842494] Retry #1 (on other host)
[2025-03-11 17:57:56,958][ INFO][PID:3842494] Trying to allocate VM
[2025-03-11 17:58:00,084][ INFO][PID:3842494] Allocated host ResallocHost, ticket_id=1750604, hostname=2620:52:3:1:dead:beef:cafe:c18f, name=copr_hv_x86_64_03_prod_01867947_20250311_175155
[2025-03-11 17:58:00,084][ INFO][PID:3842494] Allocating ssh connection to builder
[2025-03-11 17:58:00,085][ INFO][PID:3842494] Checking that builder machine is OK
[2025-03-11 17:58:00,668][ INFO][PID:3842494] Running remote command: copr-builder-ready fedora-42-x86_64
[2025-03-11 17:58:00,987][ INFO][PID:3842494] Red Hat subscription not needed for fedora-42-x86_64
Builder is ready to be used
[2025-03-11 17:58:00,988][ INFO][PID:3842494] Filling build.info file with builder info
[2025-03-11 17:58:00,999][ INFO][PID:3842494] Not re-notifying FE and msg buses for the new host.
[2025-03-11 17:58:00,999][ INFO][PID:3842494] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8751286-fedora-42-x86_64 --chroot fedora-42-x86_64 --detached
[2025-03-11 17:58:01,544][ INFO][PID:3842494] Downloading the builder-live.log file, attempt 1
[2025-03-11 17:58:01,546][ INFO][PID:3842494] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c18f copr-rpmbuild-log
[2025-03-11 18:00:16,574][ INFO][PID:3842494] Periodic builder liveness probe: alive
[2025-03-11 18:04:32,399][ INFO][PID:3842494] Downloading results from builder
[2025-03-11 18:04:32,400][ INFO][PID:3842494] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c18f]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle started
[2025-03-11 18:04:32,401][ INFO][PID:3842494] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c18f]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/ &> /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/build-08751286.rsync.log
[2025-03-11 18:04:32,942][ INFO][PID:3842494] rsyncing finished.
[2025-03-11 18:04:32,942][ INFO][PID:3842494] Releasing VM back to pool
[2025-03-11 18:04:32,958][ INFO][PID:3842494] Searching for 'success' file in resultdir
[2025-03-11 18:04:32,958][ INFO][PID:3842494] Going to sign pkgs from source: 8751286-fedora-42-x86_64 in chroot: /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64
[2025-03-11 18:04:32,959][ INFO][PID:3842494] Calling '/bin/sign -u farchord#kde-test@copr.fedorahosted.org -p' (attempt #1)
[2025-03-11 18:04:33,136][ INFO][PID:3842494] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/spectacle-debugsource-6.3.3-1.fc42.x86_64.rpm' (attempt #1)
[2025-03-11 18:04:33,920][ INFO][PID:3842494] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/spectacle-debugsource-6.3.3-1.fc42.x86_64.rpm
[2025-03-11 18:04:33,920][ INFO][PID:3842494] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/spectacle-6.3.3-1.fc42.x86_64.rpm' (attempt #1)
[2025-03-11 18:04:34,793][ INFO][PID:3842494] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/spectacle-6.3.3-1.fc42.x86_64.rpm
[2025-03-11 18:04:34,795][ INFO][PID:3842494] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/spectacle-6.3.3-1.fc42.src.rpm' (attempt #1)
[2025-03-11 18:04:35,482][ INFO][PID:3842494] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/spectacle-6.3.3-1.fc42.src.rpm
[2025-03-11 18:04:35,483][ INFO][PID:3842494] Calling '/bin/sign -4 -h sha256 -u farchord#kde-test@copr.fedorahosted.org -r /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/spectacle-debuginfo-6.3.3-1.fc42.x86_64.rpm' (attempt #1)
[2025-03-11 18:04:36,322][ INFO][PID:3842494] signed rpm: /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/spectacle-debuginfo-6.3.3-1.fc42.x86_64.rpm
[2025-03-11 18:04:36,323][ INFO][PID:3842494] Sign done
[2025-03-11 18:04:36,324][ INFO][PID:3842494] Incremental createrepo run, adding 08751286-spectacle into https://download.copr.fedorainfracloud.org/results/farchord/kde-test/fedora-42-x86_64, (auto-create-repo=True)
[2025-03-11 18:04:36,326][ INFO][PID:3842494] Running command 'copr-repo --batched /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64 --add 08751286-spectacle --no-appstream-metadata' as PID 3968610
[2025-03-11 18:04:38,610][ INFO][PID:3842494] Finished after 2 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64 --add 08751286-spectacle --no-appstream-metadata)
[2025-03-11 18:04:38,610][ INFO][PID:3842494] Getting build details
[2025-03-11 18:04:38,611][ INFO][PID:3842494] Listing built binary packages in /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle
[2025-03-11 18:04:38,611][ INFO][PID:3842494] Built packages:
['spectacle 6.3.3', 'spectacle-debuginfo 6.3.3', 'spectacle-debugsource 6.3.3']
[2025-03-11 18:04:38,612][ INFO][PID:3842494] build details: {'built_packages': 'spectacle 6.3.3\nspectacle-debuginfo 6.3.3\nspectacle-debugsource 6.3.3'}
[2025-03-11 18:04:38,613][ INFO][PID:3842494] Retrieving pubkey
[2025-03-11 18:04:38,613][ INFO][PID:3842494] Calling '/bin/sign -u farchord#kde-test@copr.fedorahosted.org -p' (attempt #1)
[2025-03-11 18:04:38,790][ INFO][PID:3842494] Added pubkey for user farchord project kde-test into: /var/lib/copr/public_html/results/farchord/kde-test/pubkey.gpg
[2025-03-11 18:04:38,790][ INFO][PID:3842494] Finished build: id=8751286 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/farchord/kde-test chroot=fedora-42-x86_64
[2025-03-11 18:04:38,792][ INFO][PID:3842494] Worker succeeded build, took 397.7933566570282
[2025-03-11 18:04:38,793][ INFO][PID:3842494] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "farchord",
"project_name": "kde-test",
"project_dirname": "kde-test",
"submitter": "farchord",
"ended_on": 1741716278.7924957,
"started_on": 1741715880.999139,
"submitted_on": null,
"status": 1,
"chroot": "fedora-42-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "8751286-fedora-42-x86_64",
"build_id": 8751286,
"package_name": "spectacle",
"package_version": "1:6.3.3-1",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/farchord/kde-test/spectacle",
"git_hash": "a4aa7414a762240acfb370b98a0cb0f72e5f3f2a",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "farchord/kde-test--farchord",
"results": {
"packages": [
{
"name": "spectacle",
"epoch": 1,
"version": "6.3.3",
"release": "1.fc42",
"arch": "x86_64"
},
{
"name": "spectacle",
"epoch": 1,
"version": "6.3.3",
"release": "1.fc42",
"arch": "src"
},
{
"name": "spectacle-debuginfo",
"epoch": 1,
"version": "6.3.3",
"release": "1.fc42",
"arch": "x86_64"
},
{
"name": "spectacle-debugsource",
"epoch": 1,
"version": "6.3.3",
"release": "1.fc42",
"arch": "x86_64"
}
]
},
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"storage": null,
"background": false,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/farchord/kde-test/fedora-42-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/farchord/kde-test",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/farchord/kde-test",
"result_dir": "08751286-spectacle",
"built_packages": "spectacle 6.3.3\nspectacle-debuginfo 6.3.3\nspectacle-debugsource 6.3.3",
"id": 8751286,
"mockchain_macros": {
"copr_username": "farchord",
"copr_projectname": "kde-test",
"vendor": "Fedora Project COPR (farchord/kde-test)"
}
}
]
}
[2025-03-11 18:04:38,825][ INFO][PID:3842494] Sending fedora-messaging bus message in build.end
[2025-03-11 18:04:38,909][ INFO][PID:3842494] Compressing /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/builder-live.log by gzip
[2025-03-11 18:04:38,910][ INFO][PID:3842494] Running command 'gzip /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/builder-live.log' as PID 3968968
[2025-03-11 18:04:38,936][ INFO][PID:3842494] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/builder-live.log)
[2025-03-11 18:04:38,937][ INFO][PID:3842494] Compressing /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/backend.log by gzip
[2025-03-11 18:04:38,938][ INFO][PID:3842494] Running command 'gzip /var/lib/copr/public_html/results/farchord/kde-test/fedora-42-x86_64/08751286-spectacle/backend.log' as PID 3968970