[2024-11-27 14:33:03,549][ INFO][PID:362509] Marking build as starting
[2024-11-27 14:33:03,657][ INFO][PID:362509] Trying to allocate VM
[2024-11-27 14:33:31,807][ INFO][PID:362509] Allocated host ResallocHost, ticket_id=6098348, hostname=44.222.95.95
[2024-11-27 14:33:31,809][ INFO][PID:362509] Allocating ssh connection to builder
[2024-11-27 14:33:31,811][ INFO][PID:362509] Checking that builder machine is OK
[2024-11-27 14:33:32,165][ INFO][PID:362509] Running remote command: copr-builder-ready fedora-39-x86_64
[2024-11-27 14:33:32,385][ INFO][PID:362509] Red Hat subscription not needed for fedora-39-x86_64
Builder is ready to be used
[2024-11-27 14:33:32,386][ INFO][PID:362509] Filling build.info file with builder info
[2024-11-27 14:33:32,389][ INFO][PID:362509] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "torsava",
"project_name": "automatic-obsoletes",
"project_dirname": "automatic-obsoletes",
"submitter": "https://src.fedoraproject.org/user/rjones",
"ended_on": null,
"started_on": 1732718012.388691,
"submitted_on": null,
"status": 3,
"chroot": "fedora-39-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "8318997-fedora-39-x86_64",
"build_id": 8318997,
"package_name": "gdal",
"package_version": "3.10.0-4",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/torsava/automatic-obsoletes/gdal",
"git_hash": "c096ce778f7109b26a15055c6acde0156967f636",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "torsava/automatic-obsoletes--https://src.fedoraproject.org/user/rjones",
"results": null,
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"storage": null,
"background": true,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes/fedora-39-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/torsava/automatic-obsoletes",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes",
"result_dir": "08318997-gdal",
"built_packages": "",
"id": 8318997,
"mockchain_macros": {
"copr_username": "torsava",
"copr_projectname": "automatic-obsoletes",
"vendor": "Fedora Project COPR (torsava/automatic-obsoletes)"
}
}
]
}
[2024-11-27 14:33:32,457][ INFO][PID:362509] Sending fedora-messaging bus message in build.start
[2024-11-27 14:33:33,286][ INFO][PID:362509] Sending fedora-messaging bus message in chroot.start
[2024-11-27 14:33:33,325][ INFO][PID:362509] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8318997-fedora-39-x86_64 --chroot fedora-39-x86_64 --detached
[2024-11-27 14:33:33,683][ INFO][PID:362509] Downloading the builder-live.log file, attempt 1
[2024-11-27 14:33:33,686][ INFO][PID:362509] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.222.95.95 copr-rpmbuild-log
[2024-11-27 14:33:34,942][ ERROR][PID:362509] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2024-11-27 14:33:38,704][ INFO][PID:362509] Periodic builder liveness probe: alive
[2024-11-27 14:33:44,945][WARNING][PID:362509] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2024-11-27 14:33:54,946][ INFO][PID:362509] Downloading the builder-live.log file, attempt 2
[2024-11-27 14:33:54,949][ INFO][PID:362509] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.222.95.95 copr-rpmbuild-log
[2024-11-27 14:34:55,019][ ERROR][PID:362509] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2024-11-27 14:35:05,021][WARNING][PID:362509] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2024-11-27 14:35:15,022][ INFO][PID:362509] Downloading the builder-live.log file, attempt 3
[2024-11-27 14:35:15,028][ INFO][PID:362509] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.222.95.95 copr-rpmbuild-log
[2024-11-27 14:36:15,097][ ERROR][PID:362509] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2024-11-27 14:36:25,099][WARNING][PID:362509] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2024-11-27 14:36:35,105][ INFO][PID:362509] Downloading the builder-live.log file, attempt 4
[2024-11-27 14:36:35,108][ INFO][PID:362509] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.222.95.95 copr-rpmbuild-log
[2024-11-27 14:37:35,186][ ERROR][PID:362509] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2024-11-27 14:37:45,193][ ERROR][PID:362509] Giving up for unstable SSH, failures: 1732718024.9455488, 1732718105.0211902, 1732718185.0991218, 1732718265.1927605
[2024-11-27 14:37:45,194][ ERROR][PID:362509] Re-try request for task on 'ResallocHost, ticket_id=6098348, hostname=44.222.95.95': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2024-11-27 14:37:45,196][ INFO][PID:362509] Releasing VM back to pool
[2024-11-27 14:37:45,230][ INFO][PID:362509] Retry #1 (on other host)
[2024-11-27 14:37:45,231][ INFO][PID:362509] Trying to allocate VM
[2024-11-27 14:39:13,452][ INFO][PID:362509] Allocated host ResallocHost, ticket_id=6098490, hostname=18.207.208.80
[2024-11-27 14:39:13,452][ INFO][PID:362509] Allocating ssh connection to builder
[2024-11-27 14:39:13,453][ INFO][PID:362509] Checking that builder machine is OK
[2024-11-27 14:39:13,878][ INFO][PID:362509] Running remote command: copr-builder-ready fedora-39-x86_64
[2024-11-27 14:39:14,152][ INFO][PID:362509] Red Hat subscription not needed for fedora-39-x86_64
Builder is ready to be used
[2024-11-27 14:39:14,153][ INFO][PID:362509] Filling build.info file with builder info
[2024-11-27 14:39:14,158][ INFO][PID:362509] Not re-notifying FE and msg buses for the new host.
[2024-11-27 14:39:14,160][ INFO][PID:362509] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8318997-fedora-39-x86_64 --chroot fedora-39-x86_64 --detached
[2024-11-27 14:39:14,537][ INFO][PID:362509] Downloading the builder-live.log file, attempt 1
[2024-11-27 14:39:14,542][ INFO][PID:362509] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.207.208.80 copr-rpmbuild-log
[2024-11-27 14:40:34,191][ INFO][PID:362509] Downloading results from builder
[2024-11-27 14:40:34,192][ INFO][PID:362509] rsyncing of mockbuilder@18.207.208.80:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-39-x86_64/08318997-gdal started
[2024-11-27 14:40:34,199][ INFO][PID:362509] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@18.207.208.80:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-39-x86_64/08318997-gdal/ &> /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-39-x86_64/08318997-gdal/build-08318997.rsync.log
[2024-11-27 14:40:34,471][ INFO][PID:362509] rsyncing finished.
[2024-11-27 14:40:34,473][ INFO][PID:362509] Releasing VM back to pool
[2024-11-27 14:40:34,553][ INFO][PID:362509] Searching for 'success' file in resultdir
[2024-11-27 14:40:34,554][ ERROR][PID:362509] Build failed: Backend process error: No success file => build failure
[2024-11-27 14:40:34,557][ INFO][PID:362509] Finished build: id=8318997 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/torsava/automatic-obsoletes chroot=fedora-39-x86_64
[2024-11-27 14:40:34,561][ ERROR][PID:362509] Backend process error: No success file => build failure
[2024-11-27 14:40:34,561][ INFO][PID:362509] Worker failed build, took 80.403728723526
[2024-11-27 14:40:34,562][ INFO][PID:362509] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "torsava",
"project_name": "automatic-obsoletes",
"project_dirname": "automatic-obsoletes",
"submitter": "https://src.fedoraproject.org/user/rjones",
"ended_on": 1732718434.561767,
"started_on": 1732718354.1580384,
"submitted_on": null,
"status": 0,
"chroot": "fedora-39-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "8318997-fedora-39-x86_64",
"build_id": 8318997,
"package_name": "gdal",
"package_version": "3.10.0-4",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/torsava/automatic-obsoletes/gdal",
"git_hash": "c096ce778f7109b26a15055c6acde0156967f636",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "torsava/automatic-obsoletes--https://src.fedoraproject.org/user/rjones",
"results": null,
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"storage": null,
"background": true,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes/fedora-39-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/torsava/automatic-obsoletes",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes",
"result_dir": "08318997-gdal",
"built_packages": "",
"id": 8318997,
"mockchain_macros": {
"copr_username": "torsava",
"copr_projectname": "automatic-obsoletes",
"vendor": "Fedora Project COPR (torsava/automatic-obsoletes)"
}
}
]
}
[2024-11-27 14:40:34,656][ INFO][PID:362509] Sending fedora-messaging bus message in build.end
[2024-11-27 14:40:34,684][ INFO][PID:362509] Compressing /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-39-x86_64/08318997-gdal/builder-live.log by gzip
[2024-11-27 14:40:34,686][ INFO][PID:362509] Running command 'gzip /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-39-x86_64/08318997-gdal/builder-live.log' as PID 426512
[2024-11-27 14:40:34,694][ INFO][PID:362509] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-39-x86_64/08318997-gdal/builder-live.log)
[2024-11-27 14:40:34,697][ INFO][PID:362509] Compressing /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-39-x86_64/08318997-gdal/backend.log by gzip
[2024-11-27 14:40:34,709][ INFO][PID:362509] Running command 'gzip /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-39-x86_64/08318997-gdal/backend.log' as PID 426513