[2024-04-11 13:26:14,824][ INFO][PID:3018406] Marking build as starting
[2024-04-11 13:26:14,959][ INFO][PID:3018406] Trying to allocate VM
[2024-04-11 13:27:23,247][ INFO][PID:3018406] Allocated host ResallocHost, ticket_id=2226623, hostname=44.201.230.63
[2024-04-11 13:27:23,248][ INFO][PID:3018406] Allocating ssh connection to builder
[2024-04-11 13:27:23,249][ INFO][PID:3018406] Checking that builder machine is OK
[2024-04-11 13:27:23,622][ INFO][PID:3018406] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.230.63 '/usr/bin/test -f /etc/mock/fedora-40-x86_64.cfg'
[2024-04-11 13:27:23,837][ INFO][PID:3018406] Filling build.info file with builder info
[2024-04-11 13:27:23,962][ INFO][PID:3018406] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "torsava",
"project_name": "automatic-obsoletes",
"project_dirname": "automatic-obsoletes",
"submitter": "https://src.fedoraproject.org/user/churchyard",
"ended_on": null,
"started_on": 1712842043.9616604,
"submitted_on": null,
"status": 3,
"chroot": "fedora-40-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "7296969-fedora-40-x86_64",
"build_id": 7296969,
"package_name": "Cython",
"package_version": "3.0.9-2",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/torsava/automatic-obsoletes/Cython",
"git_hash": "7238380e4db54f5935b3fb10513129f9e34cf745",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "torsava/automatic-obsoletes--https://src.fedoraproject.org/user/churchyard",
"results": null,
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"background": true,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes/fedora-40-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/torsava/automatic-obsoletes",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes",
"result_dir": "07296969-Cython",
"built_packages": "",
"id": 7296969,
"mockchain_macros": {
"copr_username": "torsava",
"copr_projectname": "automatic-obsoletes",
"vendor": "Fedora Project COPR (torsava/automatic-obsoletes)"
}
}
]
}
[2024-04-11 13:27:24,025][ INFO][PID:3018406] Sending fedora-messaging bus message in build.start
[2024-04-11 13:27:24,825][ INFO][PID:3018406] Sending fedora-messaging bus message in chroot.start
[2024-04-11 13:27:24,855][ INFO][PID:3018406] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/7296969-fedora-40-x86_64 --chroot fedora-40-x86_64 --detached
[2024-04-11 13:27:25,239][ INFO][PID:3018406] Downloading the builder-live.log file, attempt 1
[2024-04-11 13:27:25,464][ INFO][PID:3018406] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.230.63 copr-rpmbuild-log
[2024-04-11 13:27:30,251][ INFO][PID:3018406] Status: False
[2024-04-11 13:35:03,935][ ERROR][PID:3018406] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2024-04-11 13:35:13,937][WARNING][PID:3018406] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2024-04-11 13:35:23,938][ INFO][PID:3018406] Downloading the builder-live.log file, attempt 2
[2024-04-11 13:35:23,940][ INFO][PID:3018406] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.230.63 copr-rpmbuild-log
[2024-04-11 13:36:24,010][ ERROR][PID:3018406] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2024-04-11 13:36:34,011][WARNING][PID:3018406] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2024-04-11 13:36:44,012][ INFO][PID:3018406] Downloading the builder-live.log file, attempt 3
[2024-04-11 13:36:44,018][ INFO][PID:3018406] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.230.63 copr-rpmbuild-log
[2024-04-11 13:37:30,356][ INFO][PID:3018406] Status: False
[2024-04-11 13:37:44,097][ ERROR][PID:3018406] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2024-04-11 13:37:54,192][WARNING][PID:3018406] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2024-04-11 13:38:04,193][ INFO][PID:3018406] Downloading the builder-live.log file, attempt 4
[2024-04-11 13:38:04,205][ INFO][PID:3018406] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@44.201.230.63 copr-rpmbuild-log
[2024-04-11 13:39:04,324][ ERROR][PID:3018406] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2024-04-11 13:39:14,326][ ERROR][PID:3018406] Giving up for unstable SSH, failures: 1712842513.937551, 1712842594.0117557, 1712842674.1923764, 1712842754.3257117
[2024-04-11 13:39:14,329][ ERROR][PID:3018406] Re-try request for task on 'ResallocHost, ticket_id=2226623, hostname=44.201.230.63': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2024-04-11 13:39:14,330][ INFO][PID:3018406] Releasing VM back to pool
[2024-04-11 13:39:14,374][ INFO][PID:3018406] Retry #1 (on other host)
[2024-04-11 13:39:14,375][ INFO][PID:3018406] Trying to allocate VM
[2024-04-11 13:45:32,885][ INFO][PID:3018406] Allocated host ResallocHost, ticket_id=2226909, hostname=2620:52:3:1:dead:beef:cafe:c196
[2024-04-11 13:45:32,886][ INFO][PID:3018406] Allocating ssh connection to builder
[2024-04-11 13:45:32,886][ INFO][PID:3018406] Checking that builder machine is OK
[2024-04-11 13:45:33,513][ INFO][PID:3018406] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c196 '/usr/bin/test -f /etc/mock/fedora-40-x86_64.cfg'
[2024-04-11 13:45:33,880][ INFO][PID:3018406] Filling build.info file with builder info
[2024-04-11 13:45:33,890][ INFO][PID:3018406] Not re-notifying FE and msg buses for the new host.
[2024-04-11 13:45:33,890][ INFO][PID:3018406] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/7296969-fedora-40-x86_64 --chroot fedora-40-x86_64 --detached
[2024-04-11 13:45:34,534][ INFO][PID:3018406] Downloading the builder-live.log file, attempt 1
[2024-04-11 13:45:34,549][ INFO][PID:3018406] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c196 copr-rpmbuild-log
[2024-04-11 13:47:34,557][ INFO][PID:3018406] Status: False
[2024-04-11 13:57:34,657][ INFO][PID:3018406] Status: False
[2024-04-11 14:07:34,751][ INFO][PID:3018406] Status: False
[2024-04-11 14:17:34,870][ INFO][PID:3018406] Status: False
[2024-04-11 14:27:35,091][ INFO][PID:3018406] Status: False
[2024-04-11 14:37:35,263][ INFO][PID:3018406] Status: False
[2024-04-11 14:47:35,377][ INFO][PID:3018406] Status: False
[2024-04-11 14:57:35,490][ INFO][PID:3018406] Status: False
[2024-04-11 15:07:35,633][ INFO][PID:3018406] Status: False
[2024-04-11 15:17:35,771][ INFO][PID:3018406] Status: False
[2024-04-11 15:27:35,921][ INFO][PID:3018406] Status: False
[2024-04-11 15:37:36,041][ INFO][PID:3018406] Status: False
[2024-04-11 15:47:36,143][ INFO][PID:3018406] Status: False
[2024-04-11 15:55:00,561][ INFO][PID:3018406] Downloading results from builder
[2024-04-11 15:55:00,624][ INFO][PID:3018406] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c196]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython started
[2024-04-11 15:55:00,626][ INFO][PID:3018406] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c196]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/ &> /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/build-07296969.rsync.log
[2024-04-11 15:55:01,295][ INFO][PID:3018406] rsyncing finished.
[2024-04-11 15:55:01,296][ INFO][PID:3018406] Releasing VM back to pool
[2024-04-11 15:55:01,307][ INFO][PID:3018406] Searching for 'success' file in resultdir
[2024-04-11 15:55:01,308][ INFO][PID:3018406] Going to sign pkgs from source: 7296969-fedora-40-x86_64 in chroot: /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64
[2024-04-11 15:55:01,309][ INFO][PID:3018406] Calling '/bin/sign -u torsava#automatic-obsoletes@copr.fedorahosted.org -p' (attempt #1)
[2024-04-11 15:55:01,597][ INFO][PID:3018406] Calling '/bin/sign -4 -h sha256 -u torsava#automatic-obsoletes@copr.fedorahosted.org -r /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/Cython-3.0.9-2.fc40.src.rpm' (attempt #1)
[2024-04-11 15:55:02,459][ INFO][PID:3018406] signed rpm: /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/Cython-3.0.9-2.fc40.src.rpm
[2024-04-11 15:55:02,460][ INFO][PID:3018406] Calling '/bin/sign -4 -h sha256 -u torsava#automatic-obsoletes@copr.fedorahosted.org -r /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/Cython-debugsource-3.0.9-2.fc40.x86_64.rpm' (attempt #1)
[2024-04-11 15:55:03,476][ INFO][PID:3018406] signed rpm: /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/Cython-debugsource-3.0.9-2.fc40.x86_64.rpm
[2024-04-11 15:55:03,477][ INFO][PID:3018406] Calling '/bin/sign -4 -h sha256 -u torsava#automatic-obsoletes@copr.fedorahosted.org -r /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/python3-cython-debuginfo-3.0.9-2.fc40.x86_64.rpm' (attempt #1)
[2024-04-11 15:55:04,327][ INFO][PID:3018406] signed rpm: /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/python3-cython-debuginfo-3.0.9-2.fc40.x86_64.rpm
[2024-04-11 15:55:04,328][ INFO][PID:3018406] Calling '/bin/sign -4 -h sha256 -u torsava#automatic-obsoletes@copr.fedorahosted.org -r /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/python3-cython-3.0.9-2.fc40.x86_64.rpm' (attempt #1)
[2024-04-11 15:55:04,983][ INFO][PID:3018406] signed rpm: /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/python3-cython-3.0.9-2.fc40.x86_64.rpm
[2024-04-11 15:55:04,984][ INFO][PID:3018406] Sign done
[2024-04-11 15:55:04,985][ INFO][PID:3018406] Incremental createrepo run, adding 07296969-Cython into https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes/fedora-40-x86_64, (auto-create-repo=True)
[2024-04-11 15:55:04,987][ INFO][PID:3018406] Running command 'copr-repo --batched /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64 --add 07296969-Cython --no-appstream-metadata' as PID 3905389
[2024-04-11 15:55:18,726][ INFO][PID:3018406] Finished after 13 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64 --add 07296969-Cython --no-appstream-metadata)
[2024-04-11 15:55:18,727][ INFO][PID:3018406] Getting build details
[2024-04-11 15:55:18,728][ INFO][PID:3018406] Listing built binary packages in /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython
[2024-04-11 15:55:18,728][ INFO][PID:3018406] Built packages:
['python3-cython 3.0.9', 'Cython-debugsource 3.0.9', 'python3-cython-debuginfo 3.0.9']
[2024-04-11 15:55:18,729][ INFO][PID:3018406] build details: {'built_packages': 'python3-cython 3.0.9\nCython-debugsource 3.0.9\npython3-cython-debuginfo 3.0.9'}
[2024-04-11 15:55:18,729][ INFO][PID:3018406] Retrieving pubkey
[2024-04-11 15:55:18,730][ INFO][PID:3018406] Calling '/bin/sign -u torsava#automatic-obsoletes@copr.fedorahosted.org -p' (attempt #1)
[2024-04-11 15:55:19,205][ INFO][PID:3018406] Added pubkey for user torsava project automatic-obsoletes into: /var/lib/copr/public_html/results/torsava/automatic-obsoletes/pubkey.gpg
[2024-04-11 15:55:19,206][ INFO][PID:3018406] Finished build: id=7296969 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/torsava/automatic-obsoletes chroot=fedora-40-x86_64
[2024-04-11 15:55:19,208][ INFO][PID:3018406] Worker succeeded build, took 7785.31874537468
[2024-04-11 15:55:19,210][ INFO][PID:3018406] Sending build state back to frontend: {
"builds": [
{
"timeout": 18000,
"frontend_base_url": "https://copr.fedorainfracloud.org",
"memory_reqs": 2048,
"enable_net": false,
"project_owner": "torsava",
"project_name": "automatic-obsoletes",
"project_dirname": "automatic-obsoletes",
"submitter": "https://src.fedoraproject.org/user/churchyard",
"ended_on": 1712850919.2087922,
"started_on": 1712843133.8900468,
"submitted_on": null,
"status": 1,
"chroot": "fedora-40-x86_64",
"arch": "x86_64",
"buildroot_pkgs": [],
"task_id": "7296969-fedora-40-x86_64",
"build_id": 7296969,
"package_name": "Cython",
"package_version": "3.0.9-2",
"git_repo": "https://copr-dist-git.fedorainfracloud.org/git/torsava/automatic-obsoletes/Cython",
"git_hash": "7238380e4db54f5935b3fb10513129f9e34cf745",
"git_branch": null,
"source_type": null,
"source_json": null,
"pkg_name": null,
"pkg_main_version": null,
"pkg_epoch": null,
"pkg_release": null,
"srpm_url": null,
"uses_devel_repo": false,
"sandbox": "torsava/automatic-obsoletes--https://src.fedoraproject.org/user/churchyard",
"results": {
"packages": [
{
"name": "python3-cython",
"epoch": null,
"version": "3.0.9",
"release": "2.fc40",
"arch": "x86_64"
},
{
"name": "Cython-debugsource",
"epoch": null,
"version": "3.0.9",
"release": "2.fc40",
"arch": "x86_64"
},
{
"name": "Cython",
"epoch": null,
"version": "3.0.9",
"release": "2.fc40",
"arch": "src"
},
{
"name": "python3-cython-debuginfo",
"epoch": null,
"version": "3.0.9",
"release": "2.fc40",
"arch": "x86_64"
}
]
},
"appstream": false,
"allow_user_ssh": false,
"ssh_public_keys": null,
"background": true,
"fedora_review": false,
"isolation": "default",
"repo_priority": null,
"repos": [
{
"baseurl": "https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes/fedora-40-x86_64/",
"id": "copr_base",
"name": "Copr repository",
"priority": null
}
],
"tags": [
"arch_x86_64"
],
"with_opts": [],
"without_opts": [],
"destdir": "/var/lib/copr/public_html/results/torsava/automatic-obsoletes",
"results_repo_url": "https://download.copr.fedorainfracloud.org/results/torsava/automatic-obsoletes",
"result_dir": "07296969-Cython",
"built_packages": "python3-cython 3.0.9\nCython-debugsource 3.0.9\npython3-cython-debuginfo 3.0.9",
"id": 7296969,
"mockchain_macros": {
"copr_username": "torsava",
"copr_projectname": "automatic-obsoletes",
"vendor": "Fedora Project COPR (torsava/automatic-obsoletes)"
}
}
]
}
[2024-04-11 15:55:19,293][ INFO][PID:3018406] Sending fedora-messaging bus message in build.end
[2024-04-11 15:55:19,320][ INFO][PID:3018406] Compressing /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/builder-live.log by gzip
[2024-04-11 15:55:19,322][ INFO][PID:3018406] Running command 'gzip /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/builder-live.log' as PID 3906744
[2024-04-11 15:55:19,415][ INFO][PID:3018406] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/builder-live.log)
[2024-04-11 15:55:19,416][ INFO][PID:3018406] Compressing /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/backend.log by gzip
[2024-04-11 15:55:19,418][ INFO][PID:3018406] Running command 'gzip /var/lib/copr/public_html/results/torsava/automatic-obsoletes/fedora-40-x86_64/07296969-Cython/backend.log' as PID 3906762