[2026-06-22 11:21:10,845][ INFO][PID:1377657] Marking build as starting [2026-06-22 11:21:10,900][ INFO][PID:1377657] Checking that https://download.copr.fedorainfracloud.org/results/thrnciar/pytest/fedora-rawhide-x86_64//repodata/repomd.xml exists [2026-06-22 11:21:11,096][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:11,147][ INFO][PID:1377657] rpmeta: predicted 2 min for python-puzpy on fedora-rawhide-x86_64 -> normal builder sufficient [2026-06-22 11:21:11,149][ INFO][PID:1377657] VM allocation process starts [2026-06-22 11:21:11,161][ INFO][PID:1377657] Trying to allocate VM: ResallocHost, ticket_id=3542478, requested_tags=['arch_x86_64', 'copr_builder'] [2026-06-22 11:21:16,163][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:17,189][ INFO][PID:1377657] Allocated host ResallocHost, ticket_id=3542478, hostname=18.207.206.47, name=aws_x86_64_reserved_prod_12511290_20260622_111119, requested_tags=['arch_x86_64', 'copr_builder'] [2026-06-22 11:21:17,191][ INFO][PID:1377657] Allocating ssh connection to builder [2026-06-22 11:21:17,192][ INFO][PID:1377657] Checking that builder machine is OK [2026-06-22 11:21:17,508][ INFO][PID:1377657] Installed copr-rpmbuild version: 1.8 [2026-06-22 11:21:17,509][ INFO][PID:1377657] Running remote command: copr-builder-ready fedora-rawhide-x86_64 [2026-06-22 11:21:17,707][ INFO][PID:1377657] Red Hat subscription not needed for fedora-rawhide-x86_64 Builder is ready to be used [2026-06-22 11:21:17,709][ INFO][PID:1377657] Filling build.info file with builder info [2026-06-22 11:21:17,711][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:17,715][ INFO][PID:1377657] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "thrnciar", "project_name": "pytest", "project_dirname": "pytest", "submitter": "thrnciar", "ended_on": null, "started_on": 1782127277.7148511, "submitted_on": null, "status": 3, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "10625217-fedora-rawhide-x86_64", "build_id": 10625217, "package_name": "python-puzpy", "package_version": "0.6.1-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/thrnciar/pytest/python-puzpy", "git_hash": "2925cc4b8b296dfb2f2dbdabac5340a60bdf5ac4", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "thrnciar/pytest--thrnciar", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/thrnciar/pytest/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/thrnciar/pytest", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/thrnciar/pytest", "result_dir": "10625217-python-puzpy", "built_packages": "", "rpmeta_prediction": 2, "id": 10625217, "mockchain_macros": { "copr_username": "thrnciar", "copr_projectname": "pytest", "vendor": "Fedora Project COPR (thrnciar/pytest)" } } ] } [2026-06-22 11:21:17,768][ INFO][PID:1377657] Sending fedora-messaging bus message in build.start [2026-06-22 11:21:18,384][ INFO][PID:1377657] Sending fedora-messaging bus message in chroot.start [2026-06-22 11:21:18,459][ INFO][PID:1377657] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10625217-fedora-rawhide-x86_64 --chroot fedora-rawhide-x86_64 --detached [2026-06-22 11:21:18,910][ INFO][PID:1377657] The copr-rpmbuild seems started, per: stdout: stderr: Warning: Permanently added '18.207.206.47' (ED25519) to the list of known hosts. [2026-06-22 11:21:18,911][ INFO][PID:1377657] Downloading the builder-live.log file, attempt 1 [2026-06-22 11:21:18,917][ INFO][PID:1377657] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@18.207.206.47 copr-rpmbuild-log [2026-06-22 11:21:23,918][ INFO][PID:1377657] Periodic builder liveness probe: alive [2026-06-22 11:21:23,919][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:28,921][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:33,925][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:38,927][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:43,930][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:48,932][ INFO][PID:1377657] Checking for cancel request [2026-06-22 11:21:50,731][ INFO][PID:1377657] Downloading results from builder [2026-06-22 11:21:50,732][ INFO][PID:1377657] rsyncing of mockbuilder@18.207.206.47:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy started [2026-06-22 11:21:50,734][ INFO][PID:1377657] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@18.207.206.47:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/ &> /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/build-10625217.rsync.log [2026-06-22 11:21:50,949][ INFO][PID:1377657] rsyncing finished. [2026-06-22 11:21:50,951][ INFO][PID:1377657] VM Release request [2026-06-22 11:21:50,965][ INFO][PID:1377657] Searching for 'success' file in resultdir [2026-06-22 11:21:50,966][ INFO][PID:1377657] Going to sign pkgs from source: 10625217-fedora-rawhide-x86_64 in chroot: /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64 [2026-06-22 11:21:50,967][ INFO][PID:1377657] Calling '/bin/sign -u thrnciar#pytest@copr.fedorahosted.org -p' (attempt #1) [2026-06-22 11:21:51,243][ INFO][PID:1377657] Calling '/bin/sign -4 -h sha256 -u thrnciar#pytest@copr.fedorahosted.org -r /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python3-puzpy-0.6.1-2.fc45.noarch.rpm' (attempt #1) [2026-06-22 11:21:51,832][ INFO][PID:1377657] signed rpm: /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python3-puzpy-0.6.1-2.fc45.noarch.rpm [2026-06-22 11:21:51,834][ INFO][PID:1377657] Calling '/bin/sign -4 -h sha256 -u thrnciar#pytest@copr.fedorahosted.org -r /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python-puzpy-0.6.1-2.fc45.src.rpm' (attempt #1) [2026-06-22 11:21:52,609][ INFO][PID:1377657] signed rpm: /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python-puzpy-0.6.1-2.fc45.src.rpm [2026-06-22 11:21:52,609][ INFO][PID:1377657] Sign done [2026-06-22 11:21:52,611][ INFO][PID:1377657] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python3-puzpy-0.6.1-2.fc45.noarch.rpm [2026-06-22 11:21:52,906][WARNING][PID:1377657] Retry request #1 on https://packages.redhat.com/api/pulp/public-copr/api/v3/content/rpm/packages/upload/: Request server error on https://packages.redhat.com/api/pulp/public-copr/api/v3/content/rpm/packages/upload/: 500 Internal Server Error [2026-06-22 11:21:58,077][ ERROR][PID:1377657] Failed to create Pulp content for: /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python3-puzpy-0.6.1-2.fc45.noarch.rpm, Bad Request {"file":["The submitted file is empty."]} (attempt #1) [2026-06-22 11:21:58,288][ INFO][PID:1377657] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python3-puzpy-0.6.1-2.fc45.noarch.rpm [2026-06-22 11:21:58,968][ INFO][PID:1377657] Pulp: create_content: /api/v3/content/rpm/packages/upload/ /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python-puzpy-0.6.1-2.fc45.src.rpm [2026-06-22 11:21:58,969][ INFO][PID:1377657] [1/2] Uploaded to Pulp: /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python3-puzpy-0.6.1-2.fc45.noarch.rpm [2026-06-22 11:21:59,399][ INFO][PID:1377657] [2/2] Uploaded to Pulp: /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/python-puzpy-0.6.1-2.fc45.src.rpm [2026-06-22 11:21:59,406][ INFO][PID:1377657] Pulp: get_repository: /api/v3/repositories/rpm/rpm/?name=thrnciar%2Fpytest%2Ffedora-rawhide-x86_64&offset=0&limit=1 [2026-06-22 11:22:10,673][ INFO][PID:1377657] Key add_remove_batched::/api/pulp/public-copr/api/v3/repositories/rpm/rpm/019eee68-cffe-75f2-9e87-32fafbe5cd0f/::1360476 already processed, skip [2026-06-22 11:22:10,840][ INFO][PID:1377657] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019eef11-05f8-796d-b93e-a03e7b3a63fe/ [2026-06-22 11:22:10,842][ INFO][PID:1377657] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019eef11-05f8-796d-b93e-a03e7b3a63fe/ [2026-06-22 11:22:15,999][ INFO][PID:1377657] Pulp: polling task status: /api/pulp/public-copr/api/v3/tasks/019eef11-05f8-796d-b93e-a03e7b3a63fe/ [2026-06-22 11:22:16,000][ INFO][PID:1377657] Pulp: get_by_href: /api/pulp/public-copr/api/v3/tasks/019eef11-05f8-796d-b93e-a03e7b3a63fe/ [2026-06-22 11:22:16,494][ ERROR][PID:1377657] Failed to modify Pulp repository content /api/pulp/public-copr/api/v3/repositories/rpm/rpm/019eee68-cffe-75f2-9e87-32fafbe5cd0f/ [2026-06-22 11:22:16,511][ ERROR][PID:1377657] Response : {'pulp_href': '/api/pulp/public-copr/api/v3/tasks/019eef11-05f8-796d-b93e-a03e7b3a63fe/', 'prn': 'prn:core.task:019eef11-05f8-796d-b93e-a03e7b3a63fe', 'pulp_created': '2026-06-22T11:22:10.812860Z', 'pulp_last_updated': '2026-06-22T11:22:10.808989Z', 'state': 'failed', 'name': 'pulp_rpm.app.tasks.signing.signed_add_and_remove', 'logging_cid': '9c7ce6bd706b460896f32bd2c1813d4d', 'created_by': None, 'unblocked_at': None, 'started_at': '2026-06-22T11:22:10.886292Z', 'finished_at': '2026-06-22T11:22:13.187276Z', 'error': {'traceback': ' File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulpcore/tasking/tasks.py", line 81, in _execute_task\n result = task_function()\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulp_rpm/app/tasks/signing.py", line 240, in signed_add_and_remove\n return add_and_remove(\n ^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulpcore/app/tasks/repository.py", line 241, in add_and_remove\n with repository.new_version(base_version=base_version) as new_version:\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulpcore/app/models/repository.py", line 1515, in __exit__\n self.delete()\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/pulpcore/app/models/repository.py", line 1422, in delete\n super().delete(**kwargs)\n File "/usr/lib64/python3.11/contextlib.py", line 81, in inner\n return func(*args, **kwds)\n ^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django_lifecycle/mixins.py", line 179, in delete\n value = super().delete(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/base.py", line 1281, in delete\n return collector.delete()\n ^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/deletion.py", line 467, in delete\n count = qs._raw_delete(using=self.using)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/query.py", line 1223, in _raw_delete\n return query.get_compiler(using).execute_sql(ROW_COUNT)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/models/sql/compiler.py", line 1623, in execute_sql\n cursor.execute(sql, params)\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/sentry_sdk/utils.py", line 1890, in runner\n return sentry_patched_function(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/sentry_sdk/integrations/django/__init__.py", line 682, in execute\n result = real_execute(self, sql, params)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute\n return self._execute_with_wrappers(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers\n return executor(sql, params, many, context)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute\n with self.db.wrap_database_errors:\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/utils.py", line 91, in __exit__\n raise dj_exc_value.with_traceback(traceback) from exc_value\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute\n return self.cursor.execute(sql, params)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/usr/local/lib/pulp/lib64/python3.11/site-packages/psycopg/cursor.py", line 117, in execute\n raise ex.with_traceback(None)\n', 'description': 'relation "service_agentscanreport_repo_versions" does not exist\nLINE 1: DELETE FROM "service_agentscanreport_repo_versions" WHERE "s...\n ^'}, 'worker': None, 'parent_task': None, 'child_tasks': [], 'task_group': None, 'progress_reports': [], 'created_resources': ['/api/pulp/public-copr/api/v3/repositories/rpm/rpm/019eee68-cffe-75f2-9e87-32fafbe5cd0f/versions/342/'], 'created_resource_prns': ['prn:core.repositoryversion:019eef11-065e-7cd2-8384-0909ce96c275'], 'reserved_resources_record': ['prn:rpm.rpmrepository:019eee68-cffe-75f2-9e87-32fafbe5cd0f', 'shared:prn:core.domain:0192c408-7ef8-7dbc-8f58-7f9fcd26a040'], 'result': None} [2026-06-22 11:22:16,539][ ERROR][PID:1377657] Build failed: [2026-06-22 11:22:16,557][ INFO][PID:1377657] Finished build: id=10625217 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/thrnciar/pytest chroot=fedora-rawhide-x86_64 [2026-06-22 11:22:16,559][ ERROR][PID:1377657] Unexpected exception (in /usr/lib/python3.14/site-packages/copr_backend/pulp.py:590) [2026-06-22 11:22:16,582][ INFO][PID:1377657] Worker failed build, took 58.868016719818115 [2026-06-22 11:22:16,584][ INFO][PID:1377657] Sending build state back to frontend: { "builds": [ { "timeout": 18000, "frontend_base_url": "https://copr.fedorainfracloud.org", "memory_reqs": 2048, "enable_net": false, "project_owner": "thrnciar", "project_name": "pytest", "project_dirname": "pytest", "submitter": "thrnciar", "ended_on": 1782127336.5828679, "started_on": 1782127277.7148511, "submitted_on": null, "status": 0, "chroot": "fedora-rawhide-x86_64", "arch": "x86_64", "buildroot_pkgs": [], "task_id": "10625217-fedora-rawhide-x86_64", "build_id": 10625217, "package_name": "python-puzpy", "package_version": "0.6.1-2", "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/thrnciar/pytest/python-puzpy", "git_hash": "2925cc4b8b296dfb2f2dbdabac5340a60bdf5ac4", "git_branch": null, "source_type": null, "source_json": null, "pkg_name": null, "pkg_main_version": null, "pkg_epoch": null, "pkg_release": null, "srpm_url": null, "uses_devel_repo": false, "sandbox": "thrnciar/pytest--thrnciar", "results": null, "appstream": false, "allow_user_ssh": false, "ssh_public_keys": null, "storage": 1, "repos": [ { "baseurl": "https://download.copr.fedorainfracloud.org/results/thrnciar/pytest/fedora-rawhide-x86_64/", "id": "copr_base", "name": "Copr repository", "priority": null } ], "background": true, "fedora_review": false, "isolation": "default", "repo_priority": null, "tags": [ "arch_x86_64" ], "with_opts": [], "without_opts": [], "destdir": "/var/lib/copr/public_html/results/thrnciar/pytest", "results_repo_url": "https://download.copr.fedorainfracloud.org/results/thrnciar/pytest", "result_dir": "10625217-python-puzpy", "built_packages": "", "rpmeta_prediction": 2, "id": 10625217, "mockchain_macros": { "copr_username": "thrnciar", "copr_projectname": "pytest", "vendor": "Fedora Project COPR (thrnciar/pytest)" } } ] } [2026-06-22 11:22:16,628][ INFO][PID:1377657] Sending fedora-messaging bus message in build.end [2026-06-22 11:22:16,662][ INFO][PID:1377657] Compressing /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/builder-live.log by gzip [2026-06-22 11:22:16,665][ INFO][PID:1377657] Running command 'gzip /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/builder-live.log' as PID 1385656 [2026-06-22 11:22:16,695][ INFO][PID:1377657] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/builder-live.log) [2026-06-22 11:22:16,696][ INFO][PID:1377657] Compressing /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/backend.log by gzip [2026-06-22 11:22:16,698][ INFO][PID:1377657] Running command 'gzip /var/lib/copr/public_html/results/thrnciar/pytest/fedora-rawhide-x86_64/10625217-python-puzpy/backend.log' as PID 1385661