[2025-09-19 11:24:44,334][  INFO][PID:722346] Marking build as starting
[2025-09-19 11:24:44,383][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:24:44,385][  INFO][PID:722346] VM allocation process starts
[2025-09-19 11:24:44,438][  INFO][PID:722346] Trying to allocate VM: ResallocHost, ticket_id=5142567, requested_tags=['copr_builder', 'arch_noarch']
[2025-09-19 11:24:49,440][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:24:54,446][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:24:59,448][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:04,450][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:09,452][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:12,547][  INFO][PID:722346] Allocated host ResallocHost, ticket_id=5142567, hostname=2620:52:3:1:dead:beef:cafe:c29a, name=copr_p09_01_prod_06830661_20250919_111711, requested_tags=['copr_builder', 'arch_noarch']
[2025-09-19 11:25:12,551][  INFO][PID:722346] Allocating ssh connection to builder
[2025-09-19 11:25:12,553][  INFO][PID:722346] Checking that builder machine is OK
[2025-09-19 11:25:13,427][  INFO][PID:722346] Running remote command: copr-builder-ready srpm-builds
[2025-09-19 11:25:13,960][  INFO][PID:722346] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2025-09-19 11:25:13,961][  INFO][PID:722346] Filling build.info file with builder info
[2025-09-19 11:25:13,962][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:13,963][  INFO][PID:722346] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@python",
            "project_name": "python3.14",
            "project_dirname": "python3.14",
            "submitter": "https://src.fedoraproject.org/user/ksurma",
            "ended_on": null,
            "started_on": 1758281113.9632218,
            "submitted_on": null,
            "status": 3,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "9577096",
            "build_id": 9577096,
            "package_name": "python-digitalocean",
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 10,
            "source_json": "{\"clone_url\": \"https://src.fedoraproject.org/rpms/python-digitalocean\", \"distgit\": \"fedora\", \"committish\": \"f56677b33a66226736c9d7808f89060ed7321e36\"}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "@python/python3.14--https://src.fedoraproject.org/user/ksurma",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": true,
            "destdir": "/var/lib/copr/public_html/results/@python/python3.14",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@python/python3.14",
            "result_dir": "09577096",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "id": 9577096,
            "mockchain_macros": {
                "copr_username": "@python",
                "copr_projectname": "python3.14",
                "vendor": "Fedora Project COPR (@python/python3.14)"
            }
        }
    ]
}
[2025-09-19 11:25:13,982][  INFO][PID:722346] Sending fedora-messaging bus message in build.start
[2025-09-19 11:25:14,789][  INFO][PID:722346] Sending fedora-messaging bus message in chroot.start
[2025-09-19 11:25:14,815][  INFO][PID:722346] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9577096 --detached
[2025-09-19 11:25:15,715][  INFO][PID:722346] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c29a' (ED25519) to the list of known hosts.


[2025-09-19 11:25:15,720][  INFO][PID:722346] Downloading the builder-live.log file, attempt 1
[2025-09-19 11:25:15,724][  INFO][PID:722346] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29a copr-rpmbuild-log
[2025-09-19 11:25:20,735][  INFO][PID:722346] Periodic builder liveness probe: alive
[2025-09-19 11:25:20,737][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:25,739][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:30,741][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:35,742][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:40,744][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:45,745][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:50,749][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:25:55,750][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:00,751][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:05,754][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:10,756][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:15,759][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:15,812][ ERROR][PID:722346] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-19 11:26:20,760][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:25,762][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:25,814][WARNING][PID:722346] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-19 11:26:30,765][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:35,770][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:35,816][  INFO][PID:722346] Downloading the builder-live.log file, attempt 2
[2025-09-19 11:26:35,820][  INFO][PID:722346] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29a copr-rpmbuild-log
[2025-09-19 11:26:40,772][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:45,775][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:50,776][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:26:55,778][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:00,782][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:05,790][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:10,793][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:15,800][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:20,804][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:25,808][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:30,809][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:35,810][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:35,909][ ERROR][PID:722346] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-19 11:27:40,813][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:45,818][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:45,914][WARNING][PID:722346] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-19 11:27:50,819][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:55,828][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:27:55,918][  INFO][PID:722346] Downloading the builder-live.log file, attempt 3
[2025-09-19 11:27:55,926][  INFO][PID:722346] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29a copr-rpmbuild-log
[2025-09-19 11:28:00,841][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:05,842][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:10,848][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:15,849][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:20,853][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:25,856][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:30,865][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:35,871][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:40,877][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:45,878][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:50,885][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:55,887][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:28:56,024][ ERROR][PID:722346] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-19 11:29:00,888][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:05,897][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:06,030][WARNING][PID:722346] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-19 11:29:10,905][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:15,908][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:16,031][  INFO][PID:722346] Downloading the builder-live.log file, attempt 4
[2025-09-19 11:29:16,037][  INFO][PID:722346] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c29a copr-rpmbuild-log
[2025-09-19 11:29:20,910][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:25,911][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:30,912][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:35,917][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:40,918][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:45,920][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:50,921][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:29:55,924][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:00,926][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:05,928][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:10,930][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:15,931][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:16,112][ ERROR][PID:722346] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-19 11:30:20,934][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:25,938][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:26,113][ ERROR][PID:722346] Giving up for unstable SSH, failures: 1758281185.8145463, 1758281265.914127, 1758281346.0302424, 1758281426.113646
[2025-09-19 11:30:26,117][ ERROR][PID:722346] Re-try request for task on 'ResallocHost, ticket_id=5142567, hostname=2620:52:3:1:dead:beef:cafe:c29a, name=copr_p09_01_prod_06830661_20250919_111711, requested_tags=['copr_builder', 'arch_noarch']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-09-19 11:30:26,119][  INFO][PID:722346] Releasing VM back to pool
[2025-09-19 11:30:26,146][  INFO][PID:722346] Retry #1 (on other host)
[2025-09-19 11:30:26,147][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:26,160][  INFO][PID:722346] VM allocation process starts
[2025-09-19 11:30:26,187][  INFO][PID:722346] Trying to allocate VM: ResallocHost, ticket_id=5142935, requested_tags=['copr_builder', 'arch_noarch']
[2025-09-19 11:30:31,189][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:36,192][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:41,193][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:46,196][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:51,202][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:30:56,203][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:01,206][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:06,208][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:11,213][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:16,219][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:21,221][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:26,224][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:31,232][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:36,248][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:41,249][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:46,250][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:51,253][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:31:56,254][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:32:01,260][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:32:06,263][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:32:11,264][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:32:14,409][  INFO][PID:722346] Allocated host ResallocHost, ticket_id=5142935, hostname=2620:52:3:1:dead:beef:cafe:c198, name=copr_hv_x86_64_03_prod_06830903_20250919_112925, requested_tags=['copr_builder', 'arch_noarch']
[2025-09-19 11:32:14,411][  INFO][PID:722346] Allocating ssh connection to builder
[2025-09-19 11:32:14,414][  INFO][PID:722346] Checking that builder machine is OK
[2025-09-19 11:32:14,975][  INFO][PID:722346] Running remote command: copr-builder-ready srpm-builds
[2025-09-19 11:32:15,328][  INFO][PID:722346] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2025-09-19 11:32:15,330][  INFO][PID:722346] Filling build.info file with builder info
[2025-09-19 11:32:15,334][  INFO][PID:722346] Checking for cancel request
[2025-09-19 11:32:15,340][  INFO][PID:722346] Not re-notifying FE and msg buses for the new host.
[2025-09-19 11:32:15,346][  INFO][PID:722346] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9577096 --detached
[2025-09-19 11:32:16,006][  INFO][PID:722346] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c198' (ED25519) to the list of known hosts.


[2025-09-19 11:32:16,009][  INFO][PID:722346] Downloading the builder-live.log file, attempt 1
[2025-09-19 11:32:16,012][  INFO][PID:722346] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c198 copr-rpmbuild-log
[2025-09-19 11:32:18,224][  INFO][PID:722346] Downloading results from builder
[2025-09-19 11:32:18,226][  INFO][PID:722346] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c198]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096 started
[2025-09-19 11:32:18,231][  INFO][PID:722346] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c198]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096/ &> /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096/build-09577096.rsync.log
[2025-09-19 11:32:18,700][  INFO][PID:722346] rsyncing finished.
[2025-09-19 11:32:18,704][  INFO][PID:722346] Releasing VM back to pool
[2025-09-19 11:32:18,730][  INFO][PID:722346] Searching for 'success' file in resultdir
[2025-09-19 11:32:18,735][  INFO][PID:722346] Getting build details
[2025-09-19 11:32:18,737][  INFO][PID:722346] Retrieving SRPM info from /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096
[2025-09-19 11:32:18,739][  INFO][PID:722346] SRPM URL: https://download.copr.fedorainfracloud.org/results/@python/python3.14/srpm-builds/09577096/python-digitalocean-1.17.0-19.src.rpm
[2025-09-19 11:32:18,740][  INFO][PID:722346] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/@python/python3.14/srpm-builds/09577096/python-digitalocean-1.17.0-19.src.rpm', 'pkg_name': 'python-digitalocean', 'pkg_version': '1.17.0-19'}
[2025-09-19 11:32:18,741][  INFO][PID:722346] Finished build: id=9577096 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/@python/python3.14 chroot=srpm-builds 
[2025-09-19 11:32:18,746][  INFO][PID:722346] Worker succeeded build, took 3.4059557914733887
[2025-09-19 11:32:18,748][  INFO][PID:722346] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@python",
            "project_name": "python3.14",
            "project_dirname": "python3.14",
            "submitter": "https://src.fedoraproject.org/user/ksurma",
            "ended_on": 1758281538.7461858,
            "started_on": 1758281535.34023,
            "submitted_on": null,
            "status": 1,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "9577096",
            "build_id": 9577096,
            "package_name": "python-digitalocean",
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 10,
            "source_json": "{\"clone_url\": \"https://src.fedoraproject.org/rpms/python-digitalocean\", \"distgit\": \"fedora\", \"committish\": \"f56677b33a66226736c9d7808f89060ed7321e36\"}",
            "pkg_name": "python-digitalocean",
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": "https://download.copr.fedorainfracloud.org/results/@python/python3.14/srpm-builds/09577096/python-digitalocean-1.17.0-19.src.rpm",
            "uses_devel_repo": null,
            "sandbox": "@python/python3.14--https://src.fedoraproject.org/user/ksurma",
            "results": {
                "name": "python-digitalocean",
                "epoch": null,
                "version": "1.17.0",
                "release": "19",
                "exclusivearch": [],
                "excludearch": []
            },
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": true,
            "destdir": "/var/lib/copr/public_html/results/@python/python3.14",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@python/python3.14",
            "result_dir": "09577096",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "pkg_version": "1.17.0-19",
            "id": 9577096,
            "mockchain_macros": {
                "copr_username": "@python",
                "copr_projectname": "python3.14",
                "vendor": "Fedora Project COPR (@python/python3.14)"
            }
        }
    ]
}
[2025-09-19 11:32:18,808][  INFO][PID:722346] Sending fedora-messaging bus message in build.end
[2025-09-19 11:32:18,868][  INFO][PID:722346] Compressing /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096/builder-live.log by gzip
[2025-09-19 11:32:18,871][  INFO][PID:722346] Running command 'gzip /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096/builder-live.log' as PID 820398
[2025-09-19 11:32:18,875][  INFO][PID:722346] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096/builder-live.log)
[2025-09-19 11:32:18,879][  INFO][PID:722346] Compressing /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096/backend.log by gzip
[2025-09-19 11:32:18,881][  INFO][PID:722346] Running command 'gzip /var/lib/copr/public_html/results/@python/python3.14/srpm-builds/09577096/backend.log' as PID 820400