[2026-01-16 19:59:09,995][  INFO][PID:475391] Marking build as starting
[2026-01-16 19:59:10,056][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:10,062][  INFO][PID:475391] VM allocation process starts
[2026-01-16 19:59:10,107][  INFO][PID:475391] Trying to allocate VM: ResallocHost, ticket_id=866845, requested_tags=['copr_builder', 'arch_x86_64']
[2026-01-16 19:59:13,132][  INFO][PID:475391] Allocated host ResallocHost, ticket_id=866845, hostname=52.91.208.131, name=aws_x86_64_spot_prod_01316655_20260116_195556, requested_tags=['copr_builder', 'arch_x86_64']
[2026-01-16 19:59:13,133][  INFO][PID:475391] Allocating ssh connection to builder
[2026-01-16 19:59:13,134][  INFO][PID:475391] Checking that builder machine is OK
[2026-01-16 19:59:13,484][  INFO][PID:475391] Running remote command: copr-builder-ready srpm-builds
[2026-01-16 19:59:13,695][  INFO][PID:475391] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2026-01-16 19:59:13,696][  INFO][PID:475391] Filling build.info file with builder info
[2026-01-16 19:59:13,697][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:13,698][  INFO][PID:475391] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "averbyts",
            "project_name": "HEPrpms",
            "project_dirname": "HEPrpms",
            "submitter": "averbyts",
            "ended_on": null,
            "started_on": 1768593553.6980774,
            "submitted_on": null,
            "status": 3,
            "chroot": "srpm-builds",
            "arch": "x86_64",
            "buildroot_pkgs": null,
            "task_id": "10024008",
            "build_id": 10024008,
            "package_name": null,
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 9,
            "source_json": "{\"script\": \"#!/bin/bash\\ngit clone --depth 1 https://github.com/andriish/HEPrpms.git\\ncd HEPrpms\\nsh srpmsbuild.sh  cascade 3.3.3\\n\", \"chroot\": \"epel-8-x86_64\", \"builddeps\": \"git spectool wget\", \"resultdir\": \"HEPrpms/cascade/3.3.3/rpmbuild/SOURCES/\", \"repos\": \"\"}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "averbyts/HEPrpms--averbyts",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": false,
            "destdir": "/var/lib/copr/public_html/results/averbyts/HEPrpms",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/averbyts/HEPrpms",
            "result_dir": "10024008",
            "built_packages": "",
            "tags": [
                "arch_x86_64"
            ],
            "id": 10024008,
            "mockchain_macros": {
                "copr_username": "averbyts",
                "copr_projectname": "HEPrpms",
                "vendor": "Fedora Project COPR (averbyts/HEPrpms)"
            }
        }
    ]
}
[2026-01-16 19:59:13,741][  INFO][PID:475391] Sending fedora-messaging bus message in build.start
[2026-01-16 19:59:15,056][  INFO][PID:475391] Sending fedora-messaging bus message in chroot.start
[2026-01-16 19:59:15,155][  INFO][PID:475391] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10024008 --detached
[2026-01-16 19:59:15,499][  INFO][PID:475391] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '52.91.208.131' (ED25519) to the list of known hosts.


[2026-01-16 19:59:15,500][  INFO][PID:475391] Downloading the builder-live.log file, attempt 1
[2026-01-16 19:59:15,502][  INFO][PID:475391] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.91.208.131 copr-rpmbuild-log
[2026-01-16 19:59:20,508][  INFO][PID:475391] Periodic builder liveness probe: alive
[2026-01-16 19:59:20,508][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:25,509][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:30,511][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:35,514][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:40,515][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:45,516][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:50,517][  INFO][PID:475391] Checking for cancel request
[2026-01-16 19:59:55,519][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:00,521][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:05,522][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:10,524][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:15,525][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:20,527][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:25,528][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:30,529][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:35,531][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:40,535][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:45,536][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:50,537][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:55,539][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:00:56,532][ ERROR][PID:475391] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-16 20:01:00,540][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:05,543][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:06,538][WARNING][PID:475391] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-16 20:01:10,544][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:15,546][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:16,539][  INFO][PID:475391] Downloading the builder-live.log file, attempt 2
[2026-01-16 20:01:16,540][  INFO][PID:475391] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.91.208.131 copr-rpmbuild-log
[2026-01-16 20:01:20,547][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:25,548][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:30,550][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:35,554][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:40,555][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:45,557][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:50,558][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:01:55,559][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:00,561][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:05,562][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:10,568][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:15,570][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:16,608][ ERROR][PID:475391] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-16 20:02:20,571][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:25,573][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:26,637][WARNING][PID:475391] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-16 20:02:30,577][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:35,578][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:36,638][  INFO][PID:475391] Downloading the builder-live.log file, attempt 3
[2026-01-16 20:02:36,640][  INFO][PID:475391] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.91.208.131 copr-rpmbuild-log
[2026-01-16 20:02:40,579][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:45,580][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:50,582][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:02:55,583][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:00,584][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:05,585][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:10,586][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:15,587][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:20,590][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:25,595][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:30,596][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:35,601][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:36,708][ ERROR][PID:475391] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-16 20:03:40,602][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:45,603][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:46,712][WARNING][PID:475391] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2026-01-16 20:03:50,604][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:55,606][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:03:56,713][  INFO][PID:475391] Downloading the builder-live.log file, attempt 4
[2026-01-16 20:03:56,715][  INFO][PID:475391] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@52.91.208.131 copr-rpmbuild-log
[2026-01-16 20:04:00,607][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:05,607][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:10,608][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:15,609][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:20,611][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:25,613][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:30,614][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:35,615][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:40,616][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:45,617][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:50,619][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:55,620][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:04:56,783][ ERROR][PID:475391] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2026-01-16 20:05:00,621][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:05,622][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:06,801][ ERROR][PID:475391] Giving up for unstable SSH, failures: 1768593666.538412, 1768593746.6375911, 1768593826.7128658, 1768593906.801257
[2026-01-16 20:05:06,802][ ERROR][PID:475391] Re-try request for task on 'ResallocHost, ticket_id=866845, hostname=52.91.208.131, name=aws_x86_64_spot_prod_01316655_20260116_195556, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2026-01-16 20:05:06,802][  INFO][PID:475391] Releasing VM back to pool
[2026-01-16 20:05:06,812][  INFO][PID:475391] Retry #1 (on other host)
[2026-01-16 20:05:06,812][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:06,813][  INFO][PID:475391] VM allocation process starts
[2026-01-16 20:05:06,822][  INFO][PID:475391] Trying to allocate VM: ResallocHost, ticket_id=866963, requested_tags=['copr_builder', 'arch_x86_64']
[2026-01-16 20:05:09,841][  INFO][PID:475391] Allocated host ResallocHost, ticket_id=866963, hostname=3.90.53.168, name=aws_x86_64_normalreserved_prod_01316893_20260116_200342, requested_tags=['copr_builder', 'arch_x86_64']
[2026-01-16 20:05:09,842][  INFO][PID:475391] Allocating ssh connection to builder
[2026-01-16 20:05:09,843][  INFO][PID:475391] Checking that builder machine is OK
[2026-01-16 20:05:10,150][  INFO][PID:475391] Running remote command: copr-builder-ready srpm-builds
[2026-01-16 20:05:10,343][  INFO][PID:475391] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2026-01-16 20:05:10,344][  INFO][PID:475391] Filling build.info file with builder info
[2026-01-16 20:05:10,351][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:10,352][  INFO][PID:475391] Not re-notifying FE and msg buses for the new host.
[2026-01-16 20:05:10,352][  INFO][PID:475391] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/10024008 --detached
[2026-01-16 20:05:10,675][  INFO][PID:475391] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '3.90.53.168' (ED25519) to the list of known hosts.


[2026-01-16 20:05:10,675][  INFO][PID:475391] Downloading the builder-live.log file, attempt 1
[2026-01-16 20:05:10,677][  INFO][PID:475391] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.90.53.168 copr-rpmbuild-log
[2026-01-16 20:05:15,676][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:20,678][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:25,680][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:30,681][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:35,682][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:40,683][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:45,685][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:50,687][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:05:55,688][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:00,689][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:05,691][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:10,695][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:15,701][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:20,702][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:25,704][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:30,705][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:35,706][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:40,708][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:45,709][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:50,710][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:06:55,713][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:07:00,715][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:07:05,716][  INFO][PID:475391] Checking for cancel request
[2026-01-16 20:07:09,764][  INFO][PID:475391] Downloading results from builder
[2026-01-16 20:07:09,765][  INFO][PID:475391] rsyncing of mockbuilder@3.90.53.168:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008 started
[2026-01-16 20:07:09,767][  INFO][PID:475391] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@3.90.53.168:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008/ &> /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008/build-10024008.rsync.log
[2026-01-16 20:07:11,703][  INFO][PID:475391] rsyncing finished.
[2026-01-16 20:07:11,704][  INFO][PID:475391] Releasing VM back to pool
[2026-01-16 20:07:11,722][  INFO][PID:475391] Searching for 'success' file in resultdir
[2026-01-16 20:07:11,725][  INFO][PID:475391] Getting build details
[2026-01-16 20:07:11,726][  INFO][PID:475391] Retrieving SRPM info from /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008
[2026-01-16 20:07:11,726][  INFO][PID:475391] SRPM URL: https://download.copr.fedorainfracloud.org/results/averbyts/HEPrpms/srpm-builds/10024008/cascade-3.3.3-5.src.rpm
[2026-01-16 20:07:11,727][  INFO][PID:475391] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/averbyts/HEPrpms/srpm-builds/10024008/cascade-3.3.3-5.src.rpm', 'pkg_name': 'cascade', 'pkg_version': '3.3.3-5'}
[2026-01-16 20:07:11,727][  INFO][PID:475391] Finished build: id=10024008 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/averbyts/HEPrpms chroot=srpm-builds 
[2026-01-16 20:07:11,729][  INFO][PID:475391] Worker succeeded build, took 121.37790083885193
[2026-01-16 20:07:11,730][  INFO][PID:475391] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "averbyts",
            "project_name": "HEPrpms",
            "project_dirname": "HEPrpms",
            "submitter": "averbyts",
            "ended_on": 1768594031.7299058,
            "started_on": 1768593910.352005,
            "submitted_on": null,
            "status": 1,
            "chroot": "srpm-builds",
            "arch": "x86_64",
            "buildroot_pkgs": null,
            "task_id": "10024008",
            "build_id": 10024008,
            "package_name": null,
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 9,
            "source_json": "{\"script\": \"#!/bin/bash\\ngit clone --depth 1 https://github.com/andriish/HEPrpms.git\\ncd HEPrpms\\nsh srpmsbuild.sh  cascade 3.3.3\\n\", \"chroot\": \"epel-8-x86_64\", \"builddeps\": \"git spectool wget\", \"resultdir\": \"HEPrpms/cascade/3.3.3/rpmbuild/SOURCES/\", \"repos\": \"\"}",
            "pkg_name": "cascade",
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": "https://download.copr.fedorainfracloud.org/results/averbyts/HEPrpms/srpm-builds/10024008/cascade-3.3.3-5.src.rpm",
            "uses_devel_repo": null,
            "sandbox": "averbyts/HEPrpms--averbyts",
            "results": {
                "name": "cascade",
                "epoch": null,
                "version": "3.3.3",
                "release": "5",
                "exclusivearch": [],
                "excludearch": []
            },
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": false,
            "destdir": "/var/lib/copr/public_html/results/averbyts/HEPrpms",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/averbyts/HEPrpms",
            "result_dir": "10024008",
            "built_packages": "",
            "tags": [
                "arch_x86_64"
            ],
            "pkg_version": "3.3.3-5",
            "id": 10024008,
            "mockchain_macros": {
                "copr_username": "averbyts",
                "copr_projectname": "HEPrpms",
                "vendor": "Fedora Project COPR (averbyts/HEPrpms)"
            }
        }
    ]
}
[2026-01-16 20:07:11,783][  INFO][PID:475391] Sending fedora-messaging bus message in build.end
[2026-01-16 20:07:11,866][  INFO][PID:475391] Compressing /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008/builder-live.log by gzip
[2026-01-16 20:07:11,869][  INFO][PID:475391] Running command 'gzip /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008/builder-live.log' as PID 542612
[2026-01-16 20:07:11,880][  INFO][PID:475391] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008/builder-live.log)
[2026-01-16 20:07:11,881][  INFO][PID:475391] Compressing /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008/backend.log by gzip
[2026-01-16 20:07:11,883][  INFO][PID:475391] Running command 'gzip /var/lib/copr/public_html/results/averbyts/HEPrpms/srpm-builds/10024008/backend.log' as PID 542613