[2025-09-08 12:48:18,907][  INFO][PID:3502001] Marking build as starting
[2025-09-08 12:48:18,974][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:48:18,975][  INFO][PID:3502001] VM allocation process starts
[2025-09-08 12:48:18,988][  INFO][PID:3502001] Trying to allocate VM: ResallocHost, ticket_id=4963585, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-08 12:48:23,989][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:48:28,991][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:48:33,992][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:48:38,992][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:48:43,993][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:48:48,994][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:48:53,995][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:48:58,996][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:03,997][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:08,998][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:13,999][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:19,000][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:24,001][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:29,002][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:34,002][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:39,004][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:44,007][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:49,008][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:54,009][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:49:59,010][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:04,011][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:09,012][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:14,013][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:19,014][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:24,015][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:29,016][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:34,017][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:39,018][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:44,019][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:49,020][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:54,021][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:50:59,022][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:04,023][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:09,024][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:14,025][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:19,026][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:24,027][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:29,028][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:34,029][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:39,030][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:44,031][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:49,032][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:54,033][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:51:59,034][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:04,035][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:09,036][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:14,037][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:19,038][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:24,038][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:29,039][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:34,040][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:39,041][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:44,042][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:49,043][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:54,044][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:52:59,045][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:04,046][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:09,047][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:14,048][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:19,097][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:24,098][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:29,099][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:34,100][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:39,101][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:44,102][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:49,103][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:54,104][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:53:59,105][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:04,106][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:09,107][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:14,108][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:19,109][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:24,110][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:29,111][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:34,112][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:39,112][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:44,113][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:49,114][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:54,115][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:54:59,116][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:04,117][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:09,118][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:14,119][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:19,120][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:24,121][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:29,122][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:34,123][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:39,123][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:44,124][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:49,125][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:54,127][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:55:59,127][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:04,128][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:09,129][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:14,130][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:19,131][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:24,132][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:29,133][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:34,134][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:39,135][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:44,136][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:49,137][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:54,137][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:56:59,138][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:04,139][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:09,140][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:14,141][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:19,142][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:24,143][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:29,145][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:34,146][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:39,147][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:44,148][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:49,149][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:54,150][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:57:59,150][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:04,151][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:09,152][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:14,153][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:19,154][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:24,155][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:29,156][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:34,157][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:39,158][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:44,159][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:49,160][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:54,161][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:58:59,162][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:04,163][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:09,164][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:14,165][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:19,166][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:24,167][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:29,168][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:34,169][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:37,127][  INFO][PID:3502001] Allocated host ResallocHost, ticket_id=4963585, hostname=54.167.111.167, name=aws_x86_64_spot_prod_06668404_20250908_125722, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-08 12:59:37,127][  INFO][PID:3502001] Allocating ssh connection to builder
[2025-09-08 12:59:37,127][  INFO][PID:3502001] Checking that builder machine is OK
[2025-09-08 12:59:37,558][  INFO][PID:3502001] Running remote command: copr-builder-ready rhel-9-x86_64
[2025-09-08 12:59:37,751][  INFO][PID:3502001] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-09-08 12:59:37,752][  INFO][PID:3502001] Filling build.info file with builder info
[2025-09-08 12:59:37,752][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:37,753][  INFO][PID:3502001] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "rhcontainerbot",
            "project_name": "bootc",
            "project_dirname": "bootc",
            "submitter": "packit",
            "ended_on": null,
            "started_on": 1757336377.7530985,
            "submitted_on": null,
            "status": 3,
            "chroot": "rhel-9-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9533981-rhel-9-x86_64",
            "build_id": 9533981,
            "package_name": "bootc",
            "package_version": "202509081112.g2d5b977af4-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/bootc/bootc",
            "git_hash": "204d99ee4949f9007e4a1109bfa7a3daad168d66",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rhcontainerbot/bootc--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/rhel-9-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/bootc",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc",
            "result_dir": "09533981-bootc",
            "built_packages": "",
            "id": 9533981,
            "mockchain_macros": {
                "copr_username": "rhcontainerbot",
                "copr_projectname": "bootc",
                "vendor": "Fedora Project COPR (rhcontainerbot/bootc)"
            }
        }
    ]
}
[2025-09-08 12:59:37,837][  INFO][PID:3502001] Sending fedora-messaging bus message in build.start
[2025-09-08 12:59:38,459][  INFO][PID:3502001] Sending fedora-messaging bus message in chroot.start
[2025-09-08 12:59:38,482][  INFO][PID:3502001] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9533981-rhel-9-x86_64 --chroot rhel-9-x86_64 --detached
[2025-09-08 12:59:38,802][  INFO][PID:3502001] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '54.167.111.167' (ED25519) to the list of known hosts.


[2025-09-08 12:59:38,803][  INFO][PID:3502001] Downloading the builder-live.log file, attempt 1
[2025-09-08 12:59:38,804][  INFO][PID:3502001] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.167.111.167 copr-rpmbuild-log
[2025-09-08 12:59:43,808][  INFO][PID:3502001] Periodic builder liveness probe: alive
[2025-09-08 12:59:43,808][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:48,809][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:53,810][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 12:59:58,811][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:03,812][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:08,813][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:13,815][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:18,816][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:23,817][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:28,817][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:33,818][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:38,819][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:00:39,141][  INFO][PID:3502001] Downloading results from builder
[2025-09-08 13:00:39,142][  INFO][PID:3502001] rsyncing of mockbuilder@54.167.111.167:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc started
[2025-09-08 13:00:39,350][  INFO][PID:3502001] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.167.111.167:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/ &> /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/build-09533981.rsync.log
[2025-09-08 13:35:07,297][  INFO][PID:3502001] rsyncing finished.
[2025-09-08 13:35:07,297][ ERROR][PID:3502001] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:35:07,503][ ERROR][PID:3502001] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:35:17,503][  INFO][PID:3502001] rsyncing of mockbuilder@54.167.111.167:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc started
[2025-09-08 13:35:17,504][  INFO][PID:3502001] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.167.111.167:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/ &> /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/build-09533981.rsync.log
[2025-09-08 13:36:17,582][  INFO][PID:3502001] rsyncing finished.
[2025-09-08 13:36:31,518][ ERROR][PID:3502001] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:36:31,518][ ERROR][PID:3502001] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:36:41,519][  INFO][PID:3502001] rsyncing of mockbuilder@54.167.111.167:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc started
[2025-09-08 13:36:41,520][  INFO][PID:3502001] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@54.167.111.167:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/ &> /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/build-09533981.rsync.log
[2025-09-08 13:37:41,596][  INFO][PID:3502001] rsyncing finished.
[2025-09-08 13:37:41,597][ ERROR][PID:3502001] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:37:41,597][ ERROR][PID:3502001] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:37:51,598][ ERROR][PID:3502001] Re-try request for task on 'ResallocHost, ticket_id=4963585, hostname=54.167.111.167, name=aws_x86_64_spot_prod_06668404_20250908_125722, requested_tags=['copr_builder', 'arch_x86_64']': Unable to finish after 3 SSH attempts
[2025-09-08 13:37:51,598][  INFO][PID:3502001] Releasing VM back to pool
[2025-09-08 13:37:51,607][  INFO][PID:3502001] Retry #1 (on other host)
[2025-09-08 13:37:51,608][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:37:51,608][  INFO][PID:3502001] VM allocation process starts
[2025-09-08 13:37:51,638][  INFO][PID:3502001] Trying to allocate VM: ResallocHost, ticket_id=4963704, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-08 13:37:56,639][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:38:01,640][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:38:24,110][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:38:29,145][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:38:44,237][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:39:01,310][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:39:06,311][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:39:11,312][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:39:17,766][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:39:19,727][  INFO][PID:3502001] Allocated host ResallocHost, ticket_id=4963704, hostname=35.170.76.120, name=aws_x86_64_spot_prod_06668588_20250908_133746, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-08 13:39:19,727][  INFO][PID:3502001] Allocating ssh connection to builder
[2025-09-08 13:39:19,934][  INFO][PID:3502001] Checking that builder machine is OK
[2025-09-08 13:39:20,398][  INFO][PID:3502001] Running remote command: copr-builder-ready rhel-9-x86_64
[2025-09-08 13:39:20,591][  INFO][PID:3502001] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-09-08 13:39:20,592][  INFO][PID:3502001] Filling build.info file with builder info
[2025-09-08 13:39:20,596][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:39:20,798][  INFO][PID:3502001] Not re-notifying FE and msg buses for the new host.
[2025-09-08 13:39:20,799][  INFO][PID:3502001] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9533981-rhel-9-x86_64 --chroot rhel-9-x86_64 --detached
[2025-09-08 13:39:21,105][  INFO][PID:3502001] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '35.170.76.120' (ED25519) to the list of known hosts.


[2025-09-08 13:39:21,106][  INFO][PID:3502001] Downloading the builder-live.log file, attempt 1
[2025-09-08 13:39:21,107][  INFO][PID:3502001] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@35.170.76.120 copr-rpmbuild-log
[2025-09-08 13:39:26,112][  INFO][PID:3502001] Periodic builder liveness probe: alive
[2025-09-08 13:39:54,190][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:39:59,191][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:40:04,192][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:40:09,192][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:40:14,193][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:40:18,084][  INFO][PID:3502001] Downloading results from builder
[2025-09-08 13:40:18,084][  INFO][PID:3502001] rsyncing of mockbuilder@35.170.76.120:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc started
[2025-09-08 13:40:18,085][  INFO][PID:3502001] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@35.170.76.120:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/ &> /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/build-09533981.rsync.log
[2025-09-08 13:53:03,382][  INFO][PID:3502001] rsyncing finished.
[2025-09-08 13:53:03,382][ ERROR][PID:3502001] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:53:03,383][ ERROR][PID:3502001] SSH connection lost on #1 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:53:13,383][  INFO][PID:3502001] rsyncing of mockbuilder@35.170.76.120:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc started
[2025-09-08 13:53:14,630][  INFO][PID:3502001] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@35.170.76.120:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/ &> /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/build-09533981.rsync.log
[2025-09-08 13:54:14,707][  INFO][PID:3502001] rsyncing finished.
[2025-09-08 13:55:31,870][ ERROR][PID:3502001] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:55:31,870][ ERROR][PID:3502001] SSH connection lost on #2 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:55:41,871][  INFO][PID:3502001] rsyncing of mockbuilder@35.170.76.120:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc started
[2025-09-08 13:55:41,872][  INFO][PID:3502001] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@35.170.76.120:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/ &> /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/build-09533981.rsync.log
[2025-09-08 13:56:41,951][  INFO][PID:3502001] rsyncing finished.
[2025-09-08 13:56:42,574][ ERROR][PID:3502001] Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:56:42,574][ ERROR][PID:3502001] SSH connection lost on #3 attempt, let's retry after 10s, Failed to download data from builder due to rsync error, see the rsync log file for details.
[2025-09-08 13:56:52,575][ ERROR][PID:3502001] Re-try request for task on 'ResallocHost, ticket_id=4963704, hostname=35.170.76.120, name=aws_x86_64_spot_prod_06668588_20250908_133746, requested_tags=['copr_builder', 'arch_x86_64']': Unable to finish after 3 SSH attempts
[2025-09-08 13:56:53,198][  INFO][PID:3502001] Releasing VM back to pool
[2025-09-08 13:56:53,208][  INFO][PID:3502001] Retry #2 (on other host)
[2025-09-08 13:56:53,208][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:56:53,209][  INFO][PID:3502001] VM allocation process starts
[2025-09-08 13:56:53,222][  INFO][PID:3502001] Trying to allocate VM: ResallocHost, ticket_id=4963785, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-08 13:56:58,223][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:57:08,422][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:57:13,423][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:57:20,086][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:57:25,087][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:58:09,398][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:58:26,670][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:58:46,647][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:58:51,648][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:58:59,142][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:59:11,214][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 13:59:22,246][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:00:01,150][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:00:06,151][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:00:27,998][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:00:50,886][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:00:55,887][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:01:03,174][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:01:11,321][  INFO][PID:3502001] Allocated host ResallocHost, ticket_id=4963785, hostname=3.92.18.17, name=aws_x86_64_normalreserved_prod_06668701_20250908_135846, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-08 14:01:11,322][  INFO][PID:3502001] Allocating ssh connection to builder
[2025-09-08 14:01:11,322][  INFO][PID:3502001] Checking that builder machine is OK
[2025-09-08 14:01:11,829][  INFO][PID:3502001] Running remote command: copr-builder-ready rhel-9-x86_64
[2025-09-08 14:01:12,038][  INFO][PID:3502001] Checking Red Hat subscription (attempt #1)
Red Hat subscription active
Builder is ready to be used

[2025-09-08 14:01:12,038][  INFO][PID:3502001] Filling build.info file with builder info
[2025-09-08 14:01:12,041][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:01:12,042][  INFO][PID:3502001] Not re-notifying FE and msg buses for the new host.
[2025-09-08 14:01:12,042][  INFO][PID:3502001] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9533981-rhel-9-x86_64 --chroot rhel-9-x86_64 --detached
[2025-09-08 14:01:12,381][  INFO][PID:3502001] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '3.92.18.17' (ED25519) to the list of known hosts.


[2025-09-08 14:01:12,381][  INFO][PID:3502001] Downloading the builder-live.log file, attempt 1
[2025-09-08 14:01:12,383][  INFO][PID:3502001] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@3.92.18.17 copr-rpmbuild-log
[2025-09-08 14:01:17,392][  INFO][PID:3502001] Periodic builder liveness probe: alive
[2025-09-08 14:01:17,392][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:01:22,393][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:01:27,395][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:01:39,254][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:01:44,255][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:01:49,256][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:02:14,222][  INFO][PID:3502001] Checking for cancel request
[2025-09-08 14:02:16,945][  INFO][PID:3502001] Downloading results from builder
[2025-09-08 14:02:16,945][  INFO][PID:3502001] rsyncing of mockbuilder@3.92.18.17:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc started
[2025-09-08 14:02:16,946][  INFO][PID:3502001] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@3.92.18.17:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/ &> /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/build-09533981.rsync.log
[2025-09-08 14:45:09,660][  INFO][PID:3502001] rsyncing finished.
[2025-09-08 14:45:09,660][  INFO][PID:3502001] Releasing VM back to pool
[2025-09-08 14:45:09,871][  INFO][PID:3502001] Searching for 'success' file in resultdir
[2025-09-08 14:45:09,872][ ERROR][PID:3502001] Build failed: Backend process error: No success file => build failure
[2025-09-08 14:45:09,872][  INFO][PID:3502001] Finished build: id=9533981 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/rhcontainerbot/bootc chroot=rhel-9-x86_64 
[2025-09-08 14:45:09,874][ ERROR][PID:3502001] Backend process error: No success file => build failure
[2025-09-08 14:45:09,875][  INFO][PID:3502001] Worker failed build, took 2637.8326222896576
[2025-09-08 14:45:09,875][  INFO][PID:3502001] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "rhcontainerbot",
            "project_name": "bootc",
            "project_dirname": "bootc",
            "submitter": "packit",
            "ended_on": 1757342709.8749619,
            "started_on": 1757340072.0423396,
            "submitted_on": null,
            "status": 0,
            "chroot": "rhel-9-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9533981-rhel-9-x86_64",
            "build_id": 9533981,
            "package_name": "bootc",
            "package_version": "202509081112.g2d5b977af4-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/bootc/bootc",
            "git_hash": "204d99ee4949f9007e4a1109bfa7a3daad168d66",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rhcontainerbot/bootc--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/rhel-9-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/bootc",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc",
            "result_dir": "09533981-bootc",
            "built_packages": "",
            "id": 9533981,
            "mockchain_macros": {
                "copr_username": "rhcontainerbot",
                "copr_projectname": "bootc",
                "vendor": "Fedora Project COPR (rhcontainerbot/bootc)"
            }
        }
    ]
}
[2025-09-08 14:45:09,981][  INFO][PID:3502001] Sending fedora-messaging bus message in build.end
[2025-09-08 14:45:10,205][  INFO][PID:3502001] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/builder-live.log by gzip
[2025-09-08 14:45:10,206][  INFO][PID:3502001] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/builder-live.log' as PID 3836977
[2025-09-08 14:45:10,213][  INFO][PID:3502001] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/builder-live.log)
[2025-09-08 14:45:10,214][  INFO][PID:3502001] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/backend.log by gzip
[2025-09-08 14:45:10,215][  INFO][PID:3502001] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-x86_64/09533981-bootc/backend.log' as PID 3836982