[2025-10-21 15:33:15,940][  INFO][PID:3737976] Marking build as starting
[2025-10-21 15:33:16,026][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:33:16,027][  INFO][PID:3737976] VM allocation process starts
[2025-10-21 15:33:16,038][  INFO][PID:3737976] Trying to allocate VM: ResallocHost, ticket_id=5768293, requested_tags=['copr_builder', 'arch_s390x']
[2025-10-21 15:33:21,040][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:33:26,041][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:33:31,042][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:33:36,043][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:33:41,044][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:33:46,045][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:33:51,046][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:33:56,047][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:01,049][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:06,050][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:11,051][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:16,052][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:21,056][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:26,057][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:31,058][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:36,059][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:41,060][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:46,063][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:51,063][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:34:56,064][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:01,065][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:06,066][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:11,067][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:16,068][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:21,069][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:26,070][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:31,071][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:36,072][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:41,073][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:46,074][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:51,075][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:35:56,076][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:01,077][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:06,078][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:11,079][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:16,080][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:21,081][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:26,082][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:31,083][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:36,084][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:41,085][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:46,086][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:51,087][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:36:56,088][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:01,089][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:06,090][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:11,090][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:16,091][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:21,092][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:26,093][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:31,094][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:36,095][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:41,096][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:46,097][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:51,098][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:37:56,099][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:38:01,101][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:38:06,101][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:38:11,102][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:38:16,103][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:38:21,104][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:38:26,105][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:38:31,106][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:38:34,140][  INFO][PID:3737976] Allocated host ResallocHost, ticket_id=5768293, hostname=13.120.89.142, name=copr_ic_s390x_eu_es_1_prod_07576340_20251021_153522, requested_tags=['copr_builder', 'arch_s390x']
[2025-10-21 15:38:34,140][  INFO][PID:3737976] Allocating ssh connection to builder
[2025-10-21 15:38:34,141][  INFO][PID:3737976] Checking that builder machine is OK
[2025-10-21 15:38:35,545][  INFO][PID:3737976] Running remote command: copr-builder-ready rhel-9-s390x
[2025-10-21 15:48:36,858][  INFO][PID:3737976] Checking Red Hat subscription (attempt #1)
Checking Red Hat subscription (attempt #2)
Checking Red Hat subscription (attempt #3)
Checking Red Hat subscription (attempt #4)
Checking Red Hat subscription (attempt #5)
Checking Red Hat subscription (attempt #6)
Checking Red Hat subscription (attempt #7)
Checking Red Hat subscription (attempt #8)
Checking Red Hat subscription (attempt #9)
Checking Red Hat subscription (attempt #10)
Checking Red Hat subscription (attempt #11)
Checking Red Hat subscription (attempt #12)
Checking Red Hat subscription (attempt #13)
Checking Red Hat subscription (attempt #14)
Checking Red Hat subscription (attempt #15)
Checking Red Hat subscription (attempt #16)
Checking Red Hat subscription (attempt #17)
Checking Red Hat subscription (attempt #18)
Checking Red Hat subscription (attempt #19)
Checking Red Hat subscription (attempt #20)
Checking Red Hat subscription (attempt #21)
Waiting for Red Hat subscription timeouted!

[2025-10-21 15:48:36,859][  INFO][PID:3737976] Warning: Permanently added '13.120.89.142' (ED25519) to the list of known hosts.

[2025-10-21 15:48:36,859][ ERROR][PID:3737976] Re-try request for task on 'ResallocHost, ticket_id=5768293, hostname=13.120.89.142, name=copr_ic_s390x_eu_es_1_prod_07576340_20251021_153522, requested_tags=['copr_builder', 'arch_s390x']': Builder wasn't ready, trying a new one
[2025-10-21 15:48:36,860][  INFO][PID:3737976] Releasing VM back to pool
[2025-10-21 15:48:36,871][  INFO][PID:3737976] Retry #1 (on other host)
[2025-10-21 15:48:36,872][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:48:36,873][  INFO][PID:3737976] VM allocation process starts
[2025-10-21 15:48:36,884][  INFO][PID:3737976] Trying to allocate VM: ResallocHost, ticket_id=5768526, requested_tags=['copr_builder', 'arch_s390x']
[2025-10-21 15:48:41,885][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:48:46,887][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:48:51,888][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:48:56,889][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:01,890][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:06,891][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:11,892][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:16,893][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:21,894][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:26,895][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:31,896][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:36,897][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:41,898][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:46,899][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:51,900][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:49:56,901][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:01,902][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:06,903][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:11,904][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:16,905][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:21,907][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:26,908][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:31,909][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:36,910][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:41,910][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:46,911][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:51,912][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:50:56,913][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:01,914][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:06,915][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:11,916][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:16,917][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:21,918][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:26,919][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:31,920][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:36,921][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:41,922][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:46,923][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:51,924][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:51:56,925][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:01,926][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:06,927][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:11,928][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:16,935][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:21,936][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:26,937][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:31,938][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:36,939][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:41,940][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:46,941][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:51,941][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:52:56,942][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:01,943][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:06,944][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:11,945][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:16,946][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:21,952][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:26,953][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:31,954][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:36,955][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:41,956][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:46,957][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:51,958][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:53:55,083][  INFO][PID:3737976] Allocated host ResallocHost, ticket_id=5768526, hostname=13.120.89.142, name=copr_ic_s390x_eu_es_1_prod_07576562_20251021_155021, requested_tags=['copr_builder', 'arch_s390x']
[2025-10-21 15:53:55,084][  INFO][PID:3737976] Allocating ssh connection to builder
[2025-10-21 15:53:55,084][  INFO][PID:3737976] Checking that builder machine is OK
[2025-10-21 15:53:56,499][  INFO][PID:3737976] Running remote command: copr-builder-ready rhel-9-s390x
[2025-10-21 15:54:27,812][  INFO][PID:3737976] Checking Red Hat subscription (attempt #1)
Checking Red Hat subscription (attempt #2)
Red Hat subscription active
Builder is ready to be used

[2025-10-21 15:54:27,813][  INFO][PID:3737976] Filling build.info file with builder info
[2025-10-21 15:54:27,814][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:54:27,815][  INFO][PID:3737976] Not re-notifying FE and msg buses for the new host.
[2025-10-21 15:54:27,815][  INFO][PID:3737976] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9711620-rhel-9-s390x --chroot rhel-9-s390x --detached
[2025-10-21 15:54:29,202][  INFO][PID:3737976] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '13.120.89.142' (ED25519) to the list of known hosts.


[2025-10-21 15:54:29,203][  INFO][PID:3737976] Downloading the builder-live.log file, attempt 1
[2025-10-21 15:54:29,204][  INFO][PID:3737976] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@13.120.89.142 copr-rpmbuild-log
[2025-10-21 15:54:34,210][  INFO][PID:3737976] Periodic builder liveness probe: alive
[2025-10-21 15:54:34,211][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:54:39,211][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:54:44,212][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:54:49,214][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:54:54,215][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:54:59,216][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:04,218][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:09,219][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:14,220][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:19,221][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:24,222][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:29,228][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:34,232][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:39,233][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:44,235][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:49,236][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:54,237][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:55:59,238][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:56:04,239][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:56:09,240][  INFO][PID:3737976] Checking for cancel request
[2025-10-21 15:56:10,152][  INFO][PID:3737976] Downloading results from builder
[2025-10-21 15:56:10,153][  INFO][PID:3737976] rsyncing of mockbuilder@13.120.89.142:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-s390x/09711620-bootc started
[2025-10-21 15:56:10,154][  INFO][PID:3737976] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@13.120.89.142:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-s390x/09711620-bootc/ &> /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-s390x/09711620-bootc/build-09711620.rsync.log
[2025-10-21 15:56:13,850][  INFO][PID:3737976] rsyncing finished.
[2025-10-21 15:56:13,852][  INFO][PID:3737976] Releasing VM back to pool
[2025-10-21 15:56:13,912][  INFO][PID:3737976] Searching for 'success' file in resultdir
[2025-10-21 15:56:13,913][ ERROR][PID:3737976] Build failed: Backend process error: No success file => build failure
[2025-10-21 15:56:13,920][  INFO][PID:3737976] Finished build: id=9711620 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/rhcontainerbot/bootc chroot=rhel-9-s390x 
[2025-10-21 15:56:13,932][ ERROR][PID:3737976] Backend process error: No success file => build failure
[2025-10-21 15:56:13,932][  INFO][PID:3737976] Worker failed build, took 106.11771416664124
[2025-10-21 15:56:13,933][  INFO][PID:3737976] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "rhcontainerbot",
            "project_name": "bootc",
            "project_dirname": "bootc",
            "submitter": "packit",
            "ended_on": 1761062173.932767,
            "started_on": 1761062067.8150527,
            "submitted_on": null,
            "status": 0,
            "chroot": "rhel-9-s390x",
            "arch": "s390x",
            "buildroot_pkgs": [],
            "task_id": "9711620-rhel-9-s390x",
            "build_id": 9711620,
            "package_name": "bootc",
            "package_version": "202510211520.g817d7d11f9-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rhcontainerbot/bootc/bootc",
            "git_hash": "55951a9016a5c8a18c4ef383a5f7fb156db6b79c",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rhcontainerbot/bootc--packit",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc/rhel-9-s390x/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_s390x"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rhcontainerbot/bootc",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rhcontainerbot/bootc",
            "result_dir": "09711620-bootc",
            "built_packages": "",
            "id": 9711620,
            "mockchain_macros": {
                "copr_username": "rhcontainerbot",
                "copr_projectname": "bootc",
                "vendor": "Fedora Project COPR (rhcontainerbot/bootc)"
            }
        }
    ]
}
[2025-10-21 15:56:14,041][  INFO][PID:3737976] Sending fedora-messaging bus message in build.end
[2025-10-21 15:56:15,060][  INFO][PID:3737976] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-s390x/09711620-bootc/builder-live.log by gzip
[2025-10-21 15:56:15,062][  INFO][PID:3737976] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-s390x/09711620-bootc/builder-live.log' as PID 3884418
[2025-10-21 15:56:15,069][  INFO][PID:3737976] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-s390x/09711620-bootc/builder-live.log)
[2025-10-21 15:56:15,069][  INFO][PID:3737976] Compressing /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-s390x/09711620-bootc/backend.log by gzip
[2025-10-21 15:56:15,072][  INFO][PID:3737976] Running command 'gzip /var/lib/copr/public_html/results/rhcontainerbot/bootc/rhel-9-s390x/09711620-bootc/backend.log' as PID 3884421