[2026-02-27 13:06:46,790][  INFO][PID:2692551] Marking build as starting
[2026-02-27 13:06:46,884][  INFO][PID:2692551] Checking that /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/repodata/repomd.xml exists
[2026-02-27 13:06:47,080][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:06:47,082][  INFO][PID:2692551] VM allocation process starts
[2026-02-27 13:06:47,100][  INFO][PID:2692551] Trying to allocate VM: ResallocHost, ticket_id=1635873, requested_tags=['copr_builder', 'arch_s390x']
[2026-02-27 13:06:50,119][  INFO][PID:2692551] Allocated host ResallocHost, ticket_id=1635873, hostname=13.120.86.2, name=copr_ic_s390x_eu_es_1_prod_02695393_20260227_125105, requested_tags=['copr_builder', 'arch_s390x']
[2026-02-27 13:06:50,119][  INFO][PID:2692551] Allocating ssh connection to builder
[2026-02-27 13:06:50,120][  INFO][PID:2692551] Checking that builder machine is OK
[2026-02-27 13:06:51,544][  INFO][PID:2692551] Running remote command: copr-builder-ready fedora-rawhide-s390x
[2026-02-27 13:06:52,858][  INFO][PID:2692551] Red Hat subscription not needed for fedora-rawhide-s390x
Builder is ready to be used

[2026-02-27 13:06:52,858][  INFO][PID:2692551] Filling build.info file with builder info
[2026-02-27 13:06:52,859][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:06:52,860][  INFO][PID:2692551] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "jmarcoshp",
            "project_name": "mesa-custom",
            "project_dirname": "mesa-custom",
            "submitter": "https://api.github.com/users/github-actions%5Bbot%5D",
            "ended_on": null,
            "started_on": 1772197612.8601701,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-rawhide-s390x",
            "arch": "s390x",
            "buildroot_pkgs": [],
            "task_id": "10175424-fedora-rawhide-s390x",
            "build_id": 10175424,
            "package_name": "mesa",
            "package_version": "26.0.1-78.git14861bc",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/jmarcoshp/mesa-custom/mesa",
            "git_hash": "1d7c59e51d8f46f70f6fd25c691a54cb1d70ddbf",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "jmarcoshp/mesa-custom--https://api.github.com/users/github-actions%5Bbot%5D",
            "results": null,
            "appstream": true,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": 1
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": 1,
            "tags": [
                "arch_s390x"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/jmarcoshp/mesa-custom",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/jmarcoshp/mesa-custom",
            "result_dir": "10175424-mesa",
            "built_packages": "",
            "id": 10175424,
            "mockchain_macros": {
                "copr_username": "jmarcoshp",
                "copr_projectname": "mesa-custom",
                "vendor": "Fedora Project COPR (jmarcoshp/mesa-custom)"
            }
        }
    ]
}
[2026-02-27 13:06:52,914][  INFO][PID:2692551] Sending fedora-messaging bus message in build.start
[2026-02-27 13:06:53,472][  INFO][PID:2692551] Sending fedora-messaging bus message in chroot.start
[2026-02-27 13:06:53,501][  INFO][PID:2692551] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/10175424-fedora-rawhide-s390x --chroot fedora-rawhide-s390x --detached
[2026-02-27 13:06:54,913][  INFO][PID:2692551] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '13.120.86.2' (ED25519) to the list of known hosts.


[2026-02-27 13:06:54,914][  INFO][PID:2692551] Downloading the builder-live.log file, attempt 1
[2026-02-27 13:06:54,916][  INFO][PID:2692551] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@13.120.86.2 copr-rpmbuild-log
[2026-02-27 13:06:59,919][  INFO][PID:2692551] Periodic builder liveness probe: alive
[2026-02-27 13:06:59,938][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:04,939][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:09,940][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:14,943][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:19,944][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:24,945][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:29,946][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:34,947][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:39,948][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:44,949][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:49,950][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:54,952][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:07:59,953][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:04,954][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:09,955][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:14,956][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:19,957][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:24,958][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:29,959][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:34,960][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:39,961][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:44,962][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:49,963][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:54,964][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:08:59,965][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:04,966][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:09,967][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:14,968][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:19,969][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:24,970][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:29,971][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:34,972][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:39,973][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:44,975][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:49,975][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:54,976][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:09:59,977][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:04,978][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:09,982][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:14,983][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:19,985][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:24,988][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:29,989][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:34,990][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:39,991][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:44,992][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:49,994][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:54,995][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:10:59,996][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:04,997][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:09,998][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:14,999][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:20,000][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:25,002][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:30,004][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:35,005][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:40,006][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:45,007][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:50,008][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:11:55,009][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:00,011][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:05,012][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:10,015][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:15,025][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:20,026][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:25,027][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:30,028][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:35,029][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:40,030][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:45,031][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:50,032][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:12:55,033][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:00,033][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:05,034][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:10,035][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:15,064][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:20,065][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:25,066][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:30,067][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:35,068][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:40,069][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:45,070][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:50,071][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:13:55,072][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:00,073][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:05,075][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:10,076][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:15,077][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:20,078][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:25,079][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:30,080][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:35,081][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:40,082][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:45,083][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:50,084][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:14:55,085][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:00,087][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:05,089][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:10,089][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:15,090][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:20,091][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:25,093][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:30,094][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:35,095][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:40,097][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:45,099][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:50,100][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:15:55,101][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:00,102][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:05,103][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:10,105][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:15,109][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:20,110][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:25,111][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:30,112][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:35,113][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:40,114][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:45,131][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:50,132][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:16:55,133][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:00,139][  INFO][PID:2692551] Periodic builder liveness probe: alive
[2026-02-27 13:17:00,140][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:05,141][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:10,142][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:15,144][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:20,145][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:25,147][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:30,148][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:35,150][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:40,151][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:45,154][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:50,157][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:17:55,157][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:18:00,158][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:18:05,159][  INFO][PID:2692551] Checking for cancel request
[2026-02-27 13:18:06,839][  INFO][PID:2692551] Downloading results from builder
[2026-02-27 13:18:06,840][  INFO][PID:2692551] rsyncing of mockbuilder@13.120.86.2:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/10175424-mesa started
[2026-02-27 13:18:06,840][  INFO][PID:2692551] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@13.120.86.2:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/10175424-mesa/ &> /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/10175424-mesa/build-10175424.rsync.log
[2026-02-27 13:18:08,545][  INFO][PID:2692551] rsyncing finished.
[2026-02-27 13:18:08,546][  INFO][PID:2692551] Releasing VM back to pool
[2026-02-27 13:18:08,558][  INFO][PID:2692551] Searching for 'success' file in resultdir
[2026-02-27 13:18:08,559][ ERROR][PID:2692551] Build failed: Backend process error: No success file => build failure
[2026-02-27 13:18:08,559][  INFO][PID:2692551] Finished build: id=10175424 failed=True timeout=18000 destdir=/var/lib/copr/public_html/results/jmarcoshp/mesa-custom chroot=fedora-rawhide-s390x 
[2026-02-27 13:18:08,561][ ERROR][PID:2692551] Backend process error: No success file => build failure
[2026-02-27 13:18:08,561][  INFO][PID:2692551] Worker failed build, took 675.7015171051025
[2026-02-27 13:18:08,562][  INFO][PID:2692551] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "jmarcoshp",
            "project_name": "mesa-custom",
            "project_dirname": "mesa-custom",
            "submitter": "https://api.github.com/users/github-actions%5Bbot%5D",
            "ended_on": 1772198288.5616872,
            "started_on": 1772197612.8601701,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-rawhide-s390x",
            "arch": "s390x",
            "buildroot_pkgs": [],
            "task_id": "10175424-fedora-rawhide-s390x",
            "build_id": 10175424,
            "package_name": "mesa",
            "package_version": "26.0.1-78.git14861bc",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/jmarcoshp/mesa-custom/mesa",
            "git_hash": "1d7c59e51d8f46f70f6fd25c691a54cb1d70ddbf",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "jmarcoshp/mesa-custom--https://api.github.com/users/github-actions%5Bbot%5D",
            "results": null,
            "appstream": true,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": 1
                }
            ],
            "background": true,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": 1,
            "tags": [
                "arch_s390x"
            ],
            "with_opts": [],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/jmarcoshp/mesa-custom",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/jmarcoshp/mesa-custom",
            "result_dir": "10175424-mesa",
            "built_packages": "",
            "id": 10175424,
            "mockchain_macros": {
                "copr_username": "jmarcoshp",
                "copr_projectname": "mesa-custom",
                "vendor": "Fedora Project COPR (jmarcoshp/mesa-custom)"
            }
        }
    ]
}
[2026-02-27 13:18:08,619][  INFO][PID:2692551] Sending fedora-messaging bus message in build.end
[2026-02-27 13:18:08,649][  INFO][PID:2692551] Compressing /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/10175424-mesa/builder-live.log by gzip
[2026-02-27 13:18:08,650][  INFO][PID:2692551] Running command 'gzip /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/10175424-mesa/builder-live.log' as PID 2762565
[2026-02-27 13:18:08,651][  INFO][PID:2692551] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/10175424-mesa/builder-live.log)
[2026-02-27 13:18:08,652][  INFO][PID:2692551] Compressing /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/10175424-mesa/backend.log by gzip
[2026-02-27 13:18:08,652][  INFO][PID:2692551] Running command 'gzip /var/lib/copr/public_html/results/jmarcoshp/mesa-custom/fedora-rawhide-s390x/10175424-mesa/backend.log' as PID 2762566