[2025-09-09 19:04:33,985][  INFO][PID:528809] Marking build as starting
[2025-09-09 19:04:34,007][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:04:34,008][  INFO][PID:528809] VM allocation process starts
[2025-09-09 19:04:34,022][  INFO][PID:528809] Trying to allocate VM: ResallocHost, ticket_id=4986716, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-09 19:04:39,024][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:04:44,025][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:04:49,026][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:04:54,027][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:04:59,028][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:04,029][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:09,030][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:14,031][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:19,032][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:24,033][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:29,034][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:34,035][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:39,036][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:44,037][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:49,038][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:54,079][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:05:59,083][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:02,108][  INFO][PID:528809] Allocated host ResallocHost, ticket_id=4986716, hostname=54.86.137.111, name=aws_x86_64_spot_prod_06685924_20250909_190427, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-09 19:06:02,109][  INFO][PID:528809] Allocating ssh connection to builder
[2025-09-09 19:06:02,109][  INFO][PID:528809] Checking that builder machine is OK
[2025-09-09 19:06:02,575][  INFO][PID:528809] Running remote command: copr-builder-ready srpm-builds
[2025-09-09 19:06:02,779][  INFO][PID:528809] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2025-09-09 19:06:02,780][  INFO][PID:528809] Filling build.info file with builder info
[2025-09-09 19:06:02,781][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:02,782][  INFO][PID:528809] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "lizardbyte",
            "project_name": "pulls",
            "project_dirname": "pulls:pr:4245",
            "submitter": null,
            "ended_on": null,
            "started_on": 1757444762.781785,
            "submitted_on": null,
            "status": 3,
            "chroot": "srpm-builds",
            "arch": "x86_64",
            "buildroot_pkgs": null,
            "task_id": "9541185",
            "build_id": 9541185,
            "package_name": "Sunshine",
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 9,
            "source_json": "{\"script\": \"#!/usr/bin/env bash\\nset -x\\nset -e\\n\\nresultdir=\\\"${COPR_RESULTDIR}\\\"\\ngit clone \\\"https://github.com/${COPR_OWNER}/${COPR_PACKAGE}.git\\\" --depth 1\\ncd \\\"${COPR_PACKAGE}\\\"\\n\\n# get info from the webhook payload\\nif test -z \\\"$REVISION\\\"; then\\n  # the hook_payload file contains webhook JSON payload (copr creates it for us);\\n  # it is created only if the build is triggered by Custom webhook.\\n  if test -f \\\"$resultdir\\\"/hook_payload; then\\n    git clone https://github.com/praiskup/copr-ci-tooling \\\\\\n      \\\"$resultdir/cct\\\" --depth 1\\n    export PATH=\\\"$resultdir/cct:$PATH\\\"\\n\\n    echo \\\"---\\\"\\n    cat \\\"$resultdir\\\"/hook_payload\\n    echo \\\"---\\\"\\n\\n    # use jq to get the pr_id from the hook_payload\\n    PR=$(jq -r '.pr_id // empty' \\\"$resultdir\\\"/hook_payload)\\n    if [ -z \\\"$PR\\\" ]; then\\n      BRANCH=\\\"master\\\"\\n    else\\n      BRANCH=\\\"pr/${PR}\\\"\\n    fi\\n\\n    copr-travis-checkout \\\"$resultdir\\\"/hook_payload\\n  fi\\nelse\\n  git checkout \\\"$REVISION\\\"\\nfi\\n\\n# initialize the submodules\\ngit submodule update --init --recursive\\n\\n# get the tag of this commit IF it has one\\nTAG=$(git tag --points-at HEAD | head -n1)\\nif [ -z \\\"$TAG\\\" ]; then\\n  TAG=\\\"0.0.$PR\\\"\\nfi\\nTAG=\\\"${TAG#v}\\\"  # remove v prefix from the tag\\necho \\\"TAG=$TAG\\\"\\n\\n# get the commit\\nCOMMIT=$(git rev-parse HEAD)\\necho \\\"COMMIT=$COMMIT\\\"\\n\\n# move spec file to the correct location\\ndirectories=(\\n  \\\".\\\"\\n  \\\"./packaging/linux/fedora\\\"\\n)\\nfor dir in \\\"${directories[@]}\\\"; do\\n  if [ -f \\\"${dir}/${COPR_PACKAGE}.spec\\\" ]; then\\n    echo \\\"Found spec file in ${dir}\\\"\\n    rpmlint \\\"${dir}/${COPR_PACKAGE}.spec\\\"\\n\\n    mv \\\"${dir}/${COPR_PACKAGE}.spec\\\" \\\"${resultdir}\\\"\\n    break\\n  fi\\ndone\\n\\n# fail if the spec file is not in the resultdir\\nif [ ! -f \\\"${resultdir}/${COPR_PACKAGE}.spec\\\" ]; then\\n  echo \\\"ERROR: ${COPR_PACKAGE}.spec not found\\\"\\n  exit 1\\nfi\\n\\n# use sed to replace these values in the spec file\\nsed -i \\\"s|%global build_version 0|%global build_version ${TAG}|\\\" \\\"${resultdir}\\\"/*.spec\\nsed -i \\\"s|%global branch 0|%global branch ${BRANCH}|\\\" \\\"${resultdir}\\\"/*.spec\\nsed -i \\\"s|%global commit 0|%global commit ${COMMIT}|\\\" \\\"${resultdir}\\\"/*.spec\\n\\n# create a tarball of the source code\\ntar -czf \\\"${resultdir}/tarball.tar.gz\\\" .\\n\", \"chroot\": \"fedora-43-x86_64\", \"builddeps\": \"git jq python3 rpmlint\", \"resultdir\": \"\", \"repos\": \"\", \"tmp\": \"tmpyxupqjyk\", \"hook_data\": true}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "lizardbyte/pulls--dc934869-db9c-43d0-a647-496d25134409",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": true,
            "destdir": "/var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/lizardbyte/pulls:pr:4245",
            "result_dir": "09541185",
            "built_packages": "",
            "tags": [
                "arch_x86_64"
            ],
            "id": 9541185,
            "mockchain_macros": {
                "copr_username": "lizardbyte",
                "copr_projectname": "pulls",
                "vendor": "Fedora Project COPR (lizardbyte/pulls)"
            }
        }
    ]
}
[2025-09-09 19:06:02,810][  INFO][PID:528809] Sending fedora-messaging bus message in build.start
[2025-09-09 19:06:03,487][  INFO][PID:528809] Sending fedora-messaging bus message in chroot.start
[2025-09-09 19:06:03,513][  INFO][PID:528809] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9541185 --detached
[2025-09-09 19:06:03,834][  INFO][PID:528809] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '54.86.137.111' (ED25519) to the list of known hosts.


[2025-09-09 19:06:03,835][  INFO][PID:528809] Downloading the builder-live.log file, attempt 1
[2025-09-09 19:06:03,837][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.137.111 copr-rpmbuild-log
[2025-09-09 19:06:08,841][  INFO][PID:528809] Periodic builder liveness probe: alive
[2025-09-09 19:06:08,841][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:13,842][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:18,843][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:23,844][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:28,845][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:33,846][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:38,847][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:43,848][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:48,849][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:53,850][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:06:58,851][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:03,852][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:08,853][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:13,854][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:18,855][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:23,856][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:28,857][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:33,858][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:38,859][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:43,860][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:48,861][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:53,862][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:07:58,863][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:03,864][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:08,865][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:13,866][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:18,867][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:23,868][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:28,869][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:33,870][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:38,871][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:43,872][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:48,574][ ERROR][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 19:08:48,874][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:53,875][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:08:58,575][WARNING][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 19:08:58,876][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:03,877][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:08,576][  INFO][PID:528809] Downloading the builder-live.log file, attempt 2
[2025-09-09 19:09:08,577][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.137.111 copr-rpmbuild-log
[2025-09-09 19:09:08,878][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:13,885][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:18,886][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:23,887][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:28,888][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:33,889][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:38,890][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:43,891][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:48,892][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:53,894][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:09:58,894][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:03,895][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:08,646][ ERROR][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 19:10:08,896][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:13,898][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:18,651][WARNING][PID:528809] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 19:10:18,899][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:23,900][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:28,651][  INFO][PID:528809] Downloading the builder-live.log file, attempt 3
[2025-09-09 19:10:28,653][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.137.111 copr-rpmbuild-log
[2025-09-09 19:10:28,901][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:33,902][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:39,111][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:44,112][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:49,113][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:10:54,114][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:11:00,982][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:11:05,983][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:11:10,984][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:11:15,985][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:11:20,988][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:11:25,989][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:11:28,722][ ERROR][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 19:11:30,990][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:11:38,723][WARNING][PID:528809] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 19:14:12,406][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:14:17,406][  INFO][PID:528809] Downloading the builder-live.log file, attempt 4
[2025-09-09 19:14:17,407][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:14:17,408][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.137.111 copr-rpmbuild-log
[2025-09-09 19:14:22,408][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:14:27,409][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:14:32,410][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:14:37,412][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:14:42,413][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:14:52,614][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:14:57,615][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:15:04,278][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:15:09,279][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:15:14,280][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:15:17,475][ ERROR][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 19:15:19,281][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:15:27,190][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:15:27,477][WARNING][PID:528809] SSH connection lost on #4 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 19:17:31,593][  INFO][PID:528809] Periodic builder liveness probe: alive
[2025-09-09 19:17:31,593][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:17:36,582][  INFO][PID:528809] Downloading the builder-live.log file, attempt 5
[2025-09-09 19:17:36,583][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.137.111 copr-rpmbuild-log
[2025-09-09 19:17:36,594][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:17:41,790][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:17:46,791][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:17:51,793][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:17:56,794][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:18:01,795][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:18:36,651][ ERROR][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 19:19:04,198][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:09,199][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:09,200][WARNING][PID:528809] SSH connection lost on #5 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 19:19:14,200][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:19,200][  INFO][PID:528809] Downloading the builder-live.log file, attempt 6
[2025-09-09 19:19:19,201][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:19,211][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.137.111 copr-rpmbuild-log
[2025-09-09 19:19:24,203][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:29,204][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:34,205][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:39,206][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:44,207][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:49,208][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:54,209][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:19:59,210][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:04,211][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:19,280][ ERROR][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 19:20:20,854][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:25,855][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:29,286][WARNING][PID:528809] SSH connection lost on #6 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 19:20:30,856][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:35,857][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:39,287][  INFO][PID:528809] Downloading the builder-live.log file, attempt 7
[2025-09-09 19:20:39,288][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.137.111 copr-rpmbuild-log
[2025-09-09 19:20:40,858][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:45,859][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:50,860][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:20:55,861][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:00,862][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:05,863][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:10,864][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:15,865][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:21,791][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:39,062][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:39,356][ ERROR][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 19:21:44,064][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:49,065][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:49,358][WARNING][PID:528809] SSH connection lost on #7 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-09-09 19:21:54,066][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:59,068][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:21:59,359][  INFO][PID:528809] Downloading the builder-live.log file, attempt 8
[2025-09-09 19:21:59,360][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@54.86.137.111 copr-rpmbuild-log
[2025-09-09 19:22:04,069][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:09,070][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:26,343][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:31,345][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:36,347][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:41,348][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:46,349][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:51,350][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:56,351][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:22:59,429][ ERROR][PID:528809] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-09-09 19:23:07,801][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:23:12,801][ ERROR][PID:528809] Giving up for unstable SSH, failures: 1757444938.5753155, 1757445018.651068, 1757445098.7238398, 1757445327.4773211, 1757445549.1999128, 1757445629.2866123, 1757445709.3583207, 1757445792.8018348
[2025-09-09 19:23:12,803][ ERROR][PID:528809] Re-try request for task on 'ResallocHost, ticket_id=4986716, hostname=54.86.137.111, name=aws_x86_64_spot_prod_06685924_20250909_190427, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-09-09 19:23:12,803][  INFO][PID:528809] Releasing VM back to pool
[2025-09-09 19:23:12,814][  INFO][PID:528809] Retry #1 (on other host)
[2025-09-09 19:23:12,815][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:23:12,816][  INFO][PID:528809] VM allocation process starts
[2025-09-09 19:23:12,861][  INFO][PID:528809] Trying to allocate VM: ResallocHost, ticket_id=4986888, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-09 19:23:17,863][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:23:22,864][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:23:27,865][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:23:32,866][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:23:37,867][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:09,695][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:14,696][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:20,983][  INFO][PID:528809] Allocated host ResallocHost, ticket_id=4986888, hostname=2620:52:3:1:dead:beef:cafe:c109, name=copr_hv_x86_64_01_prod_06686120_20250909_192225, requested_tags=['copr_builder', 'arch_x86_64']
[2025-09-09 19:24:20,983][  INFO][PID:528809] Allocating ssh connection to builder
[2025-09-09 19:24:20,984][  INFO][PID:528809] Checking that builder machine is OK
[2025-09-09 19:24:21,595][  INFO][PID:528809] Running remote command: copr-builder-ready srpm-builds
[2025-09-09 19:24:21,913][  INFO][PID:528809] Red Hat subscription not needed for srpm-builds
Builder is ready to be used

[2025-09-09 19:24:21,914][  INFO][PID:528809] Filling build.info file with builder info
[2025-09-09 19:24:21,919][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:21,920][  INFO][PID:528809] Not re-notifying FE and msg buses for the new host.
[2025-09-09 19:24:21,921][  INFO][PID:528809] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --srpm --task-url https://copr.fedorainfracloud.org/backend/get-srpm-build-task/9541185 --detached
[2025-09-09 19:24:22,493][  INFO][PID:528809] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c109' (ED25519) to the list of known hosts.


[2025-09-09 19:24:22,494][  INFO][PID:528809] Downloading the builder-live.log file, attempt 1
[2025-09-09 19:24:22,495][  INFO][PID:528809] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c109 copr-rpmbuild-log
[2025-09-09 19:24:27,494][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:32,496][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:37,497][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:42,498][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:47,499][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:52,502][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:24:57,503][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:02,505][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:07,506][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:12,507][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:17,508][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:22,509][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:27,510][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:44,367][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:49,368][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:54,369][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:25:59,370][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:26:04,371][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:26:09,372][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:26:14,373][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:26:19,374][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:26:24,375][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:26:29,376][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:26:34,377][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:26:39,378][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:03,510][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:08,511][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:13,512][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:18,513][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:23,514][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:28,515][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:33,724][  INFO][PID:528809] Periodic builder liveness probe: alive
[2025-09-09 19:27:39,958][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:44,959][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:49,969][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:27:55,037][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:00,106][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:06,041][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:11,045][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:16,047][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:21,048][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:26,050][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:31,052][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:36,053][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:50,831][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:28:57,702][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:02,703][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:07,706][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:12,711][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:17,713][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:29,367][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:34,368][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:39,370][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:48,112][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:53,113][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:29:58,115][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:30:03,116][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:30:26,207][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:30:31,209][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:30:36,211][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:30:44,743][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:30:49,744][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:30:54,748][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:30:59,757][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:31:04,758][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:31:09,760][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:31:14,761][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:31:40,559][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:32:03,446][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:32:09,694][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:32:14,696][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:32:27,390][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:32:32,392][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:32:48,414][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:32:53,416][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:32:58,417][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:33:03,418][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:33:08,419][  INFO][PID:528809] Checking for cancel request
[2025-09-09 19:33:17,858][  INFO][PID:528809] Downloading results from builder
[2025-09-09 19:33:17,859][  INFO][PID:528809] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c109]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185 started
[2025-09-09 19:33:17,860][  INFO][PID:528809] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c109]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/ &> /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/build-09541185.rsync.log
[2025-09-10 04:03:53,623][  INFO][PID:528809] rsyncing finished.
[2025-09-10 04:03:53,871][  INFO][PID:528809] Releasing VM back to pool
[2025-09-10 04:03:54,168][  INFO][PID:528809] Searching for 'success' file in resultdir
[2025-09-10 04:03:54,251][  INFO][PID:528809] Getting build details
[2025-09-10 04:03:54,256][  INFO][PID:528809] Retrieving SRPM info from /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185
[2025-09-10 04:03:54,386][  INFO][PID:528809] SRPM URL: https://download.copr.fedorainfracloud.org/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/Sunshine-0.0.4245-1.src.rpm
[2025-09-10 04:03:54,389][  INFO][PID:528809] build details: {'srpm_url': 'https://download.copr.fedorainfracloud.org/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/Sunshine-0.0.4245-1.src.rpm', 'pkg_name': 'Sunshine', 'pkg_version': '0.0.4245-1'}
[2025-09-10 04:03:54,421][  INFO][PID:528809] Finished build: id=9541185 failed=False timeout=108000 destdir=/var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245 chroot=srpm-builds 
[2025-09-10 04:03:54,570][  INFO][PID:528809] Worker succeeded build, took 31172.64904475212
[2025-09-10 04:03:54,583][  INFO][PID:528809] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "lizardbyte",
            "project_name": "pulls",
            "project_dirname": "pulls:pr:4245",
            "submitter": null,
            "ended_on": 1757477034.5695877,
            "started_on": 1757445861.920543,
            "submitted_on": null,
            "status": 1,
            "chroot": "srpm-builds",
            "arch": "x86_64",
            "buildroot_pkgs": null,
            "task_id": "9541185",
            "build_id": 9541185,
            "package_name": "Sunshine",
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 9,
            "source_json": "{\"script\": \"#!/usr/bin/env bash\\nset -x\\nset -e\\n\\nresultdir=\\\"${COPR_RESULTDIR}\\\"\\ngit clone \\\"https://github.com/${COPR_OWNER}/${COPR_PACKAGE}.git\\\" --depth 1\\ncd \\\"${COPR_PACKAGE}\\\"\\n\\n# get info from the webhook payload\\nif test -z \\\"$REVISION\\\"; then\\n  # the hook_payload file contains webhook JSON payload (copr creates it for us);\\n  # it is created only if the build is triggered by Custom webhook.\\n  if test -f \\\"$resultdir\\\"/hook_payload; then\\n    git clone https://github.com/praiskup/copr-ci-tooling \\\\\\n      \\\"$resultdir/cct\\\" --depth 1\\n    export PATH=\\\"$resultdir/cct:$PATH\\\"\\n\\n    echo \\\"---\\\"\\n    cat \\\"$resultdir\\\"/hook_payload\\n    echo \\\"---\\\"\\n\\n    # use jq to get the pr_id from the hook_payload\\n    PR=$(jq -r '.pr_id // empty' \\\"$resultdir\\\"/hook_payload)\\n    if [ -z \\\"$PR\\\" ]; then\\n      BRANCH=\\\"master\\\"\\n    else\\n      BRANCH=\\\"pr/${PR}\\\"\\n    fi\\n\\n    copr-travis-checkout \\\"$resultdir\\\"/hook_payload\\n  fi\\nelse\\n  git checkout \\\"$REVISION\\\"\\nfi\\n\\n# initialize the submodules\\ngit submodule update --init --recursive\\n\\n# get the tag of this commit IF it has one\\nTAG=$(git tag --points-at HEAD | head -n1)\\nif [ -z \\\"$TAG\\\" ]; then\\n  TAG=\\\"0.0.$PR\\\"\\nfi\\nTAG=\\\"${TAG#v}\\\"  # remove v prefix from the tag\\necho \\\"TAG=$TAG\\\"\\n\\n# get the commit\\nCOMMIT=$(git rev-parse HEAD)\\necho \\\"COMMIT=$COMMIT\\\"\\n\\n# move spec file to the correct location\\ndirectories=(\\n  \\\".\\\"\\n  \\\"./packaging/linux/fedora\\\"\\n)\\nfor dir in \\\"${directories[@]}\\\"; do\\n  if [ -f \\\"${dir}/${COPR_PACKAGE}.spec\\\" ]; then\\n    echo \\\"Found spec file in ${dir}\\\"\\n    rpmlint \\\"${dir}/${COPR_PACKAGE}.spec\\\"\\n\\n    mv \\\"${dir}/${COPR_PACKAGE}.spec\\\" \\\"${resultdir}\\\"\\n    break\\n  fi\\ndone\\n\\n# fail if the spec file is not in the resultdir\\nif [ ! -f \\\"${resultdir}/${COPR_PACKAGE}.spec\\\" ]; then\\n  echo \\\"ERROR: ${COPR_PACKAGE}.spec not found\\\"\\n  exit 1\\nfi\\n\\n# use sed to replace these values in the spec file\\nsed -i \\\"s|%global build_version 0|%global build_version ${TAG}|\\\" \\\"${resultdir}\\\"/*.spec\\nsed -i \\\"s|%global branch 0|%global branch ${BRANCH}|\\\" \\\"${resultdir}\\\"/*.spec\\nsed -i \\\"s|%global commit 0|%global commit ${COMMIT}|\\\" \\\"${resultdir}\\\"/*.spec\\n\\n# create a tarball of the source code\\ntar -czf \\\"${resultdir}/tarball.tar.gz\\\" .\\n\", \"chroot\": \"fedora-43-x86_64\", \"builddeps\": \"git jq python3 rpmlint\", \"resultdir\": \"\", \"repos\": \"\", \"tmp\": \"tmpyxupqjyk\", \"hook_data\": true}",
            "pkg_name": "Sunshine",
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": "https://download.copr.fedorainfracloud.org/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/Sunshine-0.0.4245-1.src.rpm",
            "uses_devel_repo": null,
            "sandbox": "lizardbyte/pulls--dc934869-db9c-43d0-a647-496d25134409",
            "results": {
                "name": "Sunshine",
                "epoch": null,
                "version": "0.0.4245",
                "release": "1",
                "exclusivearch": [],
                "excludearch": []
            },
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": true,
            "destdir": "/var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/lizardbyte/pulls:pr:4245",
            "result_dir": "09541185",
            "built_packages": "",
            "tags": [
                "arch_x86_64"
            ],
            "pkg_version": "0.0.4245-1",
            "id": 9541185,
            "mockchain_macros": {
                "copr_username": "lizardbyte",
                "copr_projectname": "pulls",
                "vendor": "Fedora Project COPR (lizardbyte/pulls)"
            }
        }
    ]
}
[2025-09-10 04:03:55,608][  INFO][PID:528809] Sending fedora-messaging bus message in build.end
[2025-09-10 04:03:56,374][  INFO][PID:528809] Compressing /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/builder-live.log by gzip
[2025-09-10 04:03:56,441][  INFO][PID:528809] Running command 'gzip /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/builder-live.log' as PID 2332001
[2025-09-10 04:03:56,486][  INFO][PID:528809] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/builder-live.log)
[2025-09-10 04:03:56,493][  INFO][PID:528809] Compressing /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/backend.log by gzip
[2025-09-10 04:03:56,498][  INFO][PID:528809] Running command 'gzip /var/lib/copr/public_html/results/lizardbyte/pulls:pr:4245/srpm-builds/09541185/backend.log' as PID 2332007