[2025-11-08 01:11:07,726][  INFO][PID:31133] Marking build as starting
[2025-11-08 01:11:07,824][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:07,840][  INFO][PID:31133] VM allocation process starts
[2025-11-08 01:11:08,284][  INFO][PID:31133] Trying to allocate VM: ResallocHost, ticket_id=6081588, requested_tags=['copr_builder', 'arch_x86_64']
[2025-11-08 01:11:13,294][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:18,296][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:23,309][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:28,311][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:33,318][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:38,321][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:43,328][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:48,337][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:53,343][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:11:58,344][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:03,347][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:08,356][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:13,365][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:18,370][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:23,379][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:28,383][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:33,390][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:38,396][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:43,398][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:48,408][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:53,410][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:12:58,417][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:03,428][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:08,430][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:13,436][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:18,440][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:23,446][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:28,447][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:33,452][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:38,453][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:43,456][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:48,468][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:53,472][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:13:58,475][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:03,479][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:08,499][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:13,504][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:18,510][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:23,517][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:26,984][  INFO][PID:31133] Allocated host ResallocHost, ticket_id=6081588, hostname=98.92.159.192, name=aws_x86_64_spot_prod_07890210_20251108_011145, requested_tags=['copr_builder', 'arch_x86_64']
[2025-11-08 01:14:26,986][  INFO][PID:31133] Allocating ssh connection to builder
[2025-11-08 01:14:26,986][  INFO][PID:31133] Checking that builder machine is OK
[2025-11-08 01:14:27,584][  INFO][PID:31133] Running remote command: copr-builder-ready fedora-41-x86_64
[2025-11-08 01:14:27,977][  INFO][PID:31133] Red Hat subscription not needed for fedora-41-x86_64
Builder is ready to be used

[2025-11-08 01:14:27,980][  INFO][PID:31133] Filling build.info file with builder info
[2025-11-08 01:14:27,981][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:27,988][  INFO][PID:31133] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "rpmsoftwaremanagement",
            "project_name": "dnf-nightly-sanitizers",
            "project_dirname": "dnf-nightly-sanitizers",
            "submitter": "rpmsoftwaremanagement",
            "ended_on": null,
            "started_on": 1762564467.9878602,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-41-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9777722-fedora-41-x86_64",
            "build_id": 9777722,
            "package_name": "libpkgmanifest",
            "package_version": "0.5.9-20251108010924.7.g5ea5353",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rpmsoftwaremanagement/dnf-nightly-sanitizers/libpkgmanifest",
            "git_hash": "3965eacf5dc81f1f4686c8fbdb56f5787c40a1c1",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rpmsoftwaremanagement/dnf-nightly-sanitizers--rpmsoftwaremanagement",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [
                "sanitizers"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly-sanitizers",
            "result_dir": "09777722-libpkgmanifest",
            "built_packages": "",
            "id": 9777722,
            "mockchain_macros": {
                "copr_username": "rpmsoftwaremanagement",
                "copr_projectname": "dnf-nightly-sanitizers",
                "vendor": "Fedora Project COPR (rpmsoftwaremanagement/dnf-nightly-sanitizers)"
            }
        }
    ]
}
[2025-11-08 01:14:28,075][  INFO][PID:31133] Sending fedora-messaging bus message in build.start
[2025-11-08 01:14:31,545][  INFO][PID:31133] Sending fedora-messaging bus message in chroot.start
[2025-11-08 01:14:31,676][  INFO][PID:31133] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9777722-fedora-41-x86_64 --chroot fedora-41-x86_64 --detached
[2025-11-08 01:14:32,131][  INFO][PID:31133] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '98.92.159.192' (ED25519) to the list of known hosts.


[2025-11-08 01:14:32,137][  INFO][PID:31133] Downloading the builder-live.log file, attempt 1
[2025-11-08 01:14:32,150][  INFO][PID:31133] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.159.192 copr-rpmbuild-log
[2025-11-08 01:14:37,220][  INFO][PID:31133] Periodic builder liveness probe: alive
[2025-11-08 01:14:37,228][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:42,233][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:47,235][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:52,241][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:14:57,244][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:02,249][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:07,255][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:12,260][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:17,265][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:22,270][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:27,279][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:32,295][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:37,305][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:42,312][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:47,321][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:52,329][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:15:57,345][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:02,347][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:07,349][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:12,351][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:17,368][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:22,385][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:27,393][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:32,406][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:37,412][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:42,419][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:47,423][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:52,428][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:16:57,430][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:02,437][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:07,439][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:12,450][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:17,452][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:22,453][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:27,463][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:32,470][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:37,492][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:42,493][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:47,496][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:52,506][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:17:57,508][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:02,515][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:07,518][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:12,529][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:17,531][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:22,537][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:27,557][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:32,563][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:37,565][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:42,567][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:47,570][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:52,036][ ERROR][PID:31133] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-11-08 01:18:52,575][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:18:57,577][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:02,045][WARNING][PID:31133] SSH connection lost on #1 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-11-08 01:19:02,586][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:07,591][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:12,051][  INFO][PID:31133] Downloading the builder-live.log file, attempt 2
[2025-11-08 01:19:12,068][  INFO][PID:31133] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.159.192 copr-rpmbuild-log
[2025-11-08 01:19:12,612][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:17,626][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:22,630][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:27,639][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:32,656][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:37,658][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:42,664][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:47,682][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:52,685][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:19:57,707][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:02,713][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:07,718][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:12,220][ ERROR][PID:31133] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-11-08 01:20:12,729][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:17,736][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:22,227][WARNING][PID:31133] SSH connection lost on #2 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-11-08 01:20:22,756][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:27,760][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:32,228][  INFO][PID:31133] Downloading the builder-live.log file, attempt 3
[2025-11-08 01:20:32,232][  INFO][PID:31133] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.159.192 copr-rpmbuild-log
[2025-11-08 01:20:32,762][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:37,774][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:42,792][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:47,797][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:52,817][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:20:57,819][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:02,833][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:07,840][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:12,852][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:17,873][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:22,879][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:27,880][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:32,422][ ERROR][PID:31133] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-11-08 01:21:32,889][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:37,893][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:42,434][WARNING][PID:31133] SSH connection lost on #3 attempt, let's retry after 10s, Unable to finish after 1 SSH attempts
[2025-11-08 01:21:42,895][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:47,896][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:52,437][  INFO][PID:31133] Downloading the builder-live.log file, attempt 4
[2025-11-08 01:21:52,445][  INFO][PID:31133] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@98.92.159.192 copr-rpmbuild-log
[2025-11-08 01:21:52,897][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:21:57,915][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:02,919][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:07,922][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:12,923][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:17,927][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:22,940][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:27,945][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:32,951][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:37,954][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:42,959][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:47,965][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:52,522][ ERROR][PID:31133] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke.
[2025-11-08 01:22:52,973][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:22:57,974][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:02,526][ ERROR][PID:31133] Giving up for unstable SSH, failures: 1762564742.0457861, 1762564822.227798, 1762564902.4347088, 1762564982.525961
[2025-11-08 01:23:02,528][ ERROR][PID:31133] Re-try request for task on 'ResallocHost, ticket_id=6081588, hostname=98.92.159.192, name=aws_x86_64_spot_prod_07890210_20251108_011145, requested_tags=['copr_builder', 'arch_x86_64']': SSH problems when downloading live log: Stopped following builder for broken SSH: Unable to finish after 1 SSH attempts
[2025-11-08 01:23:02,533][  INFO][PID:31133] Releasing VM back to pool
[2025-11-08 01:23:02,591][  INFO][PID:31133] Retry #1 (on other host)
[2025-11-08 01:23:02,594][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:02,597][  INFO][PID:31133] VM allocation process starts
[2025-11-08 01:23:02,681][  INFO][PID:31133] Trying to allocate VM: ResallocHost, ticket_id=6081899, requested_tags=['copr_builder', 'arch_x86_64']
[2025-11-08 01:23:07,688][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:08,815][  INFO][PID:31133] Allocated host ResallocHost, ticket_id=6081899, hostname=2620:52:3:1:dead:beef:cafe:c1ce, name=copr_hv_x86_64_04_prod_07890206_20251108_011143, requested_tags=['copr_builder', 'arch_x86_64']
[2025-11-08 01:23:08,818][  INFO][PID:31133] Allocating ssh connection to builder
[2025-11-08 01:23:08,821][  INFO][PID:31133] Checking that builder machine is OK
[2025-11-08 01:23:09,923][  INFO][PID:31133] Running remote command: copr-builder-ready fedora-41-x86_64
[2025-11-08 01:23:10,312][  INFO][PID:31133] Red Hat subscription not needed for fedora-41-x86_64
Builder is ready to be used

[2025-11-08 01:23:10,314][  INFO][PID:31133] Filling build.info file with builder info
[2025-11-08 01:23:10,315][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:10,316][  INFO][PID:31133] Not re-notifying FE and msg buses for the new host.
[2025-11-08 01:23:10,317][  INFO][PID:31133] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/9777722-fedora-41-x86_64 --chroot fedora-41-x86_64 --detached
[2025-11-08 01:23:10,875][  INFO][PID:31133] The copr-rpmbuild seems started, per:
stdout: 
stderr: Warning: Permanently added '2620:52:3:1:dead:beef:cafe:c1ce' (ED25519) to the list of known hosts.


[2025-11-08 01:23:10,877][  INFO][PID:31133] Downloading the builder-live.log file, attempt 1
[2025-11-08 01:23:10,879][  INFO][PID:31133] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c1ce copr-rpmbuild-log
[2025-11-08 01:23:15,877][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:20,879][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:25,880][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:30,883][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:35,893][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:40,899][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:45,900][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:50,906][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:23:55,909][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:00,912][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:05,916][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:10,919][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:15,924][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:20,928][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:25,930][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:30,932][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:35,933][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:40,957][  INFO][PID:31133] Periodic builder liveness probe: alive
[2025-11-08 01:24:40,960][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:45,963][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:50,966][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:24:55,970][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:00,971][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:05,973][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:10,975][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:15,980][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:20,986][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:25,989][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:30,992][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:35,994][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:41,001][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:46,006][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:51,007][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:25:56,010][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:01,012][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:06,013][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:11,015][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:16,016][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:21,020][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:26,021][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:31,022][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:36,023][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:41,025][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:46,027][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:51,028][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:26:56,030][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:01,031][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:06,032][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:11,033][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:16,034][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:21,036][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:26,037][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:31,038][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:36,041][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:41,042][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:46,043][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:51,044][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:27:56,045][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:01,046][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:06,047][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:11,048][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:16,050][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:21,051][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:26,052][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:31,053][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:36,054][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:41,055][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:46,056][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:51,057][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:28:56,058][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:01,059][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:06,061][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:11,062][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:16,063][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:21,064][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:26,065][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:31,066][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:36,067][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:41,068][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:46,069][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:51,070][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:29:56,071][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:01,072][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:06,073][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:11,075][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:16,076][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:21,077][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:26,078][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:31,079][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:36,080][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:41,081][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:46,082][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:51,083][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:30:56,084][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:31:01,085][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:31:06,086][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:31:11,087][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:31:16,088][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:31:21,089][  INFO][PID:31133] Checking for cancel request
[2025-11-08 01:31:23,439][  INFO][PID:31133] Downloading results from builder
[2025-11-08 01:31:23,440][  INFO][PID:31133] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c1ce]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest started
[2025-11-08 01:31:23,441][  INFO][PID:31133] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c1ce]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/ &> /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/build-09777722.rsync.log
[2025-11-08 01:31:24,099][  INFO][PID:31133] rsyncing finished.
[2025-11-08 01:31:24,100][  INFO][PID:31133] Releasing VM back to pool
[2025-11-08 01:31:24,119][  INFO][PID:31133] Searching for 'success' file in resultdir
[2025-11-08 01:31:24,120][  INFO][PID:31133] Going to sign pkgs from source: 9777722-fedora-41-x86_64 in chroot: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64
[2025-11-08 01:31:24,121][  INFO][PID:31133] Calling '/bin/sign -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -p' (attempt #1)
[2025-11-08 01:31:24,462][  INFO][PID:31133] Calling '/bin/sign -4 -h sha256 -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm' (attempt #1)
[2025-11-08 01:31:24,930][  INFO][PID:31133] signed rpm: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm
[2025-11-08 01:31:24,931][  INFO][PID:31133] Calling '/bin/sign -4 -h sha256 -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-devel-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm' (attempt #1)
[2025-11-08 01:31:25,693][  INFO][PID:31133] signed rpm: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-devel-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm
[2025-11-08 01:31:25,693][  INFO][PID:31133] Calling '/bin/sign -4 -h sha256 -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-debugsource-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm' (attempt #1)
[2025-11-08 01:31:26,486][  INFO][PID:31133] signed rpm: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-debugsource-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm
[2025-11-08 01:31:26,487][  INFO][PID:31133] Calling '/bin/sign -4 -h sha256 -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-0.5.9-20251108010924.7.g5ea5353.fc41.src.rpm' (attempt #1)
[2025-11-08 01:31:27,095][  INFO][PID:31133] signed rpm: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-0.5.9-20251108010924.7.g5ea5353.fc41.src.rpm
[2025-11-08 01:31:27,096][  INFO][PID:31133] Calling '/bin/sign -4 -h sha256 -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/python3-libpkgmanifest-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm' (attempt #1)
[2025-11-08 01:31:28,028][  INFO][PID:31133] signed rpm: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/python3-libpkgmanifest-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm
[2025-11-08 01:31:28,029][  INFO][PID:31133] Calling '/bin/sign -4 -h sha256 -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-debuginfo-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm' (attempt #1)
[2025-11-08 01:31:29,026][  INFO][PID:31133] signed rpm: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/libpkgmanifest-debuginfo-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm
[2025-11-08 01:31:29,026][  INFO][PID:31133] Calling '/bin/sign -4 -h sha256 -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -r /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/python3-libpkgmanifest-debuginfo-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm' (attempt #1)
[2025-11-08 01:31:29,799][  INFO][PID:31133] signed rpm: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/python3-libpkgmanifest-debuginfo-0.5.9-20251108010924.7.g5ea5353.fc41.x86_64.rpm
[2025-11-08 01:31:29,799][  INFO][PID:31133] Sign done
[2025-11-08 01:31:29,800][  INFO][PID:31133] Incremental createrepo run, adding 09777722-libpkgmanifest into https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64, (auto-create-repo=True)
[2025-11-08 01:31:29,801][  INFO][PID:31133] Running command 'copr-repo --batched /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64 --add 09777722-libpkgmanifest --no-appstream-metadata' as PID 224730
[2025-11-08 01:31:30,638][  INFO][PID:31133] Finished after 0 seconds with exit code 0 (copr-repo --batched /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64 --add 09777722-libpkgmanifest --no-appstream-metadata)
[2025-11-08 01:31:30,639][  INFO][PID:31133] Getting build details
[2025-11-08 01:31:30,640][  INFO][PID:31133] Listing built binary packages in /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest
[2025-11-08 01:31:30,640][  INFO][PID:31133] Built packages:
['libpkgmanifest-debuginfo 0.5.9', 'python3-libpkgmanifest-debuginfo 0.5.9', 'libpkgmanifest 0.5.9', 'libpkgmanifest-debugsource 0.5.9', 'libpkgmanifest-devel 0.5.9', 'python3-libpkgmanifest 0.5.9']
[2025-11-08 01:31:30,641][  INFO][PID:31133] build details: {'built_packages': 'libpkgmanifest-debuginfo 0.5.9\npython3-libpkgmanifest-debuginfo 0.5.9\nlibpkgmanifest 0.5.9\nlibpkgmanifest-debugsource 0.5.9\nlibpkgmanifest-devel 0.5.9\npython3-libpkgmanifest 0.5.9'}
[2025-11-08 01:31:30,641][  INFO][PID:31133] Retrieving pubkey
[2025-11-08 01:31:30,641][  INFO][PID:31133] Calling '/bin/sign -u rpmsoftwaremanagement#dnf-nightly-sanitizers@copr.fedorahosted.org -p' (attempt #1)
[2025-11-08 01:31:30,823][  INFO][PID:31133] Added pubkey for user rpmsoftwaremanagement project dnf-nightly-sanitizers into: /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/pubkey.gpg
[2025-11-08 01:31:30,823][  INFO][PID:31133] Finished build: id=9777722 failed=False timeout=18000 destdir=/var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers chroot=fedora-41-x86_64 
[2025-11-08 01:31:30,825][  INFO][PID:31133] Worker succeeded build, took 500.50835824012756
[2025-11-08 01:31:30,825][  INFO][PID:31133] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 18000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": false,
            "project_owner": "rpmsoftwaremanagement",
            "project_name": "dnf-nightly-sanitizers",
            "project_dirname": "dnf-nightly-sanitizers",
            "submitter": "rpmsoftwaremanagement",
            "ended_on": 1762565490.8251557,
            "started_on": 1762564990.3167975,
            "submitted_on": null,
            "status": 1,
            "chroot": "fedora-41-x86_64",
            "arch": "x86_64",
            "buildroot_pkgs": [],
            "task_id": "9777722-fedora-41-x86_64",
            "build_id": 9777722,
            "package_name": "libpkgmanifest",
            "package_version": "0.5.9-20251108010924.7.g5ea5353",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/rpmsoftwaremanagement/dnf-nightly-sanitizers/libpkgmanifest",
            "git_hash": "3965eacf5dc81f1f4686c8fbdb56f5787c40a1c1",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "rpmsoftwaremanagement/dnf-nightly-sanitizers--rpmsoftwaremanagement",
            "results": {
                "packages": [
                    {
                        "name": "libpkgmanifest",
                        "epoch": null,
                        "version": "0.5.9",
                        "release": "20251108010924.7.g5ea5353.fc41",
                        "arch": "src"
                    },
                    {
                        "name": "libpkgmanifest-debuginfo",
                        "epoch": null,
                        "version": "0.5.9",
                        "release": "20251108010924.7.g5ea5353.fc41",
                        "arch": "x86_64"
                    },
                    {
                        "name": "python3-libpkgmanifest-debuginfo",
                        "epoch": null,
                        "version": "0.5.9",
                        "release": "20251108010924.7.g5ea5353.fc41",
                        "arch": "x86_64"
                    },
                    {
                        "name": "libpkgmanifest",
                        "epoch": null,
                        "version": "0.5.9",
                        "release": "20251108010924.7.g5ea5353.fc41",
                        "arch": "x86_64"
                    },
                    {
                        "name": "libpkgmanifest-debugsource",
                        "epoch": null,
                        "version": "0.5.9",
                        "release": "20251108010924.7.g5ea5353.fc41",
                        "arch": "x86_64"
                    },
                    {
                        "name": "libpkgmanifest-devel",
                        "epoch": null,
                        "version": "0.5.9",
                        "release": "20251108010924.7.g5ea5353.fc41",
                        "arch": "x86_64"
                    },
                    {
                        "name": "python3-libpkgmanifest",
                        "epoch": null,
                        "version": "0.5.9",
                        "release": "20251108010924.7.g5ea5353.fc41",
                        "arch": "x86_64"
                    }
                ]
            },
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/",
                    "id": "copr_base",
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "background": false,
            "bootstrap": "off",
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "tags": [
                "arch_x86_64"
            ],
            "with_opts": [
                "sanitizers"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/rpmsoftwaremanagement/dnf-nightly-sanitizers",
            "result_dir": "09777722-libpkgmanifest",
            "built_packages": "libpkgmanifest-debuginfo 0.5.9\npython3-libpkgmanifest-debuginfo 0.5.9\nlibpkgmanifest 0.5.9\nlibpkgmanifest-debugsource 0.5.9\nlibpkgmanifest-devel 0.5.9\npython3-libpkgmanifest 0.5.9",
            "id": 9777722,
            "mockchain_macros": {
                "copr_username": "rpmsoftwaremanagement",
                "copr_projectname": "dnf-nightly-sanitizers",
                "vendor": "Fedora Project COPR (rpmsoftwaremanagement/dnf-nightly-sanitizers)"
            }
        }
    ]
}
[2025-11-08 01:31:30,867][  INFO][PID:31133] Sending fedora-messaging bus message in build.end
[2025-11-08 01:31:30,891][  INFO][PID:31133] Compressing /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/builder-live.log by gzip
[2025-11-08 01:31:30,892][  INFO][PID:31133] Running command 'gzip /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/builder-live.log' as PID 224809
[2025-11-08 01:31:30,900][  INFO][PID:31133] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/builder-live.log)
[2025-11-08 01:31:30,901][  INFO][PID:31133] Compressing /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/backend.log by gzip
[2025-11-08 01:31:30,902][  INFO][PID:31133] Running command 'gzip /var/lib/copr/public_html/results/rpmsoftwaremanagement/dnf-nightly-sanitizers/fedora-41-x86_64/09777722-libpkgmanifest/backend.log' as PID 224811