[2024-11-22 02:19:56,065][  INFO][PID:3599050] Marking build as starting
[2024-11-22 02:19:56,470][  INFO][PID:3599050] Trying to allocate VM
[2024-11-22 02:20:08,650][  INFO][PID:3599050] Allocated host ResallocHost, ticket_id=6014297, hostname=2620:52:3:1:dead:beef:cafe:c24f
[2024-11-22 02:20:08,652][  INFO][PID:3599050] Allocating ssh connection to builder
[2024-11-22 02:20:08,656][  INFO][PID:3599050] Checking that builder machine is OK
[2024-11-22 02:20:09,498][  INFO][PID:3599050] Running remote command: copr-builder-ready fedora-39-ppc64le
[2024-11-22 02:20:10,202][  INFO][PID:3599050] Red Hat subscription not needed for fedora-39-ppc64le
Builder is ready to be used

[2024-11-22 02:20:10,206][  INFO][PID:3599050] Filling build.info file with builder info
[2024-11-22 02:20:10,210][  INFO][PID:3599050] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "@fedora-llvm-team",
            "project_name": "llvm-snapshots-pgo-20241122",
            "project_dirname": "llvm-snapshots-pgo-20241122",
            "submitter": "tstellar",
            "ended_on": null,
            "started_on": 1732242010.2098925,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-39-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "8300227-fedora-39-ppc64le",
            "build_id": 8300227,
            "package_name": "llvm",
            "package_version": "19.1.3-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/llvm-snapshots-pgo-20241122/llvm",
            "git_hash": "eb625212ea1b1f440766a5af783d9c81ce0af722",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@fedora-llvm-team/llvm-snapshots-pgo-20241122--tstellar",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/",
                    "id": "copr_base",
                    "module_hotfixes": true,
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [
                "snapshot_build"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122",
            "result_dir": "08300227-llvm",
            "built_packages": "",
            "id": 8300227,
            "mockchain_macros": {
                "copr_username": "@fedora-llvm-team",
                "copr_projectname": "llvm-snapshots-pgo-20241122",
                "vendor": "Fedora Project COPR (@fedora-llvm-team/llvm-snapshots-pgo-20241122)"
            }
        }
    ]
}
[2024-11-22 02:20:10,330][  INFO][PID:3599050] Sending fedora-messaging bus message in build.start
[2024-11-22 02:20:11,371][  INFO][PID:3599050] Sending fedora-messaging bus message in chroot.start
[2024-11-22 02:20:11,396][  INFO][PID:3599050] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8300227-fedora-39-ppc64le --chroot fedora-39-ppc64le --detached
[2024-11-22 02:20:12,178][  INFO][PID:3599050] Downloading the builder-live.log file, attempt 1
[2024-11-22 02:20:12,181][  INFO][PID:3599050] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@2620:52:3:1:dead:beef:cafe:c24f copr-rpmbuild-log
[2024-11-22 02:20:17,191][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 02:30:17,309][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 02:40:17,386][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 02:50:17,462][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 03:00:17,544][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 03:10:17,661][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 03:20:17,762][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 03:30:17,836][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 03:40:17,941][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 03:50:18,018][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 04:00:18,104][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 04:10:18,321][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 04:20:18,787][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 04:30:18,880][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 04:40:18,962][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 04:50:19,050][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 05:00:19,135][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 05:10:19,251][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 05:20:19,355][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 05:30:19,444][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 05:40:19,529][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 05:50:19,615][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 06:00:19,698][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 06:10:19,813][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 06:20:19,920][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 06:30:20,029][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 06:40:20,131][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 06:50:20,216][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 07:00:20,302][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 07:10:20,400][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 07:20:20,506][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 07:30:20,591][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 07:40:20,678][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 07:50:20,821][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 08:00:20,963][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 08:10:21,092][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 08:20:21,202][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 08:30:21,312][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 08:40:21,495][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 08:50:21,738][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 09:00:22,004][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 09:10:22,270][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 09:20:22,608][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 09:30:22,700][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 09:40:22,792][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 09:50:22,914][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 10:00:23,101][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 10:10:23,257][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 10:20:23,486][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 10:30:23,611][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 10:40:23,712][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 10:50:23,788][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 11:00:23,907][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 11:10:24,037][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 11:20:24,137][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 11:30:24,552][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 11:40:25,047][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 11:50:25,539][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 12:00:25,633][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 12:10:25,739][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 12:20:25,891][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 12:30:25,984][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 12:40:26,081][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 12:50:26,159][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 13:00:26,288][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 13:10:26,428][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 13:20:26,526][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 13:30:26,602][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 13:40:26,691][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 13:50:26,778][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 14:00:26,873][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 14:10:27,036][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 14:20:27,175][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 14:30:27,258][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 14:40:27,342][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 14:50:27,555][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 15:00:27,716][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 15:10:27,856][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 15:20:28,083][  INFO][PID:3599050] Periodic builder liveness probe: alive
[2024-11-22 15:20:42,444][  INFO][PID:3599050] Downloading results from builder
[2024-11-22 15:20:42,486][  INFO][PID:3599050] rsyncing of mockbuilder@[2620:52:3:1:dead:beef:cafe:c24f]:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/08300227-llvm started
[2024-11-22 15:20:42,527][  INFO][PID:3599050] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@[2620:52:3:1:dead:beef:cafe:c24f]:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/08300227-llvm/ &> /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/08300227-llvm/build-08300227.rsync.log
[2024-11-22 15:20:49,024][  INFO][PID:3599050] rsyncing finished.
[2024-11-22 15:20:49,035][  INFO][PID:3599050] Releasing VM back to pool
[2024-11-22 15:20:49,091][  INFO][PID:3599050] Searching for 'success' file in resultdir
[2024-11-22 15:20:49,113][ ERROR][PID:3599050] Build failed: Backend process error: No success file => build failure
[2024-11-22 15:20:49,271][  INFO][PID:3599050] Finished build: id=8300227 failed=True timeout=108000 destdir=/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122 chroot=fedora-39-ppc64le 
[2024-11-22 15:20:49,374][ ERROR][PID:3599050] Backend process error: No success file => build failure
[2024-11-22 15:20:49,378][  INFO][PID:3599050] Worker failed build, took 46839.16770029068
[2024-11-22 15:20:49,397][  INFO][PID:3599050] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "@fedora-llvm-team",
            "project_name": "llvm-snapshots-pgo-20241122",
            "project_dirname": "llvm-snapshots-pgo-20241122",
            "submitter": "tstellar",
            "ended_on": 1732288849.3775928,
            "started_on": 1732242010.2098925,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-39-ppc64le",
            "arch": "ppc64le",
            "buildroot_pkgs": [],
            "task_id": "8300227-fedora-39-ppc64le",
            "build_id": 8300227,
            "package_name": "llvm",
            "package_version": "19.1.3-1",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/llvm-snapshots-pgo-20241122/llvm",
            "git_hash": "eb625212ea1b1f440766a5af783d9c81ce0af722",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@fedora-llvm-team/llvm-snapshots-pgo-20241122--tstellar",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/",
                    "id": "copr_base",
                    "module_hotfixes": true,
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "tags": [
                "arch_ppc64le"
            ],
            "with_opts": [
                "snapshot_build"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122",
            "result_dir": "08300227-llvm",
            "built_packages": "",
            "id": 8300227,
            "mockchain_macros": {
                "copr_username": "@fedora-llvm-team",
                "copr_projectname": "llvm-snapshots-pgo-20241122",
                "vendor": "Fedora Project COPR (@fedora-llvm-team/llvm-snapshots-pgo-20241122)"
            }
        }
    ]
}
[2024-11-22 15:20:50,765][  INFO][PID:3599050] Sending fedora-messaging bus message in build.end
[2024-11-22 15:20:51,146][  INFO][PID:3599050] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/08300227-llvm/builder-live.log by gzip
[2024-11-22 15:20:51,171][  INFO][PID:3599050] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/08300227-llvm/builder-live.log' as PID 2816046
[2024-11-22 15:20:51,928][  INFO][PID:3599050] Finished after 0 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/08300227-llvm/builder-live.log)
[2024-11-22 15:20:51,939][  INFO][PID:3599050] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/08300227-llvm/backend.log by gzip
[2024-11-22 15:20:51,949][  INFO][PID:3599050] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241122/fedora-39-ppc64le/08300227-llvm/backend.log' as PID 2816124