[2024-11-15 02:21:20,939][  INFO][PID:319423] Marking build as starting
[2024-11-15 02:21:21,042][  INFO][PID:319423] Trying to allocate VM
[2024-11-15 02:21:24,096][  INFO][PID:319423] Allocated host ResallocHost, ticket_id=5889877, hostname=23.20.150.234
[2024-11-15 02:21:24,097][  INFO][PID:319423] Allocating ssh connection to builder
[2024-11-15 02:21:24,098][  INFO][PID:319423] Checking that builder machine is OK
[2024-11-15 02:21:24,546][  INFO][PID:319423] Running remote command: copr-builder-ready fedora-40-aarch64
[2024-11-15 02:21:24,802][  INFO][PID:319423] Red Hat subscription not needed for fedora-40-aarch64
Builder is ready to be used

[2024-11-15 02:21:24,803][  INFO][PID:319423] Filling build.info file with builder info
[2024-11-15 02:21:24,805][  INFO][PID:319423] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "@fedora-llvm-team",
            "project_name": "llvm-snapshots-pgo-20241115",
            "project_dirname": "llvm-snapshots-pgo-20241115",
            "submitter": "tstellar",
            "ended_on": null,
            "started_on": 1731637284.8048573,
            "submitted_on": null,
            "status": 3,
            "chroot": "fedora-40-aarch64",
            "arch": "aarch64",
            "buildroot_pkgs": [],
            "task_id": "8264265-fedora-40-aarch64",
            "build_id": 8264265,
            "package_name": "llvm",
            "package_version": "19.1.0-2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/llvm-snapshots-pgo-20241115/llvm",
            "git_hash": "35ecd33e97d0542492434de88abc8d99055e608c",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@fedora-llvm-team/llvm-snapshots-pgo-20241115--tstellar",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/",
                    "id": "copr_base",
                    "module_hotfixes": true,
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "tags": [
                "arch_aarch64"
            ],
            "with_opts": [
                "snapshot_build"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115",
            "result_dir": "08264265-llvm",
            "built_packages": "",
            "id": 8264265,
            "mockchain_macros": {
                "copr_username": "@fedora-llvm-team",
                "copr_projectname": "llvm-snapshots-pgo-20241115",
                "vendor": "Fedora Project COPR (@fedora-llvm-team/llvm-snapshots-pgo-20241115)"
            }
        }
    ]
}
[2024-11-15 02:21:24,868][  INFO][PID:319423] Sending fedora-messaging bus message in build.start
[2024-11-15 02:21:25,656][  INFO][PID:319423] Sending fedora-messaging bus message in chroot.start
[2024-11-15 02:21:25,708][  INFO][PID:319423] Starting remote build: copr-rpmbuild --verbose --drop-resultdir --task-url https://copr.fedorainfracloud.org/backend/get-build-task/8264265-fedora-40-aarch64 --chroot fedora-40-aarch64 --detached
[2024-11-15 02:21:26,202][  INFO][PID:319423] Downloading the builder-live.log file, attempt 1
[2024-11-15 02:21:26,204][  INFO][PID:319423] Popen command started: ssh -F /home/copr/.ssh/config mockbuilder@23.20.150.234 copr-rpmbuild-log
[2024-11-15 02:21:31,258][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 02:31:31,369][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 02:41:31,453][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 02:51:31,542][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 03:01:31,628][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 03:11:31,709][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 03:21:31,787][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 03:31:31,868][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 03:41:31,953][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 03:51:32,094][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 04:01:32,278][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 04:11:32,494][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 04:21:32,601][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 04:31:32,757][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 04:41:33,049][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 04:51:33,173][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 05:01:33,262][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 05:11:33,383][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 05:21:33,479][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 05:31:33,567][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 05:41:33,653][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 05:51:33,748][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 06:01:33,840][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 06:11:34,084][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 06:21:34,430][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 06:31:34,587][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 06:41:34,676][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 06:51:34,768][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 07:01:34,863][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 07:11:35,024][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 07:21:35,126][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 07:31:35,226][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 07:41:35,324][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 07:51:35,519][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 08:01:35,642][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 08:11:35,735][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 08:21:35,815][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 08:31:35,908][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 08:41:36,109][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 08:51:36,199][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 09:01:36,382][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 09:11:36,624][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 09:21:37,061][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 09:31:37,193][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 09:41:37,275][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 09:51:37,430][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 10:01:37,899][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 10:11:38,325][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 10:21:38,682][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 10:31:38,940][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 10:41:39,184][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 10:51:39,276][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 11:01:39,414][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 11:11:39,620][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 11:21:40,013][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 11:31:40,102][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 11:41:40,185][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 11:51:40,268][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 12:01:41,163][  INFO][PID:319423] Periodic builder liveness probe: alive
[2024-11-15 12:09:03,821][  INFO][PID:319423] Downloading results from builder
[2024-11-15 12:09:03,872][  INFO][PID:319423] rsyncing of mockbuilder@23.20.150.234:/var/lib/copr-rpmbuild/results/ to /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/08264265-llvm started
[2024-11-15 12:09:03,917][  INFO][PID:319423] Popen command started: /usr/bin/rsync -rltDvH --chmod=D755,F644 -e 'ssh -F /home/copr/.ssh/config' mockbuilder@23.20.150.234:/var/lib/copr-rpmbuild/results/ /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/08264265-llvm/ &> /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/08264265-llvm/build-08264265.rsync.log
[2024-11-15 12:09:07,042][  INFO][PID:319423] rsyncing finished.
[2024-11-15 12:09:07,044][  INFO][PID:319423] Releasing VM back to pool
[2024-11-15 12:09:07,068][  INFO][PID:319423] Searching for 'success' file in resultdir
[2024-11-15 12:09:07,077][ ERROR][PID:319423] Build failed: Backend process error: No success file => build failure
[2024-11-15 12:09:07,203][  INFO][PID:319423] Finished build: id=8264265 failed=True timeout=108000 destdir=/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115 chroot=fedora-40-aarch64 
[2024-11-15 12:09:07,308][ ERROR][PID:319423] Backend process error: No success file => build failure
[2024-11-15 12:09:07,311][  INFO][PID:319423] Worker failed build, took 35262.505303144455
[2024-11-15 12:09:07,325][  INFO][PID:319423] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.fedorainfracloud.org",
            "memory_reqs": 2048,
            "enable_net": true,
            "project_owner": "@fedora-llvm-team",
            "project_name": "llvm-snapshots-pgo-20241115",
            "project_dirname": "llvm-snapshots-pgo-20241115",
            "submitter": "tstellar",
            "ended_on": 1731672547.3101604,
            "started_on": 1731637284.8048573,
            "submitted_on": null,
            "status": 0,
            "chroot": "fedora-40-aarch64",
            "arch": "aarch64",
            "buildroot_pkgs": [],
            "task_id": "8264265-fedora-40-aarch64",
            "build_id": 8264265,
            "package_name": "llvm",
            "package_version": "19.1.0-2",
            "git_repo": "https://copr-dist-git.fedorainfracloud.org/git/@fedora-llvm-team/llvm-snapshots-pgo-20241115/llvm",
            "git_hash": "35ecd33e97d0542492434de88abc8d99055e608c",
            "git_branch": null,
            "source_type": null,
            "source_json": null,
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": false,
            "sandbox": "@fedora-llvm-team/llvm-snapshots-pgo-20241115--tstellar",
            "results": null,
            "appstream": false,
            "allow_user_ssh": false,
            "ssh_public_keys": null,
            "storage": 0,
            "background": false,
            "fedora_review": false,
            "isolation": "default",
            "repo_priority": null,
            "repos": [
                {
                    "baseurl": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/",
                    "id": "copr_base",
                    "module_hotfixes": true,
                    "name": "Copr repository",
                    "priority": null
                }
            ],
            "tags": [
                "arch_aarch64"
            ],
            "with_opts": [
                "snapshot_build"
            ],
            "without_opts": [],
            "destdir": "/var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115",
            "results_repo_url": "https://download.copr.fedorainfracloud.org/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115",
            "result_dir": "08264265-llvm",
            "built_packages": "",
            "id": 8264265,
            "mockchain_macros": {
                "copr_username": "@fedora-llvm-team",
                "copr_projectname": "llvm-snapshots-pgo-20241115",
                "vendor": "Fedora Project COPR (@fedora-llvm-team/llvm-snapshots-pgo-20241115)"
            }
        }
    ]
}
[2024-11-15 12:09:08,407][  INFO][PID:319423] Sending fedora-messaging bus message in build.end
[2024-11-15 12:09:08,704][  INFO][PID:319423] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/08264265-llvm/builder-live.log by gzip
[2024-11-15 12:09:08,725][  INFO][PID:319423] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/08264265-llvm/builder-live.log' as PID 2849323
[2024-11-15 12:09:10,774][  INFO][PID:319423] Finished after 2 seconds with exit code 0 (gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/08264265-llvm/builder-live.log)
[2024-11-15 12:09:10,788][  INFO][PID:319423] Compressing /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/08264265-llvm/backend.log by gzip
[2024-11-15 12:09:10,795][  INFO][PID:319423] Running command 'gzip /var/lib/copr/public_html/results/@fedora-llvm-team/llvm-snapshots-pgo-20241115/fedora-40-aarch64/08264265-llvm/backend.log' as PID 2849619