# Backend reference

<!-- This file is generated by docs/tools/gen_backend_boilerplate.py from the C++ backend
descriptors. Do not edit the regions between the GENERATED markers by hand; run
the generator instead. Prose outside the markers is preserved. -->

## Backends

<!-- BEGIN GENERATED: backends-overview -->
| Recipe | Name | Selectable backend | Uses ctx_size | Backends |
|--------|------|--------------------|---------------|----------|
| `acestep` | ACE-Step | yes | no | cuda, rocm, vulkan |
| `flm` | FastFlowLM NPU | no | yes | npu |
| `kokoro` | Kokoro | no | no | cpu, metal |
| `llamacpp` | Llama.cpp GPU | yes | yes | cpu, cuda, metal, rocm, system, vulkan |
| `moonshine` | Moonshine | no | no | cpu |
| `onnxruntime` | ONNX Runtime | no | no | cpu |
| `openmoss` | OpenMOSS TTS | yes | no | cuda, rocm, vulkan |
| `ryzenai-llm` | Ryzen AI LLM | no | yes | npu |
| `sd-cpp` | StableDiffusion.cpp | yes | no | cpu, cuda, metal, rocm, vulkan |
| `thenoise` | TheNoise ROCm (experimental) | yes | no | rocm |
| `thinksound` | ThinkSound | yes | no | cuda, rocm, vulkan |
| `trellis` | TRELLIS.2 | yes | no | cuda, rocm, vulkan |
| `vllm` | vLLM ROCm (experimental) | yes | yes | rocm |
| `whispercpp` | Whisper.cpp | yes | no | cpu, metal, npu, rocm, vulkan |
<!-- END GENERATED: backends-overview -->

## Support matrix

<!-- BEGIN GENERATED: backends-matrix -->
| Recipe | Backend | OS | Device families |
|--------|---------|----|-----------------|
| `acestep` | cuda | linux, windows | nvidia_gpu |
| `acestep` | vulkan | linux, windows | amd_gpu; cpu (x86_64); nvidia_gpu |
| `acestep` | rocm | linux, windows | amd_gpu (gfx103X, gfx110X, gfx1150, gfx1151, gfx1152, gfx120X) |
| `flm` | npu | linux, windows | amd_npu (XDNA2) |
| `kokoro` | metal | macos | metal |
| `kokoro` | cpu | linux, windows | cpu (x86_64) |
| `llamacpp` | system | linux | cpu (arm64, x86_64) |
| `llamacpp` | metal | macos | metal |
| `llamacpp` | cuda | linux, windows | nvidia_gpu (sm_100, sm_120, sm_121, sm_75, sm_80, sm_86, sm_89, sm_90) |
| `llamacpp` | vulkan | linux, windows | amd_gpu; cpu (arm64, x86_64) |
| `llamacpp` | rocm | linux, windows | amd_gpu (gfx103X, gfx110X, gfx1150, gfx1151, gfx1152, gfx120X, gfx908, gfx90a, gfx942, gfx950) |
| `llamacpp` | cpu | linux, windows | cpu (arm64, x86_64) |
| `moonshine` | cpu | windows | cpu (x86_64) |
| `moonshine` | cpu | linux | cpu (arm64, x86_64) |
| `moonshine` | cpu | macos | cpu (arm64) |
| `onnxruntime` | cpu | windows | cpu (x86_64) |
| `onnxruntime` | cpu | linux | cpu (arm64, x86_64) |
| `onnxruntime` | cpu | macos | cpu (arm64) |
| `openmoss` | cuda | linux, windows | nvidia_gpu |
| `openmoss` | vulkan | linux, windows | amd_gpu; cpu (x86_64); nvidia_gpu |
| `openmoss` | rocm | linux, windows | amd_gpu |
| `ryzenai-llm` | npu | windows | amd_npu (XDNA2) |
| `sd-cpp` | metal | macos | metal |
| `sd-cpp` | cuda | linux, windows | nvidia_gpu (sm_100, sm_120, sm_121, sm_75, sm_80, sm_86, sm_89, sm_90) |
| `sd-cpp` | vulkan | linux, windows | amd_gpu; cpu (x86_64); nvidia_gpu |
| `sd-cpp` | rocm | linux, windows | amd_gpu (gfx103X, gfx110X, gfx1150, gfx1151, gfx1152, gfx120X) |
| `sd-cpp` | cpu | linux, windows | cpu (x86_64) |
| `thenoise` | rocm | linux | amd_gpu (gfx1150, gfx1151, gfx1152) |
| `thinksound` | cuda | linux, windows | nvidia_gpu |
| `thinksound` | vulkan | linux, windows | amd_gpu; cpu (x86_64); nvidia_gpu |
| `thinksound` | rocm | linux, windows | amd_gpu (gfx103X, gfx110X, gfx1150, gfx1151, gfx1152, gfx120X) |
| `trellis` | cuda | linux, windows | nvidia_gpu |
| `trellis` | vulkan | linux, windows | amd_gpu; cpu (x86_64); nvidia_gpu |
| `trellis` | rocm | linux, windows | amd_gpu (gfx103X, gfx110X, gfx1150, gfx1151, gfx1152, gfx120X) |
| `vllm` | rocm | linux | amd_gpu (gfx110X, gfx1150, gfx1151, gfx120X) |
| `whispercpp` | npu | windows | amd_npu (XDNA2) |
| `whispercpp` | metal | macos | metal |
| `whispercpp` | vulkan | linux, windows | amd_gpu; cpu (x86_64) |
| `whispercpp` | rocm | linux, windows | amd_gpu (gfx110X, gfx1150, gfx1151, gfx120X) |
| `whispercpp` | cpu | linux, windows | cpu (x86_64) |
<!-- END GENERATED: backends-matrix -->

> **Note:** The `llamacpp` `rocm` row lists `linux, windows` for the family as a
> whole, but MI350X (`gfx950`) is currently gated to **Linux + stable channel
> only** — the Windows TheRock distribution and the ROCm nightly build for
> gfx950 are not yet published, so `gfx950` installs are rejected on Windows and
> on the nightly channel. The OS column reflects the row's overall reach; the
> per-architecture restriction is enforced by the backend's install gate.

## Recipe options

<!-- BEGIN GENERATED: backend-options -->
#### `acestep` — ACE-Step

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `acestep_backend` | `--acestep` | BACKEND | "" | ACE-Step backend to use |

#### `flm` — FastFlowLM NPU

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `ctx_size` | `--ctx-size` | SIZE | -1 | Context size for the model |
| `flm_args` | `--flm-args` | ARGS | "" | Safe flm serve tuning args: --pmode, --prefill-chunk-len, --img-pre-resize, --socket, --q-len, --preemption |

#### `llamacpp` — Llama.cpp GPU

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `ctx_size` | `--ctx-size` | SIZE | -1 | Context size for the model |
| `llamacpp_backend` | `--llamacpp` | BACKEND | "" | LlamaCpp backend to use |
| `llamacpp_device` | `--llamacpp-device` | DEVICES | "" | Comma-separated list of accelerator devices to use (e.g. Vulkan0) |
| `llamacpp_args` | `--llamacpp-args` | ARGS | "" | Custom arguments to pass to llama-server |

#### `moonshine` — Moonshine

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `moonshine_args` | `--moonshine-args` | ARGS | "" | Custom arguments to pass to moonshine-server |

#### `onnxruntime` — ONNX Runtime

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `onnxruntime_args` | `--onnxruntime-args` | ARGS | "" | Custom arguments to pass to ort-server |

#### `openmoss` — OpenMOSS TTS

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `openmoss_backend` | `--openmoss` | BACKEND | "" | OpenMOSS TTS backend to use |

#### `sd-cpp` — StableDiffusion.cpp

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `sd-cpp_backend` | `--sdcpp` | BACKEND | "" | SD.cpp backend to use |
| `sdcpp_args` | `--sdcpp-args` | ARGS | "" | Custom arguments to pass to sd-server (must not conflict with managed args) |
| `steps` | — | SIZE | 20 | Number of diffusion steps |
| `cfg_scale` | — | SIZE | 7.0 | Classifier-free guidance scale |
| `width` | — | SIZE | 512 | Output image width |
| `height` | — | SIZE | 512 | Output image height |
| `sampling_method` | — | ARGS | "" | Sampling method |
| `flow_shift` | — | SIZE | 0.0 | Flow shift |

#### `thenoise` — TheNoise ROCm (experimental)

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `thenoise_backend` | `--thenoise` | BACKEND | "" | TheNoise backend to use |
| `steps` | — | SIZE | 20 | Number of denoising steps |
| `cfg_scale` | — | SIZE | 7.0 | CFG scale (<= 1.0 disables CFG) |
| `width` | — | SIZE | 512 | Output image width |
| `height` | — | SIZE | 512 | Output image height |
| `sampler` | — | ARGS | "" | Denoising solver (euler \| er_sde) |
| `negative_prompt` | — | ARGS | "" | Negative prompt |
| `qwen_vae_enhance` | — | BOOL | false | Nyquist notch post-filter (removes 2px grid artifacts) |
| `film_grain` | — | SIZE | 0.0 | Film grain strength (0.0-10.0) |
| `sharpening` | — | SIZE | 0.0 | RCAS sharpening strength (0.0-1.0) |
| `lora_specs` | — | ARGS | "" | Comma-separated LoRA specs, e.g. "style:0.8,sub/detail:0.5" |

#### `thinksound` — ThinkSound

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `thinksound_backend` | `--thinksound` | BACKEND | "" | ThinkSound backend to use |

#### `trellis` — TRELLIS.2

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `trellis_backend` | `--trellis` | BACKEND | "" | Trellis backend to use |
| `trellis_args` | `--trellis-args` | ARGS | "" | Custom arguments to pass to trellis-server |

#### `vllm` — vLLM ROCm (experimental)

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `ctx_size` | `--ctx-size` | SIZE | -1 | Context size for the model |
| `vllm_backend` | `--vllm` | BACKEND | "" | vLLM backend to use |
| `vllm_args` | `--vllm-args` | ARGS | "" | Custom arguments to pass to vllm-server |

#### `whispercpp` — Whisper.cpp

| Option | CLI flag | Type | Default | Description |
|--------|----------|------|---------|-------------|
| `whispercpp_backend` | `--whispercpp` | BACKEND | "" | WhisperCpp backend to use |
| `whispercpp_args` | `--whispercpp-args` | ARGS | "" | Custom arguments to pass to whisper-server |
<!-- END GENERATED: backend-options -->

## Implementation notes

### ACE-Step (`acestep`)

`ace-server` exposes an asynchronous job API: `POST /lm` or `POST /synth` returns a job id immediately, `GET /job?id=N` polls the status, and `GET /job?id=N&result=1` fetches the finished result. `AceStepServer::run_job` wraps this submit/poll/fetch cycle with a ceiling of roughly 20 minutes per stage at a 1-second poll cadence. Synth results arrive as `multipart/mixed` (an audio part plus a latent part); Lemonade extracts the first audio part.

Vocals are a two-stage pipeline. `POST /lm` with `lm_mode: "generate"` runs the ACE-Step language model, which turns the caption and lyrics into audio codes plus LM-filled metadata, returned as a JSON array of enriched requests. That array is accepted by `POST /synth` verbatim, so Lemonade feeds it through unchanged. `POST /synth` on its own is the DiT-only instrumental path — it has no language model and cannot sing, so a `lyrics` value (other than the sentinel below) is what routes a request through `/lm` first.

`[Instrumental]` — any case, surrounding whitespace ignored — is ACE-Step's sentinel for the no-vocals path, matching the Python reference implementation. Instrumental requests send the sentinel explicitly rather than an empty string because the synth stage also feeds the lyrics text into its conditioning.

Errors from `audio_generations` are written into the response sink as a JSON error payload; the endpoint handler turns that into an HTTP error instead of shipping it as audio.

The model download fetches the DiT checkpoint variant plus three companions when present in the repo: the language model (`acestep-5Hz-lm-4B-Q8_0.gguf`, required for vocals and auto-lyrics), the Qwen3 text encoder, and the VAE. The checkpoint path handed to `--models` is the directory of GGUFs; `ace-server` scans it by architecture, and `--keep-loaded` keeps models resident across requests.

### Backend auto-selection

When a recipe's backend is not pinned in `config.json` (the `backend` key is absent or `"auto"`), the default backend reported in `system-info` — and used by `RecipeOptions` when resolving `*_backend` options — is chosen as follows: the first supported backend in `RECIPE_DEFS` preference order wins, unless a later supported backend is already locally installed (state `installed`, `update_available`, or `update_required`) while the earlier candidates are merely installable. In that case the first installed one wins. This makes explicitly installing a variant (e.g. the Vulkan build of a GPU backend) an effective override: auto-selection uses what is on disk instead of downloading the preference-order winner. An explicit `backend` value in `config.json` always takes precedence over both rules. The llamacpp `system` variant is never auto-selected unless `prefer_system` is set.
