# Lemonade Server Configuration

## Overview

Lemonade Server starts automatically with the OS after installation. Persistent JSON configuration is stored in Lemonade's config directory; caches and downloaded artifacts stay in the cache directory.

## config.json

If you used an installer from the Lemonade release your `config.json` will be at these locations depending on your OS:

- **Linux — `apt`/`.deb` (Debian/Ubuntu):** `/var/lib/lemonade/config.json`
- **Linux — `dnf`/`.rpm` (Fedora/Red Hat):** `/var/lib/lemonade/config.json`

  > Note: The systemd service runs as the `lemonade` user. Persistent config lives in `/var/lib/lemonade` (systemd StateDirectory), while cached models go to `/var/cache/huggingface` (CacheDirectory). For Debian/Ubuntu, upgrading the package automatically migrates data from the old `/opt/var/lib/lemonade` path to `/var/lib/lemonade`.

- **Windows:** `%USERPROFILE%\.config\lemonade\config.json`
- **macOS:** `/Library/Application Support/lemonade/.config/config.json`

If you are using a standalone `lemond` executable, the default location is `~/.config/lemonade/config.json`.

On startup, Lemonade automatically migrates persistent JSON files from the legacy `.cache` location into the new `.config` location.

> Note: `config.json` is a sparse override file. It only contains settings you explicitly customize; all unspecified settings automatically inherit default values and receive upstream improvements across updates.

### Defaults Precedence & Layering

When `lemond` starts, effective configuration is resolved by deep-merging settings in increasing precedence:

1. **Built-in Defaults**: Factory defaults baked into the release (`resources/defaults.json` and backend descriptors).
2. **Distro / System Defaults**: On Linux, `lemond` merges `/usr/share/lemonade/defaults.json` if it exists, so distro packages can ship system-level defaults (e.g. backend `*_bin` paths pointing at system-installed binaries).
3. **Environment Defaults**: Set the `LEMONADE_DEFAULTS_PATH` environment variable to a `defaults.json` at any location to merge on top (for non-FHS distros like Nix/Guix that cannot write under `/usr/share`).
4. **User Overrides (`config.json`)**: Values explicitly set in your `config.json` override defaults.
5. **CLI Flags**: Arguments passed to `lemond` (e.g. `--port`, `--host`).

### Example config.json

<!-- BEGIN GENERATED: config-example -->
```json
{
  "_generated": "GENERATED by docs/tools/gen_backend_boilerplate.py -- do not hand-edit per-recipe sections (they come from each backend's descriptor config_defaults()). Global keys are hand-maintained in this file. Regenerate and verify with that script; CI --check fails on drift.",
  "acestep": {
    "backend": "auto",
    "cuda_bin": "builtin",
    "rocm_bin": "builtin",
    "vulkan_bin": "builtin"
  },
  "auto_check_model_updates": true,
  "auto_update_models": false,
  "broadcast": true,
  "cloud_providers": [],
  "config_version": 2,
  "ctx_size": -1,
  "default_model_source": "huggingface",
  "disable_model_filtering": false,
  "download_rate_limit": "",
  "ds4": {
    "args": ""
  },
  "enable_dgpu_gtt": false,
  "extra_models_dir": "",
  "flm": {
    "args": "",
    "prefer_system": false
  },
  "global_timeout": 600,
  "host": "localhost",
  "inhibit_suspend": true,
  "kokoro": {
    "cpu_bin": "builtin"
  },
  "llamacpp": {
    "args": "",
    "backend": "auto",
    "cpu_args": "",
    "cpu_bin": "builtin",
    "cuda_bin": "builtin",
    "prefer_system": true,
    "rocm_args": "",
    "rocm_bin": "builtin",
    "vulkan_args": "",
    "vulkan_bin": "builtin"
  },
  "log_level": "info",
  "max_loaded_models": 1,
  "models_dir": "auto",
  "moonshine": {
    "args": "",
    "cpu_args": "",
    "cpu_bin": "builtin"
  },
  "no_fetch_executables": false,
  "offline": false,
  "onnxruntime": {
    "args": "",
    "cpu_args": "",
    "cpu_bin": "builtin"
  },
  "openmoss": {
    "backend": "auto",
    "cuda_bin": "builtin",
    "rocm_bin": "builtin",
    "vulkan_bin": "builtin"
  },
  "port": 13305,
  "rocm_channel": "stable",
  "rocm_install_method": "auto",
  "ryzenai": {
    "server_bin": "builtin"
  },
  "sdcpp": {
    "args": "",
    "backend": "auto",
    "cfg_scale": 7.0,
    "cpu_args": "",
    "cpu_bin": "builtin",
    "cuda_args": "",
    "cuda_bin": "builtin",
    "height": 512,
    "rocm_args": "",
    "rocm_bin": "builtin",
    "steps": 20,
    "vulkan_args": "",
    "vulkan_bin": "builtin",
    "width": 512
  },
  "telemetry": {
    "enabled": false,
    "hide_inputs": false,
    "hide_outputs": false,
    "hide_thinking": false,
    "max_queue_capacity": 1000,
    "otlp": {
      "batch_timeout_s": 1.0,
      "endpoint": "http://localhost:4318/v1/traces",
      "headers": {},
      "max_retries": 0,
      "protocol": "http/protobuf",
      "retry_backoff_base_s": 5.0,
      "semantics": [
        "openinference",
        "otel_genai"
      ],
      "send_batch_size": 100
    },
    "session": {
      "headers": {
        "client": [],
        "id": []
      }
    },
    "trust_incoming_trace_context": false
  },
  "thenoise": {
    "backend": "auto",
    "lora_dir": "",
    "rocm_bin": "builtin",
    "upscaler_dir": ""
  },
  "thinksound": {
    "backend": "auto",
    "cuda_bin": "builtin",
    "rocm_bin": "builtin",
    "vulkan_bin": "builtin"
  },
  "trellis": {
    "args": "",
    "backend": "auto",
    "cuda_bin": "builtin",
    "rocm_bin": "builtin",
    "vulkan_bin": "builtin"
  },
  "vllm": {
    "args": "",
    "backend": "auto"
  },
  "websocket_port": "auto",
  "whispercpp": {
    "args": "",
    "backend": "auto",
    "cpu_args": "",
    "cpu_bin": "builtin",
    "npu_args": "",
    "npu_bin": "builtin"
  }
}
```
<!-- END GENERATED: config-example -->

### Settings Reference

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `port` | int | 13305 | Port number for the HTTP server |
| `host` | string | "localhost" | Address to bind for connections |
| `log_level` | string | "info" | Logging level (trace, debug, info, warning, error, fatal, none) |
| `global_timeout` | int | 600 | Timeout in seconds for HTTP, inference, and readiness checks |
| `max_loaded_models` | int | 1 | Max models per type slot. Use -1 for unlimited |
| `broadcast` | bool | true | Enable or disable UDP broadcasting for server discovery |
| `extra_models_dir` | string | "" | Secondary directory recursively scanned for GGUF model files. Empty disables extra discovery; existing paths must be readable by `lemond` |
| `models_dir` | string | "auto" | Directory for cached model files. `"auto"` follows `HF_HUB_CACHE` / `HF_HOME` / platform default |
| `ctx_size` | int | -1 | Default context size for LLM models. Use `-1` for auto-resolution: the server computes the largest context that fits in available device memory using GGUF architecture metadata. Use a positive integer to set an explicit size. |
| `default_model_source` | string | "huggingface" | Remote registry used to pull checkpoints when a request does not name one (`huggingface` or `modelscope`). Explicit `--source`, a `source`/`registry_source` field, or a provider URL always overrides it. |
| `download_rate_limit` | string | "" | Caps model/backend download speed. Byte rate with curl-style suffixes: `512`, `100K`, `10M`, etc. Use `""` for unlimited download speed. A non-empty value also serializes concurrent transfers. |
| `offline` | bool | false | Skip model downloads |
| `auto_check_model_updates` | bool | true | Check downloaded Hugging Face-backed models for updates during server startup. Set to `false` to check only with `lemonade check-updates` or `POST /v1/models/check-updates`. Manual downloads and updates remain enabled. |
| `auto_update_models` | bool | false | Automatically download model updates when available. Can be overridden per model via `auto_update` in model options. |
| `no_fetch_executables` | bool | false | Prevent downloading backend executable artifacts; backends must already be installed or use the system backend |
| `disable_model_filtering` | bool | false | Show all models regardless of hardware capabilities |
| `inhibit_suspend` | bool | true | Prevent the OS from suspending while inference is active. Linux only (uses systemd-logind); no-op on Windows/macOS/non-systemd environments. |
| `enable_dgpu_gtt` | bool | false | Include GTT for hardware-based model filtering |
| `rocm_channel` | string | "stable" | ROCm backend channel: "stable" (default) or "nightly". See [llama.cpp Backend](./llamacpp.md) for details |
| `rocm_install_method` | string | "auto" | How to install the bundled ROCm runtime: "auto" (pip wheels, tarball fallback), "wheel" (wheels only), or "tarball" (no Python/pip). See [llama.cpp Backend](./llamacpp.md#choosing-the-rocm-install-method) for details |

Both `models_dir` and `extra_models_dir` can be changed at runtime through `POST /internal/set`. Existing `extra_models_dir` paths are preflighted as directories and must be enumerable by the `lemond` process. Nonexistent paths are accepted so the directory watcher can observe them if they are created later.

### Backend Configuration

Backend-specific settings are nested under their backend name:

**llamacpp** — LLM inference via llama.cpp:
| Key | Default | Description |
|-----|---------|-------------|
| `backend` | "auto" | Backend to use: "auto" means "choose for me" |
| `args` | "" | Custom arguments to pass to llama-server (fallback, unused when backend-specific args defined) |
| `*_args` | "" | Backend-specific custom arguments to pass to llama-server |
| `device` | "" | Comma-separated list of devices to use for offloading. Empty is auto. |
| `prefer_system` | false | Prefer system-installed llama.cpp over bundled |
| `*_bin` | "builtin" | Backend binary selection — see [Backend binary selection](#backend-binary-selection) |

**whispercpp** — Audio transcription:
| Key | Default | Description |
|-----|---------|-------------|
| `backend` | "auto" | Backend to use: "auto" means "choose for me" |
| `args` | "" | Custom arguments to pass to whisper-server (fallback, unused when backend-specific args defined) |
| `*_args` | "" | Backend-specific custom arguments to pass to whisper-server |
| `*_bin` | "builtin" | Backend binary selection — see [Backend binary selection](#backend-binary-selection) |

**sdcpp** — Image generation:
| Key | Default | Description |
|-----|---------|-------------|
| `backend` | "auto" | Backend to use: "auto" means "choose for me" |
| `args` | "" | Custom arguments to pass to `sd-server` (fallback, unused when backend-specific args defined) |
| `*_args` | "" | Backend-specific custom arguments to pass to `sd-server` |
| `steps` | 20 | Number of inference steps |
| `cfg_scale` | 7.0 | Classifier-free guidance scale |
| `width` | 512 | Image width in pixels |
| `height` | 512 | Image height in pixels |
| `*_bin` | "builtin" | Backend binary selection — see [Backend binary selection](#backend-binary-selection) |

**flm** — FastFlowLM NPU inference:
| Key | Default | Description |
|-----|---------|-------------|
| `args` | "" | Custom arguments to pass to flm serve |
| `prefer_system` | false | Prefer system-installed FLM (on PATH) over auto-installed portable binary |

**ryzenai** — RyzenAI NPU inference:
| Key | Default | Description |
|-----|---------|-------------|
| `server_bin` | "builtin" | Backend binary selection — see [Backend binary selection](#backend-binary-selection) |

**kokoro** — Text-to-speech:
| Key | Default | Description |
|-----|---------|-------------|
| `cpu_bin` | "builtin" | Backend binary selection — see [Backend binary selection](#backend-binary-selection) |

**cloud_providers** — Cloud OpenAI-compatible providers (see [Cloud Offload](./cloud.md)). Array, one object per installed provider:

| Key | Description |
|-----|-------------|
| `name` | Short identifier (e.g. `fireworks`). Used as the model-name prefix. |
| `base_url` | OpenAI-compatible base URL ending in `/v1` (or equivalent). |
| `allow_insecure_http` | Whether this provider may receive its API key over `http://`. |
| `auth_header_name` | Header the API key is sent in. Omitted when it is the default `Authorization`. |
| `auth_header_prefix` | Value prefix before the key. Omitted when it is the default `"Bearer "`. |
| `wire_format` | `openai` or `anthropic`. Omitted when it is the default `openai`. |

API keys for these providers are **not** stored in `config.json` — they live in `LEMONADE_<PROVIDER>_API_KEY` env vars (persistent) or `lemond` process memory via `POST /v1/cloud/auth` (ephemeral). Manage providers with `lemonade cloud install/uninstall/auth/list` rather than editing this section by hand.

**telemetry** — Unified telemetry and tracing configurations:

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `enabled` | bool | false | Enable or disable telemetry tracing. |
| `hide_inputs` | bool | false | Redact prompt message content from spans. |
| `hide_outputs` | bool | false | Redact generated assistant message content from spans. |
| `hide_thinking` | bool | false | Redact reasoning/thought content from spans. |
| `trust_incoming_trace_context` | bool | false | Honor a caller-supplied W3C `traceparent` header so inference spans join the caller's distributed trace instead of starting a fresh root. Opt-in because span parentage then depends on client-supplied input. |
| `max_queue_capacity` | int | 1000 | The maximum capacity of the in-memory telemetry queue buffer. Oldest spans are dropped when full. Must be `> 0`. |
| `otlp` | object | (nested object) | Sub-block grouping OTLP transport details (see below). |

**telemetry.otlp** — Nested OTLP settings:

| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `endpoint` | string | "http://localhost:4318/v1/traces" | The OTLP endpoint to send traces to. |
| `protocol` | string | "http/protobuf" | Supported OTLP trace protocol: `"http/protobuf"` or `"http/json"`. |
| `semantics` | array of strings | ["openinference", "otel_genai"] | Active trace semantics. Supported values: `"openinference"` and `"otel_genai"`. |
| `headers` | object | {} | Map of custom HTTP headers to pass to the OTLP receiver. |
| `max_retries` | int | 0 | Maximum number of retry attempts for failed exports. Set to `0` to disable retries and discard failed spans immediately. Must be `>= 0`. |
| `retry_backoff_base_s` | double | 5.0 | Base delay in seconds for exponential backoff retries. Must be `>= 0`. |
| `send_batch_size` | int | 100 | Target maximum number of spans to group in a single batched OTLP request. Must be `>= 1`. |
| `batch_timeout_s` | double | 1.0 | Maximum time to wait in seconds before exporting a partially filled batch of spans. Must be `> 0`. |

#### Telemetry and Tracing Details

Lemonade uses a unified telemetry subsystem to trace requests and capture critical execution spans. The following technical behaviors apply:

- **Multi-Standard Semantic Conventions**: Supports exporting traces using two co-existing semantics:
  - **OpenInference**: Uses Arize Phoenix-compatible properties (always prefixed with `openinference.span.kind`, `llm.model_name`, `llm.token_count.*`).
  - **OpenTelemetry GenAI**: Uses standard OpenTelemetry GenAI properties (`gen_ai.system`, `gen_ai.request.model`, `gen_ai.usage.input_tokens`, `gen_ai.input.messages`, `gen_ai.output.messages`).
  When both semantics are specified in `telemetry.otlp.semantics`, trace spans carry attributes for both conventions in a single network payload. This allows the collector to parse either convention without duplicate network requests.
- **Dynamic Attribute Prefixing**: Span attributes are dynamically prefixed based on the query type to simplify filtering:
  - `llm.*` for standard chat and completion spans.
  - `embedding.*` for text embedding generation spans.
  - `reranker.*` for document reranking spans.
- **Token Tracking**: Captures and reports token usage metrics using semantic attributes depending on the enabled semantics:
  - For **OpenInference**: Token count is prefixed with `llm.token_count` across all span kinds (`llm.token_count.prompt`, `llm.token_count.completion`, `llm.token_count.total`) alongside legacy keys like `llm.usage.prompt_tokens`.
  - For **OpenTelemetry GenAI**: Token count uses standard fields like `gen_ai.usage.input_tokens` and `gen_ai.usage.output_tokens`.
- **Calculated Performance Metrics**: In streaming mode, the server automatically computes and records throughput (`llm.performance.tokens_per_second` / `gen_ai.usage.tokens_per_second` depending on semantic conventions) and prefill latency (`llm.performance.time_to_first_token` / `gen_ai.performance.time_to_first_token`) if not natively returned by the backend (e.g., for vLLM and Cloud models).
- **vLLM Engine Telemetry**: For the vLLM backend, the server queries the local `/metrics` endpoint on completion to attach scheduler queue metrics (`llm.vllm.num_requests_waiting`, `llm.vllm.num_requests_running`, `llm.vllm.num_requests_swapped`) and KV cache utilization (`llm.vllm.gpu_cache_usage_factor`, `llm.vllm.cpu_cache_usage_factor`) directly to the trace spans.
- **Reasoning Model Support**: For reasoning models (e.g., DeepSeek models), the server extracts and records `reasoning_content` from the assistant's generation. Any variant thought-termination tags (e.g., `</think|>`) are automatically standardized to the canonical `</think>` tag.
- **Exporter Retry Backoff**: When retries are enabled (i.e., `max_retries > 0`), the exporter uses an exponential backoff strategy combined with randomized jitter for failed posts. The base retry interval starts at `retry_backoff_base_s` seconds (defaulting to 5), doubling on each subsequent failure (e.g., 5s, 10s, 20s, 40s), up to a maximum cap of 60 seconds. A randomized jitter factor between `0.5` and `1.5` is applied to each calculated delay to prevent a "thundering herd" when the collector recovers. Permanent client errors (`4xx` HTTP status codes, excluding `429 Too Many Requests`) are classified as non-retryable and cause the batch to be dropped immediately to save resources.
- **OTLP Trace Batching**: Spans are aggregated in an in-memory queue buffer and exported in batches to minimize network overhead and maximize compression efficiency. Batching operates on a dual-trigger system: a batch is immediately serialized and dispatched if it reaches `send_batch_size` (default: `100`), or if `batch_timeout_s` (default: `1.0` second) has elapsed since the oldest span in the batch arrived. All remaining traces are flushed cleanly to the OTel collector upon server shutdown. Users can also trigger a manual flush at any time via the `POST /internal/telemetry/flush` endpoint.
- **Request Failure Tracing**: Captures request failures directly on the telemetry spans. If a model fails to load, a request is rejected by the router, or a streaming connection encounters an exception or a non-200 HTTP status code from the backend, the span is ended with `Error` status and the specific error message is attached.
- **Queue Blocking & Thundering Herd Prevention**: To prevent client requests from hanging and to avoid exhausting resources when the telemetry receiver endpoint is down, Lemonade employs a fail-fast mechanism. The exporter memory buffer is strictly bounded to a capacity of `max_queue_capacity` spans (default: `1000`). When full, a head-drop (FIFO) eviction policy is applied to drop the oldest telemetry spans to make room for newer ones, prioritizing current application state. If a telemetry transmission task fails all of its retries and is dropped, the endpoint is marked as **unreachable**. While in this unreachable state, subsequent spans in the transmission queue are attempted only once and immediately dropped without backoff delay if they fail, preventing the telemetry queue from blocking server operations. A single successful span delivery to the endpoint automatically resets the unreachable state and restores normal retry behavior.

### Backend binary selection

Every `*_bin` key (e.g. `llamacpp.vulkan_bin`, `whispercpp.cpu_bin`, `sdcpp.rocm_bin`) accepts the same set of values:

| Value | Meaning |
|---|---|
| `"builtin"` *(default)* | Use the version of the upstream backend that lemonade pins in its release. Recommended for most users — these versions are tested with this lemonade build. |
| `""` | Same as `"builtin"`. |
| `"latest"` | Resolve to the most-recent upstream GitHub release on first install or first status query for that backend, then install on demand. The resolved tag is recorded in `<lemonade-home>/bin/<recipe>/<backend>/version.txt`. |
| `"b8664"` / `"v1.8.2"` / etc. | A specific upstream release tag. Lemonade downloads that exact version from GitHub. |
| `"/path/to/bin"` | A directory you populated yourself (e.g. a local build). Lemonade uses the executable inside this directory and never downloads. The path must exist when set. |

> Note: the `latest` setting is experimental.

> Note: `llamacpp.rocm_bin` version tags are channel-specific. Each ROCm channel downloads from a different GitHub repository, so you must set the correct `rocm_channel` before pinning `rocm_bin` to a specific tag. See [Pinning to a Specific Version Tag](./llamacpp.md#pinning-to-a-specific-version-tag) for details.

Examples:

```bash
# Track upstream llama.cpp Vulkan releases (auto-resolve at lemond start)
lemonade config set llamacpp.vulkan_bin=latest

# Pin to a specific llama.cpp build
lemonade config set llamacpp.vulkan_bin=b8664

# Use your own llama.cpp build
lemonade config set llamacpp.vulkan_bin=/home/me/llama.cpp/build/bin

# Revert to the version lemonade ships
lemonade config set llamacpp.vulkan_bin=builtin
```

#### Behavior when `*_bin` changes

Changing a `*_bin` value applies live: lemonade unloads any model currently using that backend, downloads the new binary if needed, and reloads the model on the new binary. No `lemond` restart is required.

#### `latest` re-resolution

`"latest"` is resolved once per `lemond` process. The first install or status query for a `latest`-pinned backend hits the GitHub API; the resolved tag is then cached in memory for the rest of the process lifetime. Subsequent installs and status queries (including manual `lemonade backends install`) reuse the cached tag and do not re-query GitHub. **Restart `lemond` to pick up a newer upstream release.**

#### Upgrade signals in `lemonade backends`

The `lemonade backends` listing surfaces two upgrade signals for backends pinned to `"latest"`:

- **`update_available`** — A newer upstream release exists than what's installed. The backend keeps running on the installed version; the listed `action` is the install command to apply the upgrade when you're ready.
- **`update_required`** — The installed version is *older* than the version lemonade ships in this release. This forces an upgrade prompt because running below the lemonade-shipped baseline is not supported.

Backends pinned to a specific tag (e.g. `b8664`) do not get either signal — they're treated as an explicit user choice.

#### Interactions with other config

- `offline: true` blocks the GitHub call for `"latest"`. If a previously-installed `version.txt` exists in the install directory, lemonade reuses that version with a warning. Otherwise the install fails.
- `no_fetch_executables: true` blocks all downloads, including resolving and installing `"latest"` and any version-tag pin. Existing installs continue to work.

## Editing Configuration

### lemonade config (recommended)

Use the `lemonade config` CLI to view and modify settings while the server is running. Changes are applied immediately and persisted to config.json.

```bash
# View all current settings
lemonade config

# Set one or more values
lemonade config set key=value [key=value ...]
```

Top-level settings use their JSON key name directly. Nested backend settings use dot notation (`section.key=value`):

```bash
# Change the server port and log level
lemonade config set port=9000 log_level=debug

# Change a backend setting
lemonade config set llamacpp.backend=rocm

# Set multiple values at once
lemonade config set port=9000 llamacpp.backend=rocm sdcpp.steps=30

# Set complex types like JSON arrays and objects (use single quotes to prevent shell expansion)
lemonade config set telemetry.otlp.semantics='["openinference"]'
lemonade config set telemetry.otlp.headers='{"Authorization":"Bearer key"}'
```

### lemond CLI arguments (runtime overrides)

`lemond` accepts `--port` and `--host` as CLI arguments to provide ephemeral runtime overrides for that specific server process without modifying `config.json`:

```bash
lemond --port 9000 --host 0.0.0.0
```

Because CLI overrides are ephemeral and do not mutate `config.json` on disk, if the server cannot start due to an invalid port or host setting in `config.json`:
1. Start `lemond` using the CLI override: `lemond --port 9000`
2. Run `lemonade config set port=9000` against the running server to persist the fix for subsequent restarts.
3. Alternatively, edit `config.json` directly as shown below.

### Edit config.json manually (last resort)

If the server won't start and CLI tools aren't sufficient, you can edit config.json directly. Restart the server after making changes:

```bash
# Linux (Debian/Ubuntu and Fedora/Red Hat)
sudo nano /var/lib/lemonade/config.json

sudo systemctl restart lemond

# Windows — edit with your preferred text editor:
# %USERPROFILE%\.config\lemonade\config.json
# Then quit and relaunch from the Start Menu
```

## lemond CLI

```
lemond [cache_dir] [config_dir] [--port PORT] [--host HOST] [--broadcast] [--no-broadcast]
```

- **cache_dir** — Path to the lemonade cache/data directory. Optional; defaults to the platform-specific cache location.
- **config_dir** — Path to the lemonade config directory for persistent JSON state. Optional; defaults to the platform-specific config location.
- **--port** — Port to serve on (runtime override, does not mutate config.json).
- **--host** — Address to bind (runtime override, does not mutate config.json).
- **--broadcast** / **--no-broadcast** — Enable or disable UDP broadcasting for server discovery (non-persistent override).

## API Key and Security

### Regular API Key

The `LEMONADE_API_KEY` environment variable sets an API key for authentication on regular API endpoints (`/api/*`, `/v0/*`, `/v1/*`). On Linux with systemd, set it in the service environment (e.g., via a systemd override or drop-in file). On Windows, set it as a system environment variable.

When `LEMONADE_API_KEY` is set, the inference and model-management endpoints reject any request that does not present a matching Bearer token. This is the only credential that gates those endpoints, so it controls whether unauthenticated clients can reach the server at all. When it is unset, those endpoints are reachable without authentication.

### Admin API Key

The `LEMONADE_ADMIN_API_KEY` environment variable provides elevated access to both regular API endpoints and internal endpoints (`/internal/*`). When set, it takes precedence over `LEMONADE_API_KEY` for client authentication.

`LEMONADE_ADMIN_API_KEY` enables privilege separation between two classes of authenticated clients. Holders of `LEMONADE_API_KEY` can reach the regular API endpoints, while only holders of `LEMONADE_ADMIN_API_KEY` can reach the internal control endpoints (`/internal/*`, e.g. shutdown and configuration). A client presenting only `LEMONADE_API_KEY` cannot reach `/internal/*` if `LEMONADE_ADMIN_API_KEY` is set to a distinct value. If `LEMONADE_ADMIN_API_KEY` is not set, it defaults to the value of `LEMONADE_API_KEY`, so the regular key then also authenticates against `/internal/*` and no privilege separation exists.

**Authentication Hierarchy:**

| Scenario | `LEMONADE_API_KEY` | `LEMONADE_ADMIN_API_KEY` | Internal Endpoints | Regular API Endpoints |
|----------|-------------------|--------------------------|-------------------|----------------------|
| No keys set | (not set) | (not set) | No auth required | No auth required |
| Only API key | "secret" | (not set) | Requires key | Requires key |
| Only admin key | (not set) | "admin" | Requires admin key | No auth required |
| Both keys different | "regular" | "admin" | Requires admin key | Either key accepted |

**Client Behavior:** Clients (CLI, tray app) automatically prefer `LEMONADE_ADMIN_API_KEY` if set, otherwise fall back to `LEMONADE_API_KEY`.

### Allowed Origins

The `LEMONADE_ALLOWED_ORIGINS` environment variable controls which remote web origins are authorized to connect to the server (specifically for CORS headers on HTTP endpoints and origin validation on WebSocket connections).

> [!NOTE]
> `LEMONADE_ALLOWED_ORIGINS` specifies the **client application/web page's origin** (where the request originates), **not** the target Lemonade server URL. Non-browser HTTP clients (such as CLI tools, cURL, or server-side SDKs) do not send an `Origin` header and are not restricted by origin validation.

- **Format**: A comma-separated list of complete origins including the scheme and optional port (e.g., `https://app.lemonade.dev,http://localhost:3000`).
  > [!WARNING]
  > Allowing a non-local plain-HTTP origin (e.g., `http://app.example.com`) is vulnerable to on-path modification (man-in-the-middle) and interception. It is highly recommended to use HTTPS (`https://`) for all remote/non-local allowed origins.
- **Wildcard (`*`)**: Setting the variable to `*` allows any origin to connect.
- **Security Implications of `*`**:
  > [!WARNING]
  > Using `LEMONADE_ALLOWED_ORIGINS=*` permits any website running in a user's browser to make requests to your local Lemonade server. In particular, if `LEMONADE_API_KEY` is not configured, this exposes the server to unauthenticated remote access and cross-origin attacks from malicious websites. Use wildcards only for development or in secure, isolated environments.
- **Local/Loopback & Desktop Access**: Loopback addresses (`localhost`, `127.0.0.1`, `[::1]`, `*.localhost`) and custom desktop application schemes (`file://`, `app://.`, `vscode-webview://`, `jan://`, etc.) are permitted for local client connections. Opaque `null` origins (e.g. from sandboxed browser iframes) are rejected unless explicitly listed in `LEMONADE_ALLOWED_ORIGINS` to prevent CSWSH attacks.

## Model Synchronization & Auto-Updates

Lemonade supports manual model synchronization via `lemonade update-models` as well as automatic background updates.


### Configuration
- **Global Config**: Set `auto_update_models: true` in server configuration or via `lemonade config set auto_update_models=true`.
- **Per-Model Override**: Include `"auto_update": true` or `"auto_update": false` in custom model recipes (`user.*`) to override the global default on a per-model basis.
- **Administrative Authorization**: Because model synchronization endpoints are administrative (`/internal/*`), manual synchronization via the CLI or direct API requests requires the appropriate admin credentials (`LEMONADE_ADMIN_API_KEY`, falling back to `LEMONADE_API_KEY` if the admin key is not explicitly configured) when server-side authentication is enabled.

### Risks of Automatic Model Updates

> [!WARNING]
> Enabling automatic updates (`auto_update_models: true` or per-model `"auto_update": true`) introduces operational risks that should be carefully considered:
>
> 1. **Output & Behavioral Drift**: Upstream Hugging Face or ModelScope repositories can release updated weights, revised tokenizers, or altered chat templates. These changes can alter model responses, reasoning characteristics, and prompt compatibility.
> 2. **High Bandwidth & Disk Consumption**: Model updates involve downloading multi-gigabyte file revisions (e.g. GGUF quantizations). Unattended updates can unexpectedly consume substantial network bandwidth and disk space.
> 3. **Startup & Inference Delays**: Automatic update downloads trigger on server startup when upstream revisions are detected, delaying server readiness and inference availability.
> 4. **Quantization & Recipe Shifts**: Remote repository changes may alter file naming or tensor layouts, requiring updated recipe configurations.
>
> **Recommendation**: Leave `auto_update_models` set to `false` (default) in production environments. Use manual sync (`lemonade update-models --check` followed by `lemonade update-models`) to inspect and control model updates.


## Remote Server Connection

To make Lemonade Server accessible from other machines on your network, set the host to `0.0.0.0`:

```bash
lemonade config set host=0.0.0.0
```

> **Warning:** Using `host: "0.0.0.0"` allows connections from any machine on the network — including to the internal control endpoints (`/internal/*`, e.g. shutdown and config). Only do this on trusted networks, and set an API key to manage access. `LEMONADE_API_KEY` secures all endpoints; `LEMONADE_ADMIN_API_KEY` on its own secures only `/internal/*` and leaves the inference and model-management endpoints (`/api`, `/v0`, `/v1`) open, so set `LEMONADE_API_KEY` to protect those too. The server logs a warning at startup when bound to a non-loopback host without the regular key.

## Next Steps

The [Server Specification](../../api/README.md) provides more information about how to integrate Lemonade Server into an application.

<!--Copyright (c) 2025 AMD-->
