LLMs on Linux with FastFlowLM - Lemonade Server
Description: LLMs on Linux are now supported with FastFlowLM. This guide helps you set up the FLM stack on supported AMD XDNA 2 NPUs and troubleshoot common validation issues.
Date: 2026-03-11

Linux NPU Support

# LLMs on Linux with FastFlowLM

This article will teach you how to run LLMs on your AMD XDNA 2 NPU on Linux using FastFlowLM. Get set up and then show us what you build!
Date:March 11, 2026
Authors:Lemonade and FastFlowLM contributors
Getting Started

## Troubleshooting

Lemonade detected that some additional setup might be needed to get FastFlowLM fully operational on this system. Please review this guide and follow the steps provided.

Announcement

## Updates for Linux Support

Today, it is possible to run LLMs and Whisper on the AMD XDNA 2 NPU. This solution is made up of:

- Upstream NPU driver in the Linux 7.0+ kernel (with backports for 6.xx kernels).
- AMD IRON compiler for XDNA NPUs.
- FastFlowLM, a lightweight LLM runtime for AMD NPUs, which today has added Linux support.
- Lemonade, ties everything together with a streamlined user experience.

Getting Started

## FastFlowLM

FastFlowLM is a lightweight LLM runtime optimized for AMD NPUs. Today, FastFlowLM is adding support for Ubuntu, Arch, and other distros to enable fast, low-power LLMs on Ryzen™ AI PCs that run Linux.

This article will help you:

- Understand Linux NPU support status and required platform versions
- Install the FLM + driver stack for your distribution
- Validate your setup with `flm validate`
- Fix common firmware, driver, and memlock issues

HW Requirements

## Supported processors

FastFlowLM on Linux requires an AMD XDNA 2 NPU.

Ryzen AI family | Codename | Status
--- | --- | ---
Max 300-series | Strix Halo | Supported
300-series | Kraken Point, Strix Point | Supported
400-series | Gorgon Point | Supported
Z2 Extreme | Handheld devices | Supported

Note: Ryzen AI 7000/8000/200-series chips have XDNA 1, which is not supported.

SW Requirements

## Runtime stack

This solution requires specific firmware, kernel version, driver, and runtime software to function. The quickstart guide below will help you install these requirements.

Item | Requirement
--- | ---
NPU firmware | Version 1.1.0.0 or later
Kernel + driver | Kernel 7.0+ with amdxdna, or amdxdna-dkms
Runtime | FastFlowLM (optional — auto-installed by Lemonade if not in PATH)
Memlock limit | Must be high enough for NPU execution

Quickstart

## Setup by distribution

Select your Linux distribution and follow the exact install path.

Select your Linux distribution:
For Ubuntu 24.04

1. Add the PPA:
   `sudo add-apt-repository ppa:lemonade-team/stable
   sudo apt update`
   See lemonade-team/stable for details.
2. Install packages:
   `sudo apt install libxrt-npu2 amdxdna-dkms`
3. Reboot:
   `sudo reboot`
4. Install FLM from upstream releases, or let Lemonade auto-install it:
   `sudo apt install ./<flm_package.deb>`

For Ubuntu 25.10

1. Add the PPA:
   `sudo add-apt-repository ppa:lemonade-team/stable
   sudo apt update`
   See lemonade-team/stable for details.
2. Install packages:
   `sudo apt install libxrt-npu2 amdxdna-dkms`
3. Reboot:
   `sudo reboot`
4. Install FLM from upstream releases, or let Lemonade auto-install it:
   `sudo apt install ./<flm_package.deb>`

For Ubuntu 26.04

1. Optionally install FLM from upstream releases:
   `sudo apt install ./<flm_package.deb>`
   Lemonade will auto-install the portable FLM binary if not already available.

For Debian 13

1. Enable backports by creating `/etc/apt/sources.list.d/debian-backports.sources`:
   `sudo tee /etc/apt/sources.list.d/debian-backports.sources <<EOF
   Types: deb
   URIs: http://deb.debian.org/debian
   Suites: trixie-backports
   Components: main
   Enabled: yes
   Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
   EOF
   sudo apt update`
   Backports is needed so apt can automatically satisfy FLM's runtime/driver dependencies. See backports.debian.org for details.
2. Optionally install FLM from upstream releases:
   `sudo apt install ./<flm_package.deb>`
   Lemonade will auto-install the portable FLM binary if not in PATH.

For Arch Linux

1. Install XRT:
   `sudo pacman -S xrt`
2. Install `xrt-plugin-amdxdna`:
   `sudo pacman -S xrt-plugin-amdxdna`
3. Update to kernel 7.0-rc2 or later:
   `sudo pacman -S linux`
4. For older kernels (6.18, 6.19), use AUR:
   `paru -S amdxdna-dkms`
5. Update firmware package:
   `sudo pacman -S linux-firmware`
6. Install FastFlowLM:
   `sudo pacman -S fastflowlm`

For Other Distributions

1. Build and install XRT and the AMD XDNA plugin from source: amd/xdna-driver.
2. Update to kernel 7.0-rc2 or later and update `linux-firmware` to 20260221 or later.
3. Clone FastFlowLM:
   `git clone https://github.com/ROCm/FastFlowLM.git`
4. Build and install FLM:
   `cmake -S src --preset linux-default
   ninja -C src/build
   sudo ninja -C src/build install`

Troubleshooting

## Common issues

If setup does not work, use these checks to isolate the failing layer quickly.

The first thing you should do is run `flm validate` to check your system's compatibility and identify any issues. A successful validation will look like this:
❮ flm validate
[Linux] Kernel: 7.0.0-rc1-00052-g27936bfca73d
[Linux] NPU: /dev/accel/accel0
[Linux] NPU FW Version: 1.1.2.64
[Linux] Memlock Limit: infinity

### 1. Outdated NPU firmware

If `flm validate` reports firmware issues, ensure version 1.1.0.0 or later.

`cat /sys/bus/pci/drivers/amdxdna/*/fw_version`

If older, update through your distribution (typically by updating the `linux-firmware` package).

### 2. Missing or outdated amdxdna driver

This solution requires `amdxdna`, provided in Linux kernel 7.0+ or via `amdxdna-dkms`.

`lsmod | grep amdxdna`

If not listed, upgrade to a kernel with in-tree support or install the DKMS package.

`uname -r`

Kernel version matters mainly for in-tree driver support. With DKMS, package version determines the driver.

### 3. Memlock limit too low

NPU workloads need memory locking. If you see Memlock limit is too low (8MB), increase limits.

Most users do not need manual changes when running Lemonade as a service: Lemonade's systemd unit sets `LimitMEMLOCK=infinity`.

If you run FLM outside the service, edit limits manually:

`sudo nano /etc/security/limits.conf`

    *    soft    memlock    unlimited
    *    hard    memlock    unlimited

Log out and log back in for the change to apply.

### 4. AMD IOMMU disabled

If AMD IOMMU is disabled on your system, your NPU will not be detected:

`$ flm validate [Linux] Kernel: 7.0.0-rc2 [ERROR] No NPU device found. [Linux] Memlock Limit: set to infinity`

Make sure `amd_iommu=off` is not part of your kernel command line. This is a common setting to change on Strix Halo machines as it slighlty increases GPU performance in llama.cpp, but unfortunately disables the NPU.

Resources

## Additional resources

- FastFlowLM GitHub repository
- Lemonade GitHub issues
- Lemonade Discord

## Ready to enable Linux NPU LLMs?

Install Lemonade, set up the dependencies described above, then start chatting and building!

Install Lemonade