# Cross-Vendor Command Stream Translation — Design Document

**Status:** Design ✅ | **Implementation:** Not started  
**Last updated:** 2026-05-24  
**Priority:** High (next major production gap)

---

## 1. Problem Statement

MVGAL's current interception layers (Vulkan, OpenCL, CUDA, D3D, Metal) can
intercept API calls and re-route workloads to different GPUs at the *API level*.
However, they cannot translate a **command buffer** written for one GPU
architecture into commands executable on another.

For example:
- An application submits a Vulkan command buffer containing AMD-specific
  pipeline barriers and descriptor layouts.
- MVGAL routes it to an NVIDIA GPU.
- The NVIDIA driver does not understand AMD-specific commands.
- The application crashes or renders incorrectly.

The same problem exists in reverse (NVIDIA→AMD), and for OpenCL kernels,
CUDA kernels (NVIDIA→AMD), and D3D shaders routed through Wine/Proton.

### Current Workaround

MVGAL's existing "transparent" interception works by:
1. **Replicating work**: In AFR mode, each GPU renders complete frames
   independently — no cross-vendor command translation needed.
2. **Single-GPU dispatch**: Route the entire workload to one GPU.
3. **Host-side split**: Split the workload before GPU submission (e.g.,
   task-based scheduling offloads compute to one GPU, graphics to another).

These workarounds avoid the translation problem but limit aggregation
efficiency.

### Production Requirement

To achieve true transparent heterogeneous aggregation, MVGAL needs a
**cross-vendor command stream translator** that converts GPU commands
between architectures at the kernel driver or userspace driver level.

---

## 2. Architecture

### 2.1 Design Decision: Userspace Translation Layer

Inter-architecture command translation belongs in **userspace**, not the
kernel. Rationale:

1. **Complexity**: GPU command formats are poorly documented and change
   frequently. Userspace is easier to update and debug.
2. **Safety**: A translation bug in the kernel can crash the entire system.
   In userspace, it crashes only the application.
3. **Vendor NDAs**: GPU command formats are often proprietary. A userspace
   layer can work with closed-source user-mode drivers without GPL
   contamination.
4. **Performance**: Translation overhead is dominated by the translation
   logic, not kernel vs. userspace dispatch.

### 2.2 Layer Placement

```
Application (Vulkan/OpenCL/CUDA)
       │
       ▼
┌──────────────────────────────┐
│  MVGAL API Interception      │  ← intercepts API calls
│  (Vulkan Layer / CUDA Shim)  │
└──────────┬───────────────────┘
           │
           ▼
┌──────────────────────────────┐
│  MVGAL Command Stream        │  ← NEW: translates between GPU ISAs
│  Translation Layer           │
│                              │
│  ┌────────────────────────┐  │
│  │  ISA Descriptor DB     │  │  ← maps GPU arch → capabilities
│  └────────────────────────┘  │
│  ┌────────────────────────┐  │
│  │  Pattern Matcher       │  │  ← recognizes cmd patterns
│  └────────────────────────┘  │
│  ┌────────────────────────┐  │
│  │  Emitter               │  │  ← emits native commands
│  └────────────────────────┘  │
└──────────┬───────────────────┘
           │
     ┌─────┴─────┐
     ▼           ▼
  amdgpu.ko   nvidia.ko   ...
```

### 2.3 Abstraction Model

The translation layer uses a **three-level abstraction**:

```
Level 3: API Semantics
  ─ Draw call: vkCmdDraw, vkCmdDispatch
  ─ Resource barrier: vkCmdPipelineBarrier
  ─ Descriptor: vkCmdBindDescriptorSets

Level 2: GPU ISA Primitives
  ─ ALU operations: ADD, MAD, MUL, FMA
  ─ Memory operations: LOAD, STORE, ATOMIC
  ─ Control flow: BRANCH, CALL, BARRIER
  ─ Texture operations: SAMPLE, GATHER, LOD

Level 1: Vendor-Specific Encoding
  ─ AMD: PAL command buffer format, RDNA ISA
  ─ NVIDIA: ChID method submission, Maxwell+ ISA
  ─ Intel: Gen/Xe command streamer, GPGPU ISA
  ─ MTT: Chunxiao command format (inferred from mtgpu-drv)
```

Translation happens at Level 2: each GPU ISA primitive has a canonical
representation. The pattern matcher converts Level 3 API semantics into
Level 2 primitives, and the emitter converts Level 2 primitives into
Level 1 vendor-specific encoding.

---

## 3. Translation Strategies

### 3.1 Full Translation (High Fidelity)

The ideal approach: translate every command with 100% semantic accuracy.

```
Input: NVIDIA ChID methods
  → Pattern matcher recognizes: "BARRIER_GL + TEX_SAMPLE_2D + ALU_MAD"
  → Canonical L2: {type: BARRIER, scope: "device", ALU: [MAD, MAD, ADD], TEX: [SAMPLE_2D]}
  → AMD emitter: PAL CmdBarrier() + CmdDraw() with RDNA shader
```

**Challenges:**
- Requires complete ISA documentation for all vendors
- NVIDIA ISA is largely undocumented (reverse-engineering needed)
- MTT ISA is derived from reverse-engineered mtgpu-drv
- Testing requires real hardware pairs (AMD+NVIDIA, Intel+AMD, etc.)

### 3.2 Shader Translation Only (Practical First Step)

Translate only the **shader binaries** (SPIR-V → vendor-specific ISAs).
The command buffer structure (barriers, draw calls, resource binding)
remains unchanged — each GPU driver handles its own native commands.

```
Application → SPIR-V (Vulkan)
  ─ NVIDIA → NVRTC / PTX → SASS (existing)
  ─ AMD → LLVM/AMDGPU → RDNA ISA (existing)
  ─ Intel → IGC → Gen ISA (existing)
  ─ MTT → MTT compiler (proprietary)

MVGAL intercepts at SPIR-V level:
  1. Extract SPIR-V from vkCreateShaderModule
  2. For target GPU, compile SPIR-V → target ISA via vendor compiler
  3. Replace shader module handle before submission
```

**Advantages:**
- Existing vendor compilers handle ISA-specific optimization
- No need to reverse-engineer undocumented command buffer formats
- Works within the Vulkan/OpenCL/CUDA API model
- SPIR-V is a well-documented intermediate representation

**Limitations:**
- Does not translate non-shader commands (barriers, clears, copies)
- Assumes cross-vendor command buffer structures are compatible
- May produce suboptimal code (vendor compilers optimize for their own HW)

### 3.3 Hybrid Approach (Recommended Initial Path)

Start with Strategy 3.2 (shader translation) for immediate value, then
add command buffer translation incrementally:

**Phase 1 — SPIR-V routing (3-4 weeks):**
- Intercept `vkCreateShaderModule` in the Vulkan layer
- Cache SPIR-V → vendor binary translations
- Route shaders to the correct GPU's compiler
- Validate on AMD+NVIDIA pair

**Phase 2 — Barrier translation (4-6 weeks):**
- Translate `vkCmdPipelineBarrier` between vendor formats
- Handle image layout transitions, memory barriers, queue family ownership
- Add `vkCmdClear*` and `vkCmdCopy*` translation

**Phase 3 — Full command buffer capture/replay (8-12 weeks):**
- Capture Vulkan command buffers as a directed acyclic graph (DAG)
- Translate DAG nodes per target GPU architecture
- Replay translated commands on target GPU
- Validate with real games under Proton (DXVK, VKD3D-Proton)

---

## 4. Implementation Plan

### 4.1 Phase 1: SPIR-V Routing

```c
// In mvgal_vulkan_layer.c — new file: mvgal_shader_translate.c

typedef struct {
    VkShaderModule       original_module;
    VkShaderModule       translated_modules[MAX_GPUS];
    size_t               spirv_size;
    uint32_t            *spirv_data;
    VkShaderStageFlagBits stage;
} mvgal_shader_cache_entry_t;

// Hook vkCreateShaderModule
VkResult MVGAL_VKAPI mvgal_CreateShaderModule(
    VkDevice device,
    const VkShaderModuleCreateInfo* pCreateInfo,
    const VkAllocationCallbacks* pAllocator,
    VkShaderModule* pShaderModule)
{
    // 1. Call original vkCreateShaderModule (stores SPIR-V)
    VkResult result = orig_CreateShaderModule(device, pCreateInfo, pAllocator, pShaderModule);

    // 2. Cache the SPIR-V for later translation
    uint32_t gpu_count;
    mvgal_get_physical_device_count(&gpu_count);

    mvgal_shader_cache_entry_t* entry = malloc(sizeof(*entry) + pCreateInfo->codeSize);
    entry->original_module = *pShaderModule;
    entry->spirv_size = pCreateInfo->codeSize;
    memcpy(entry->spirv_data, pCreateInfo->pCode, pCreateInfo->codeSize);
    entry->stage = spirv_get_stage(pCreateInfo->pCode);
    memset(entry->translated_modules, 0, sizeof(entry->translated_modules));

    // 3. Translate for each non-primary GPU
    for (uint32_t i = 0; i < gpu_count; i++) {
        mvgal_gpu_info_t gpu;
        mvgal_get_physical_device_info(i, &gpu);
        if (gpu.vendor_id != primary_gpu_vendor) {
            entry->translated_modules[i] = compile_spirv_for_vendor(
                pCreateInfo->pCode, pCreateInfo->codeSize,
                gpu.vendor_id, stage);
        }
    }

    hashmap_put(shader_cache, *pShaderModule, entry);
    return result;
}
```

**Dependencies:**
- SPIRV-Tools for SPIR-V parsing
- Vulkan SDK (already available) for `VK_KHR_external_memory` and
  cross-process shader module handles
- Vendor compilers: AMD's LLVM backend, NVIDIA's NVRTC, Intel's IGC

### 4.2 Phase 2: Barrier Translation

```c
// Barrier semantics normalization
typedef enum {
    MVGAL_BARRIER_TYPE_MEMORY,
    MVGAL_BARRIER_TYPE_IMAGE,
    MVGAL_BARRIER_TYPE_BUFFER,
    MVGAL_BARRIER_TYPE_EVENT
} mvgal_barrier_type_t;

typedef struct {
    mvgal_barrier_type_t type;
    VkPipelineStageFlags2 src_stage;
    VkPipelineStageFlags2 dst_stage;
    VkAccessFlags2        src_access;
    VkAccessFlags2        dst_access;
    union {
        VkImageMemoryBarrier2  image;
        VkBufferMemoryBarrier2 buffer;
        VkMemoryBarrier2       memory;
    };
    VkDependencyFlags      dependency_flags;
} mvgal_normalized_barrier_t;

// Translate from any vendor to canonical form
mvgal_normalized_barrier_t mvgal_normalize_barrier(
    VkPipelineStageFlags src_stage, VkPipelineStageFlags dst_stage,
    VkAccessFlags src_access, VkAccessFlags dst_access,
    const VkImageMemoryBarrier* barrier);

// Emit barrier in target vendor format
void mvgal_emit_amd_barrier(mvgal_normalized_barrier_t* barrier);
void mvgal_emit_nvidia_barrier(mvgal_normalized_barrier_t* barrier);
void mvgal_emit_intel_barrier(mvgal_normalized_barrier_t* barrier);
void mvgal_emit_mtt_barrier(mvgal_normalized_barrier_t* barrier);
```

### 4.3 Phase 3: Full Command Buffer DAG

```c
// Command buffer represented as a DAG of operations
typedef struct mvgal_cmd_node {
    mvgal_cmd_type_t        type;     // DRAW, DISPATCH, CLEAR, COPY, BARRIER, etc.
    mvgal_normalized_args_t args;     // Canonical arguments
    struct mvgal_cmd_node  *deps;     // Up to N dependencies (sync, semaphores)
    size_t                  dep_count;
    struct mvgal_cmd_node  *next;     // Linked list for ordering
} mvgal_cmd_node_t;

// Capture → Translate → Emit pipeline
mvgal_cmd_dag_t* mvgal_capture_command_buffer(VkCommandBuffer cmd);
mvgal_cmd_dag_t* mvgal_translate_dag(mvgal_cmd_dag_t* dag, uint32_t target_gpu);
VkResult         mvgal_emit_dag(mvgal_cmd_dag_t* dag, VkCommandBuffer target_cmd);
```

---

## 5. Vendor-Specific Integration

### 5.1 AMD (RDNA 1/2/3)

| Component | Integration Point | Notes |
|-----------|------------------|-------|
| PAL | `amdvlk` PAL library | AMD's Platform Abstraction Library |
| LLVM/AMDGPU | `/usr/lib/llvm-*/lib` | SPIR-V → AMDGPU codegen |
| ROCm | `/opt/rocm` | HIP compiler for compute |
| ACO | `ac-*` LLVM backends | Shader compiler used by RADV |

**Translation path:**
- Vulkan → SPIR-V → LLVM/AMDGPU → RDNA ISA
- OpenCL → SPIR-V → ROCm compiler → GCN/RDNA ISA
- Use ACO for faster shader compilation when targeting AMD GPUs

### 5.2 NVIDIA (Turing, Ampere, Ada)

| Component | Integration Point | Notes |
|-----------|------------------|-------|
| CUDA driver | `libcuda.so` | PTX compilation |
| NVRTC | `libnvidia-rtc.so` | Runtime CUDA→PTX compilation |
| nvidia-open | kernel module | ChID submission format |
| Vulkan | `libnvidia-vulkan.so` | Via proprietary driver |

**Translation path:**
- Vulkan → SPIR-V → NVRTC† → PTX → SASS
- CUDA → CUDA C++ → NVCC → PTX → SASS
- †NVRTC compiles CUDA C++, not SPIR-V. Use SPIRV-Cross to generate
  GLSL, then NVIDIA's GLSL→PTX pipeline
- **Alternative**: Use `libnvidia-vulkan.so` / `libGLX_nvidia.so` directly
  for Vulkan/OpenGL translation, bypassing command translation entirely.
  For compute, intercept CUDA API calls and re-target them to the
  destination GPU's compute API.

### 5.3 Intel (Gen 9-12, Xe, Arc)

| Component | Integration Point | Notes |
|-----------|------------------|-------|
| IGC | `libigc.so` | Intel Graphics Compiler |
| CM | `libcm.so` | Compute runtime |
| media-driver | VA-API | Media encode/decode |
| Level Zero | `libze_loader.so` | oneAPI Level Zero API |

**Translation path:**
- Vulkan → SPIR-V → IGC → Gen/Xe ISA
- OpenCL → SPIR-V → IGC → Gen/Xe ISA
- SYCL → oneAPI DPC++ → IGC → Gen/Xe ISA
- Intel has the best SPIR-V support of all vendors — IGC accepts SPIR-V
  directly with minimal conversion.

### 5.4 Moore Threads (Chunxiao)

| Component | Integration Point | Notes |
|-----------|------------------|-------|
| mtgpu-drv | `/dev/dri/cardN` | DRM driver, dma-buf, fences |
| MTT compiler | Proprietary | Requires vendor SDK |

**Translation path:**
- Vulkan → SPIR-V → MTT proprietary compiler → Chunxiao ISA
- No public documentation for Chunxiao ISA
- **Fallback**: Use single-GPU dispatch for MTT cards in heterogeneous
  configurations until MTT releases public compiler documentation
- MTT's primary value in MVGAL is as an additional compute resource
  where API-level interception (task-based scheduling) works without
  command translation

---

## 6. Testing Strategy

### 6.1 Unit Tests

| Test | What It Validates |
|------|-------------------|
| `test_spirv_extraction` | SPIR-V bytecode can be extracted from `VkShaderModule` |
| `test_spirv_vendor_compile` | SPIR-V → vendor ISA compilation succeeds |
| `test_barrier_normalization` | All barrier types produce canonical form |
| `test_barrier_amd_emit` | Canonical barrier → AMD PAL commands |
| `test_barrier_nvidia_emit` | Canonical barrier → NVIDIA ChID methods |
| `test_dag_construction` | Command buffer → DAG conversion preserves ordering |
| `test_dag_translate` | DAG node translation across vendors |

### 6.2 Integration Tests

| Test | What It Validates |
|------|-------------------|
| `test_same_shader_all_gpus` | Same SPIR-V compiles on all vendor compilers |
| `test_barrier_roundtrip` | AMD→canonical→NVIDIA barrier produces valid NVIDIA barrier |
| `test_cmd_buffer_replay` | Captured commands replay correctly on target GPU |
| `test_render_same_vulkan` | Same frame rendered on 2 GPUs produces identical output |
| `test_amd_nvidia_pair` | Real AMD+NVIDIA pair, full pipeline validation |

### 6.3 Hardware Pairing

The following GPU pairs should be validated:

| Primary GPU | Secondary GPU | Use Case |
|-------------|---------------|----------|
| AMD RX 7900 XTX | NVIDIA RTX 4090 | High-end gaming + compute |
| Intel Arc A770 | NVIDIA RTX 4060 | Mid-range hybrid |
| AMD RX 6600 | Intel UHD 770 | Integrated + discrete |
| AMD RX 7900 XTX | MTT S80 | Linux-focused + compute offload |
| NVIDIA RTX 4080 | MTT S80 | CUDA primary + MTT compute |

---

## 7. Risks and Mitigations

| Risk | Impact | Mitigation |
|------|--------|------------|
| NVIDIA ISA is undocumented | High — full translation impossible | Start with SPIR-V routing (Phase 1); NVIDIA Vulkan ICD can receive pre-compiled shaders via `VK_KHR_external_shader` |
| MTT ISA is proprietary | Medium — limits MTT integration | Use API-level interception for MTT (no translation needed); push for MTT developer documentation |
| SPIR-V → vendor compiler quality varies | Medium — suboptimal shaders on some GPUs | Fall back to single-GPU dispatch when translation quality is low; runtime benchmarking to select best GPU for each shader |
| Command buffer DAG capture overhead | Low — CPU-side only | Implement with `VK_KHR_performance_query` markers; DAG capture only for commands that cross GPU boundaries |
| Testing requires expensive hardware | Medium | Start with virtual GPU pairs (Lavapipe + llvmpipe); add real HW pairs incrementally |

---

## 8. Timeline Estimate

| Phase | Scope | Effort | Hardware Required |
|-------|-------|--------|-------------------|
| Phase 1 | SPIR-V routing + caching | 3-4 weeks | None (Lavapipe + vendor compilers) |
| Phase 2 | Barrier translation | 4-6 weeks | 1 AMD + 1 NVIDIA GPU |
| Phase 3 | Full command buffer DAG | 8-12 weeks | 2+ real GPUs (any vendors) |
| Phase 4 | Cross-vendor conformance testing | 4-8 weeks | HW pairs from 5.3 |

**Total estimated effort: 19-30 weeks** for production-grade cross-vendor
command stream translation.

---

## 9. Open Questions

1. **SPIR-V → NVIDIA**: What is the most reliable path from SPIR-V to NVIDIA's
   SASS? Options: SPIRV-Cross → GLSL → NVIDIA GLSL compiler, or
   SPIRV-Cross → HLSL → DXC → NVIDIA Vulkan driver.

2. **UVM compatibility**: Does NVIDIA's Unified Virtual Memory work with
   shaders compiled from SPIR-V via non-CUDA paths?

3. **MTT compiler access**: Can Moore Threads provide an SDK/compiler for
   Chunxiao ISA? Without it, MTT GPUs will be limited to API-level interception.

4. **BAR1 vs GART addressing**: When re-emitting commands for a different GPU,
   do resource addresses (textures, buffers, descriptors) need translation?
   If GPUs share a unified address space via DMA-BUF, this may not be needed.

5. **Vulkan video extensions**: How to handle `VK_KHR_video_encode/decode`
   when one GPU supports NVENC but another doesn't?

---

## 10. References

- [Vulkan Loader Specification](https://github.com/KhronosGroup/Vulkan-Loader)
- [SPIRV-Tools](https://github.com/KhronosGroup/SPIRV-Tools)
- [SPIRV-Cross](https://github.com/KhronosGroup/SPIRV-Cross)
- [AMDGPU LLVM Backend](https://github.com/llvm/llvm-project/tree/main/llvm/lib/Target/AMDGPU)
- [Intel Graphics Compiler](https://github.com/intel/intel-graphics-compiler)
- [NVIDIA open-gpu-kernel-modules](https://github.com/NVIDIA/open-gpu-kernel-modules)
- [Moore Threads mtgpu-drv](https://github.com/dixyes/mtgpu-drv)
- [Vulkan Video extensions](https://www.khronos.org/vulkan/)
- [Mesa ACO compiler](https://gitlab.freedesktop.org/mesa/mesa/-/tree/main/src/amd/compiler)
