# Priority 3: Network GPU Pooling Architecture

## Overview

MVGAL's architecture already supports network-distributed GPU pooling through its IPC server design. This document describes how to extend the existing infrastructure for cross-machine GPU aggregation.

## Existing Infrastructure

### IPC Server (runtime/daemon/ipc_server.cpp)

The daemon already includes:
- **Unix socket** for local communication (`/run/mvgal/mvgal.sock`)
- **TCP/UDP discovery** for peer detection
- **Message-based protocol** with 30+ message types
- **Peer state machine** (DISCOVERED, CONNECTED, ACTIVE, DISCONNECTED)

### Supported Message Types

| Category | Types |
|----------|-------|
| Connection | HELLO, GOODBYE |
| Device Query | QUERY_DEVICES, QUERY_DEVICE_CAPABILITIES |
| Memory | ALLOC_MEMORY, FREE_MEMORY, IMPORT_DMABUF, EXPORT_DMABUF |
| Workload | SUBMIT_WORKLOAD, WAIT_WORKLOAD |
| Scheduling | SET_SCHEDULING_MODE, SET_GPU_PRIORITY |
| Statistics | GET_STATISTICS, SUBSCRIBE_TELEMETRY |
| Configuration | GET_CONFIG, SET_CONFIG, LOAD_CONFIG, SAVE_CONFIG |

### Discovery Protocol

Peers announce via UDP broadcast with:
- `ANNOUNCE` - "I exist with N GPUs"
- `HEARTBEAT` - "Still active"
- `GPU_ANNOUNCE` - "GPU capabilities update"

## Network GPU Pooling Implementation

### Required Extensions

1. **Network Transport Layer**
   ```cpp
   // Add to ipc_server.cpp
   class IpcNetworkServer : public IpcServer {
       // TCP socket for reliable transfers
       // RDMA/RoCE support for zero-copy
       // Connection pooling
   };
   ```

2. **Remote GPU Registry**
   ```cpp
   // Extend DeviceRegistry
   class RemoteDeviceRegistry : public DeviceRegistry {
       std::map<uint64_t, std::vector<RemoteGpuInfo>> remoteGpus;
       uint64_t aggregateTotalVram();
   };
   ```

3. **Cross-Network Memory Management**
   - DMA-BUF export over network
   - RDMA transfer for P2P emulation
   - Memory pinning for zero-copy

### Configuration

```ini
# /etc/mvgal/mvgal.conf
[network]
enabled = true
listen_port = 7777
discovery_port = 7778
enable_rdmacopy = false  # RoCE/RDMA support

[peer_policy]
max_peers = 8
trust_local_network = true
authentication = token
```

### Security Considerations

- **Authentication:** Token-based or mutual TLS
- **Authorization:** Per-GPU access controls
- **Encryption:** TLS 1.3 for production networks
- **Rate Limiting:** Prevent DoS from malicious peers

## Cloud GPU Integration

The network layer also enables:
- **AWS EC2 GPU instances** as remote peers
- **Azure NVv4/NVv5** instances
- **GCP A2/AT2** instances

Configuration example:
```ini
[cloud]
provider = aws
region = us-east-1
instance_type = g4dn.xlarge
ssh_key_path = ~/.ssh/mvgal_cloud
```

## AI Scheduler Integration

### Model Loading

The scheduler supports `APPLICATION_PROFILE` mode which can load ONNX models:

```cpp
// In scheduler.cpp
int Scheduler::loadModel(const std::string& modelPath) {
    // Load /etc/mvgal/models/scheduler.onnx
    // ONNX Runtime integration
    // Return handle for async dispatch
}
```

### Feature Vector

The AI model receives:
- GPU utilization percentages (up to 8 GPUs)
- VRAM pressure per GPU
- Temperature readings
- Workload history (last 16 submissions)
- Application fingerprint (hash of binary + command line)

## Implementation Roadmap

### Phase 1: Network Discovery (Estimated: 2 weeks)
- [ ] Enable TCP/UDP socket in IpcServer
- [ ] Implement peer announcement protocol
- [ ] Add remote device registration path

### Phase 2: Remote Memory (Estimated: 3 weeks)
- [ ] Extend DMA-BUF for network export
- [ ] Add RDMA transfer support
- [ ] Memory migration across network

### Phase 3: Scheduler Integration (Estimated: 1 week)
- [ ] ONNX Runtime dependency
- [ ] Feature extraction hooks
- [ ] Model hot-reload support

### Phase 4: Security & Production (Estimated: 2 weeks)
- [ ] TLS encryption
- [ ] Authentication tokens
- [ ] Rate limiting
- [ ] Documentation

## Current Status

| Feature | Status | Notes |
|---------|--------|-------|
| IPC Protocol | ✅ Complete | Message types defined |
| Discovery | ✅ Complete | UDP/TCP code exists |
| Network Socket | ✅ Exists | Not enabled by default |
| Remote Devices | 🔄 Partial | Placeholder data structures |
| RDMA Support | ⭕ Future | Design documented |
| AI Model Hook | 🔄 Partial | APPLICATION_PROFILE mode exists |

## References

- `runtime/daemon/ipc_server.hpp` - Full IPC protocol definition
- `runtime/daemon/scheduler.cpp` - SCHEDULING_MODE_APPLICATION_PROFILE
- `include/mvgal/mvgal.h` - Public API for remote management