# Systing Trace Compatibility

## Overview

scxtop now fully supports parsing and analyzing perfetto traces generated by
[systing](https://github.com/josefbacik/systing), a system-wide tracing tool
that uses perfetto's CompactSched format for efficient scheduler event
encoding.

## What Changed

### CompactSched Parsing Support

Added CompactSched format parsing to `perfetto_parser.rs` (lines 132-220, 345-358):

- **Delta-encoded timestamps**: Reconstructs absolute timestamps from delta-encoded values
- **Interned string table**: Resolves command names from intern table indices
- **Event expansion**: Converts compact format into individual `FtraceEvent` objects
- **Event types supported**:
  - `sched_switch` - Process context switches
  - `sched_waking` - Task wakeup events

### Compatibility

**Systing Format**:
- ✅ CompactSched-encoded scheduler events (sched_switch, sched_waking)
- ✅ Individual FtraceEvents (softirq, irq, etc.)
- ⚠️ TrackEvent format (partially supported)
- ❌ InternedData for callstacks (not yet implemented)

**Analyzers Verified**:
All core schedulng analyzers work with systing traces:
1. CPU Utilization
2. Task State Analysis
3. Wakeup Chain Detection
4. Latency Breakdown
5. Wakeup Latency
6. Preemption Analysis

## Usage

```bash
# Generate systing trace
cd ~/systing
sudo ./systing -o trace.pb

# Analyze with scxtop
cd ~/scx/tools/scxtop
cargo run --release -- mcp <<EOF
{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"load_perfetto_trace","arguments":{"file_path":"/home/hodgesd/systing/trace.pb","trace_id":"systing"}}}
EOF
```

Or use the Claude Code `/analyze_trace` command:
```
/analyze_trace ~/systing/trace.pb
```

## Test Results

### Before CompactSched Support
```
Total events: 8,575
Event types: ["softirq"]
```

### After CompactSched Support
```
Total events: 17,306 (more than doubled!)
Event types: ["sched_switch", "sched_waking", "softirq"]
```

## Implementation Details

### Parser Flow

1. **Detect CompactSched**: Check if `FtraceEventBundle.compact_sched` field is present
2. **Expand sched_waking events**:
   - Iterate through `waking_timestamp` deltas
   - Reconstruct absolute timestamps
   - Resolve command names from `intern_table`
   - Create `SchedWakingFtraceEvent` with pid, target_cpu, prio, comm
3. **Expand sched_switch events**:
   - Iterate through `switch_timestamp` deltas
   - Reconstruct absolute timestamps
   - Resolve next_comm from `intern_table`
   - Create `SchedSwitchFtraceEvent` with prev_state, next_pid, next_prio, next_comm
4. **Merge and sort**: Combine with individual FtraceEvents and sort by timestamp

### Code Locations

- **CompactSched expansion**: `tools/scxtop/src/mcp/perfetto_parser.rs:132-220`
- **FtraceBundle parsing**: `tools/scxtop/src/mcp/perfetto_parser.rs:324-359`
- **Systing compatibility tests**: `tools/scxtop/tests/test_systing_compat.rs`

## Known Limitations

1. **prev_pid/prev_comm missing**: CompactSched format (as used by systing) doesn't include prev_pid/prev_comm in sched_switch events. These could be inferred from previous events but aren't currently.

2. **InternedData not parsed**: Callstack information (frames, mappings) stored in InternedData is not yet parsed. This is needed for full stack trace support.

3. **TrackEvent partial support**: Custom probe events (USDT, uprobe, kprobe) use TrackEvent format which has partial support (Phase 9 infrastructure exists but not fully activated).

## Future Work

- [ ] Parse InternedData for callstack/frame/mapping information
- [ ] Add prev_pid/prev_comm inference from context
- [ ] Complete TrackEvent format support for custom probes
- [ ] Add systing-specific metadata extraction

## Testing

Run systing compatibility tests:
```bash
cd ~/scx/tools/scxtop
cargo test --release test_systing -- --nocapture
```

Expected output:
```
✓ Successfully loaded systing trace!
  Duration: 1.00 seconds
  Total events: 17306
  Event types: ["sched_switch", "sched_waking", "softirq"]

✓ Ran 6 analyzers successfully on systing trace!
```

## References

- **Systing**: https://github.com/josefbacik/systing
- **Perfetto CompactSched**: https://perfetto.dev/docs/reference/trace-packet-proto#CompactSched
- **scxtop Perfetto Guide**: `tools/scxtop/PERFETTO_ANALYZER_GUIDE.md`
