# Patchpal development

## Local inference

Out of the box, the gui will connect to a proxy for internal inference servers,
at patchpal.usersys.redhat.com, port 9080. If all you care about is the gui,
you can simply run it on your local system and it will use the provided
inference server(s). If you wish to experiment with running the server locally,
there are a few levels of involvement. As noted in the backend README.md, the
model training itself was done on an Nvidia A100 with 80GB of VRAM. Running the
trained model without quantization and LoRA consumes about 21GB of VRAM. The
same model can be squeezed into roughly 7GB of VRAM with quantization and LoRA.

To summarize the possible levels of involvement and hardware requirements:

1. Just run the gui client and use hosted model -- no gpu requirement
2. Run gui and quantized/LoRA pre-trained model locally -- 8GB GPU
3. Run gui and full pre-trained model locally -- 24GB GPU
4. Run gui and train your own model locally -- 80GB GPU

Both Nvidia (cuda) and AMD GPUs (rocm) are supported, but AMD setup is far more
tempermental. Patches welcomed if you can help improve support and/or add
support for another accelerator stack.

The server backend and further details on resource requirements, the rpc
interface, and how to train the models can be found here:  
  https://gitlab.com/patchpal-ai/patchpal-backend

To let the gui connect to your inference server, add to `~/.config/patchpal.conf`:

```
rpc = http://your.server:9080/v1
```

## Extract Data

In addition to the GUI, this project also contains a tool to extract the
training data. See:

```
~/patchpal-gui/target/release/extractor --help
```

There's no `-g` option. All repositories must contain also objects from
upstream.
