README.md

---
license: cc-by-nc-4.0
base_model:
- PatronusAI/glider
---

Available GGUF versions for the [PatronusAI/glider](https://huggingface.co/PatronusAI/glider) model: [`BF16`, `Q8_0`, `Q5_K_M`, `Q4_K_M`]

How to load your desired quantized model:

1. Select the appropraite GGUF quantization from the available list above
2. Run the following code:
```bash
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("PatronusAI/glider-gguf", gguf_file="glider_{version_from_list}.gguf")
```
For loading the Q8_0 version, this script will change to:
```bash
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("PatronusAI/glider-gguf", gguf_file="glider_Q8_0.gguf")
```

For any issues or questions, reach out to [Darshan Deshpande](https://huggingface.co/darshandeshpande) or [Rebecca Qian](https://huggingface.co/RebeccaQian1)