exl2 quant (measurement.json in main branch)
check revisions for quants
StarDust-12b-v1
GGUF Quants
(This is made by me, I'm slowly figuring out how to quant them)
Description | Usecase
The result of this merge is in my opinion a more vibrant and less generic sonnet inspired prose, it's able to be gentle and harsh where asked. I've personally been trying to get a more spice while also compensating for the Magnum-v2.5 having the issue on my end that it simply won't stop yapping.
- This model is intended to be used as a Role-playing model.
- Its direct conversational output is... I can't even say it's luck, it's just not made for it.
- Extension to Conversational output: The Model is designed for roleplay, direct instructing or general purpose is NOT recommended.
Prompting
Both Mistral and ChatML should work though I had better results with ChatML: ChatML Example:
"""<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
"""
Merge Details
Merge Method
This model was merged using the DARE TIES merge method using Sao10K/MN-12B-Lyra-v3 as a base.
Models Merged
The following models were included in the merge:
- Gryphe/Pantheon-RP-1.6-12b-Nemo
- anthracite-org/magnum-v2.5-12b-kto
- nbeerbower/mistral-nemo-bophades-12B
- Sao10K/MN-12B-Lyra-v3
Special Thanks
Special thanks to the SillyTilly and myself for helping me find the energy to finish this.
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.
Model tree for lucyknada/Luni_StarDust-12b-v1-exl2
Base model
mistralai/Mistral-Nemo-Base-2407
Finetuned
Gryphe/Pantheon-RP-1.6-12b-Nemo