General discussion.

#1
by Lewdiculous - opened
Nitral's Archive org
edited Mar 1, 2024

@Lewdiculous lmao its in the que for the leaderboard, but ive been testing locally and it cooks so far. Waiting to stretch the context after a few more ideas i want to get inside the model.

Curious for leaderboard results 👀

Nitral's Archive org

image.png

improvement but not much:

image.png

Hey, it's an improvement nonetheless! 👏

Prima-LelantaclesV6-7b-Q4_K_M-imatrix.gguf
Prima-LelantaclesV6-7b-Q4_K_S-imatrix.gguf
Prima-LelantaclesV6-7b-IQ4_NL-imatrix.gguf
Prima-LelantaclesV6-7b-IQ4_XS-imatrix.gguf (only supported in koboldcpp 1.60 or higher)
Prima-LelantaclesV6-7b-Q5_K_M-imatrix.gguf
Prima-LelantaclesV6-7b-Q5_K_S-imatrix.gguf
Prima-LelantaclesV6-7b-Q6_K-imatrix.gguf
Prima-LelantaclesV6-7b-Q8_0-imatrix.gguf
Prima-LelantaclesV6-7b-IQ3_M-imatrix.gguf
Prima-LelantaclesV6-7b-IQ3_S-imatrix.gguf
Prima-LelantaclesV6-7b-IQ3_XS-imatrix.gguf
Prima-LelantaclesV6-7b-IQ3_XXS-imatrix.gguf

The GGUF situation is getting intense XD

I think things are sort of getting re-unified again soon, maybe? Hopefully.

Lewdiculous/Prima-LelantaclesV6-7b-GGUF-IQ-Imatrix

Fixed: Link

Nitral's Archive org

Appreciated as always! i went kind of nuts tonight and did a bunch of stuff. Will update with more information towards the weekend and as i have more time.

@Lewdiculous Is support coming in the next koboldcpp release or are you referring to 1.59.1? I cant find it in lostruins' releases

@saishf - Yes, I meant in the next release of KCPP, for the IQ4_XS support to be merged, this is the PR that needs to be merged downstream.

Okay, thank you!

Nitral's Archive org
edited Mar 2, 2024

The Chaotic Neutrals have cooked up a merge using the two front runner models from the org: Eris-7b coming soon. (LelantaclesV6 x Prodigy) from both jeiku and myself. https://huggingface.co/ChaoticNeutrals/Eris_7B

Great news. 🎉

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment