Model Loading Error

#3
by imranali291 - opened

Running your code in colab but getting error: Error loading models: .to is not supported for 4-bit or 8-bit bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct dtype.

Thanks for the report. I wonder if the library's specifications have changed...🤔
I don't know the direct cause, so I've tried removing the suspicious parts for now.

@John6666
The error still seems to be occurring. Could you please check?

Running on cuda
Loading in NF4
Loading CLIP 📎
Loading VLM's custom vision model 📎
Loading tokenizer 🪙
Loading LLM: unsloth/Meta-Llama-3.1-8B-bnb-4bit 🤖
Error loading models: .to is not supported for 4-bit or 8-bit bitsandbytes models. Please use the model as it is, since the model has already been set to the correct devices and casted to the correct dtype.

Owner

It seems that accelerate and bitsandbytes were fighting. It's probably because of the version upgrade of accelerate in Colab. I think the change I just made will fix this.
https://github.com/OpenBMB/MiniCPM-o/issues/379

Thank you, it worked properly!

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment