有没有在3090上部署这个awq版本的，速度只有6tokens/s，正常吗
2
#4 opened 4 days ago
by
Jsoooooo
AWQ Quant Settings?
#3 opened 8 days ago
by
radna
Performance loss of AWQ compared to the original model
#2 opened 8 days ago
by
Saaiet
disable thinking for some requests
3
#1 opened 9 days ago
by
devops724