google/gemma-3-27b-it
Image-Text-to-Text
•
Updated
•
38.5k
•
463
Arthur Zucker
ArthurZ
AI & ML interests
None yet
Recent Activity
liked
a model
1 day ago
google/gemma-3-27b-it
liked
a model
12 days ago
microsoft/Magma-8B
liked
a model
27 days ago
Qwen/Qwen2.5-3B
Organizations
ArthurZ's activity
liked
a
model
1 day ago
liked
a
model
12 days ago
liked
a
model
27 days ago
liked
3
models
about 1 month ago
Fastest way for inference?
3
#28 opened about 1 month ago
by
psycy
liked
a
model
about 1 month ago
model-00078-of-000163.safetensors not marked safe?
2
#80 opened about 1 month ago
by
aborst
liked
a
model
about 1 month ago
updated
a
model
about 1 month ago
upvoted
an
article
about 1 month ago
Article
Welcome to Inference Providers on the Hub 🔥
•
429
published
a
model
about 1 month ago
reacted to
mitkox's
post with 🚀
about 2 months ago
Post
2499
llama.cpp is 26.8% faster than ollama.
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison.
Total duration:
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec
Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms
Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms
Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
I have upgraded both, and using the same settings, I am running the same DeepSeek R1 Distill 1.5B on the same hardware. It's an Apples to Apples comparison.
Total duration:
llama.cpp 6.85 sec <- 26.8% faster
ollama 8.69 sec
Breakdown by phase:
Model loading
llama.cpp 241 ms <- 2x faster
ollama 553 ms
Prompt processing
llama.cpp 416.04 tokens/s with an eval time 45.67 ms <- 10x faster
ollama 42.17 tokens/s with an eval time of 498 ms
Token generation
llama.cpp 137.79 tokens/s with an eval time 6.62 sec <- 13% faster
ollama 122.07 tokens/s with an eval time 7.64 sec
llama.cpp is LLM inference in C/C++; ollama adds abstraction layers and marketing.
Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.
- 7 replies
liked
a
Space
about 2 months ago
upvoted
an
article
about 2 months ago
Article
SmolVLM Grows Smaller – Introducing the 250M & 500M Models!
•
154
liked
a
model
about 2 months ago
upvoted
an
article
about 2 months ago
Article
Mastering Long Contexts in LLMs with KVPress
By
and 1 other
•
•
64
liked
2
models
about 2 months ago