This is why this is a bad idea.

If you look at the actual source, it is not Deepseek R1:

• https://github.com/canonical/deepseek-r1-snap/tree/main/components/model-distill-qwen-7b-q4-k-m-gguf

…It is the Qwen 2.5 7B Deepseek R1 distil.

This is a terrible model. It is:

• Old, obsolete.
• Not the “real” Deepseek MoE, but misleadingly labeled.
• Dumb even when it was released.
• Not optimized for runtime CPU repacking.
• A “flat” Q4KM without extra weight to sensitive layers because it’s an ancient bartowski quantization.

This is extremely frustrating. Llama.cpp has about a bajillion different corporate entities developing “easy” front ends for it, all doing it wrong anyway, who don’t even acknowledge the original project. Much less contribute to it.

What’s more, advertising it as “easy AI” is going to give users bad impressions when they run it, and it’s as slow as molasses and as dumb as dirt.

The only redeeming quality I can find in this project is that they package the Intel OpenVINO runtimes for CPU/GPU/NPU much more intelligently (eg Intel must have helped them). And this is awesome because OpenVINO is very hard to set up, but that’s about it.

Hence, the reason basically every ML enthusiast recommends “figure out base llama.cpp” is that no one else downstream gets it right, they mess it up, except maybe kobold.cpp/croco.cpp. They just layer abstraction on top and screw stuff up so they can tell some boss “Look! 1 click AI!”

Intel and ARM Ampere systems.

Does this mean they optimized for CPU instead of GPU? I doubt they target Intel GPUs tbh, so they really optimized for CPU… interesting!

I wish these were on brew too.