i dont even have a GPU and the 14b model runs at an acceptable speed. but yes, faster and bigger would be nice… or knowing how to distill the biggest one, cuz I only use it for something very specific.
Correct. But what’s more expensive a single computing instance that’s local or cloud based credit eating SAS AI that does not produce significantly better results?
You still need an expensive hardware to run it. Unless myceliumwebserver project will start
Removed by mod
How much vram does your TI pack? Is that the standard 8gb ddr6?
I will because I’m surprised and impressed that a 14b model runs smoothly.
Thanks for the insights!
Removed by mod
No worries, thank you!
i dont even have a GPU and the 14b model runs at an acceptable speed. but yes, faster and bigger would be nice… or knowing how to distill the biggest one, cuz I only use it for something very specific.
Correct. But what’s more expensive a single computing instance that’s local or cloud based credit eating SAS AI that does not produce significantly better results?