Xylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-223 hours agoMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:media1.tenor.comexternal-linkmessage-square11fedilinkarrow-up179arrow-down15
arrow-up174arrow-down1external-linkMy 8gb vram system as i try to load GLM-4.6-Q0.00001_XXXS.gguf:media1.tenor.comXylight@lemdro.id to LocalLLaMA@sh.itjust.worksEnglish · edit-223 hours agomessage-square11fedilink
minus-squareffhein@lemmy.worldlinkfedilinkEnglisharrow-up1·8 hours agoI guess there’s some automatic vram paging going on. How many tokens per second do you get while generating?
I guess there’s some automatic vram paging going on. How many tokens per second do you get while generating?