SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 5 days agoDo you host your own AI?message-squaremessage-square203linkfedilinkarrow-up1181arrow-down142file-text
arrow-up1139arrow-down1message-squareDo you host your own AI?SuspiciousCarrot78@aussie.zone to Selfhosted@lemmy.worldEnglish · 5 days agomessage-square203linkfedilinkfile-text
minus-squarerobber@lemmy.mllinkfedilinkEnglisharrow-up1·4 days agoYou can control how much context should be fitted with --fit-ctx and how much space the algorithm should leave unallocated (even on a per-GPU basis) with --fit-target.
You can control how much context should be fitted with
--fit-ctxand how much space the algorithm should leave unallocated (even on a per-GPU basis) with--fit-target.