• 1 Post
  • 29 Comments
Joined 1 year ago
cake
Cake day: August 8th, 2023

help-circle

  • I assume the “kill it” comment was a little tongue-in-cheek. On small SBCs, like a Pi, or old hardware, it could be a problem. I’ve seen people with flatpaks taking up 30GB of space, which is significant. I’m not sure how much RAM it wastes. I assume running 6 different applications that have loaded 6 different versions of Qt libraries would also use significantly more RAM than just loading the system’s shared Qt libraries once.

















  • IDK, looks like 48GB cloud pricing would be 0.35/hr => $255/month. Used 3090s go for $700. Two 3090s would give you 48GB of VRAM, and cost $1400 (I’m assuming you can do “model-parallel” will Llama; never tried running an LLM, but it should be possible and work well). So, the break-even point would be <6 months. Hmm, but if Severless works well, that could be pretty cheap. Would probably take a few minutes to process and load a ~48GB model every cold start though?