The AI buttle is deflating right now, CPUs are getting cheaper by about 20%, as well as ram and storage, as old boy Sammy can’t hold up his exaggerations and had to admit he can only spend half as much in the next 5 years. I really hope to get a gpu with 16-24g in the next 2 years, running AI locally will only get better



As everyone recommends something else, I will throw in yet another suggestion and maybe a way to put it in perspective.
hardware:
What ever you have right now is enough. I would recommend something that has more than one drive, but you can upgrade lager all the time. Watch out for power consumption.
OS
my recommendation is proxmox. You can run multiple OS on it, test things, make backups and restore them. It takes out the pressure. It is a bit more advanced but you can configure most things regarding VM creation via GUI. You can run Ubuntu, Debian, nixos, whatever. What I like about it, is that you can install home assistant as its own VM and it runs independent of your are stack or immich if you like.
the purist would probably suggest plain Debian or so. You can access it via ssh and install docker or whatever. You will learn a lot, just like with proxmox, but here you can’t do backups as easily as with proxmox. You can also install Debian or so directly in proxmox.
plug and play could be yunohost, CasaOS and things like that. It is an “server app store OS”, so you install things you want to run from their app store. Its mostly one click and it runs. Its nice if you don’t want to learn all that crap, but you are also limited in what you can do. You could install this on proxmox. Some don’t allow good storage management, best you’d o your own research if you are interested in this route.
the storage
If you want to store all you data and images on there, you propably want to go with 2 hdds mirrored, so you don’t loose your local data if one fails. I would go with no less than, 1tb, but data usage depends I guess. To run the os, definetly use ssd or nvme for their speed.
There is the 321 backup rule.
I have a 4 1 1.5 set up:
321 is ideal but 211 would also be ok for a home lab. Some run 110 and hope for the best.
how to access it
There are multiple ways to get to your data.
The headache free one for me is a vpn mesh (tailscale, pangolin, netbird), so all your devices are in a mesh and you can access your stiff from everywhere. pro:
You can also use a wireguard connection to your home router and expose you complete home network to your vpn. Also secure, my router (Fritz! Box from avm) offers this natively, but I would argue the vpn mesh solution is easier.
There is the cloud flared tunnel which some recommend, I cant say anything about it, did not use it yet.
You could also do port forwarding on your network. That way you can expose a reverse proxy for example through your home router, and access it from the outside. That way you rely on the services you run to be secure, not have a zero day and to do the authentication well. For me the risk that I forgot to update a service and there being a security risk to my data is too high, so I use the vpn route.
services to run
What ever route you choose, here are suggestions I found nice:
Have fun! Do what ever feels fun and don’t put your goals too high, it will burn you out :)