For app data, Borg as backup/restore software. Backup data is then stored on Hetzner as an offsite backup - super easy and cheap to setup. Also add healthchecks.io to get notified if a backup failed.

Edit: Backup docker compose files and other scripts (without API keys!!!) with git to GitHub.

I’d recommend Debian Linux. It’s free, stable, has all the software you’ll need with long term support, ton of online resources and communities to learn from. You can start with or without UI.