Proxmox with a bunch of small dedicated lxc containers (running small Ubuntu lxcs mostly). It works for me anyway, though I don’t use any of the rr programs. My Plex lxc gets its Media files from an openmediavault NFS mount (Running in a VM instead of lxc), and I use smb file sharing for my windows desktop to access the same files. Your mileage may vary, what works for me night not be for someone else.
I don’t use Calibre for this (but I do for epub and pdf ebooks). I use Plex and use Plexamp for my music collection. For audio books I used to have Chronicle Audiobook Player installed (which is open source, but not actively maintained) but am now trying out Listen Audiobook Player (which is not open source as a heads up). Both of those allow me to download books wherever I am.
If you have no interest in running a Plex server neither of them are suitable for you, but I just wanted to mention this solution in case you’re not dead set on using Calibre. Plex has a cost factor which may come into play (an obvious non starter for some). I have had Plex Pass for a long time so it was natural for me to use it, but might not be for you.