VR on linux actually works just fine from my experience. I’ve never had a game not work. The big issue is just headset support. The HTC Vive and Valve Index are the only headsets with official drivers, since they were made by Valve. Standalone headsets, like the Quest for example, also work using ALVR. Anything else doesn’t really work. There are open source drivers but they’re not complete enough to be useable unless something majorly changed there since I last checked.
I have the same GPU and I use koboldcpp with Vulkan as the backend. Works perfectly fine. I have a 12B model and it’s extremely fast. I could probably even fit a bigger model into the VRAM. Using tabbyAPI for EXL2 models didn’t work for me, it always generated gibberish (I tried 2 different models). For context, I’m on Linux, so maybe that’s not an issue on other operating systems.