Yeah, this is the way to do this. I'm pretty sure that if you for some reason do not want to run wireguard on all your servers you could fairly easily adjust this recipe to have a centralized wg gateway on your local network instead.

I think I've seen some scripts floating around to automate this process but can't remember where. There are lots of good related tools listed here: https://github.com/anderspitman/awesome-tunneling