I'd at least consider moving older draw calls into an xlib replacement. Not all of them are suitable for that, but e.g. sufficient font handling to beat Xorgs server side handling requires the Render extension plus ca. 1500 lines of C for a basic TrueType renderer, or half that in a higher level language, or just use FreeType which is a dependency for most X clients anyway.