Most people who work in "the arts" probably aren't communicating anything directly either - they just create the scenes, sound effects, textures, animation, models +++ that someone above them in the organization has asked them to create for their project.

What's the difference between having an idea, then putting an actor on a set, lighting them, doing background green screen set extension afterwards, digital clean up, etc. vs doing all of that generatively?

How is asking a VFX house for animated footage any different than generating it? If art is intent, there is no reason you can't generate the building blocks that reflect that intent, no?