The probability distribution over next tokens given previous tokens is deterministic. The sampling algorithm for that distribution is non-deterministic.
And sampling from a (now fixed) distribution can be made deterministic...
So the total generation of text from an LLM can be made fully deterministic. The problem for scientists is that we cant do that in the deployed systems...
You can set the temperature to zero in most APIs, which gives deterministic output. The only problem with that is some models produce inferior results with zero temperature, including lots of slop and AI-isms.
