One of the problems programmers have is loading a problem into working memory. It can take an hour. An interruption, a phone call, or a meeting can mean that you have to start over (or, if not completely over, you still have to redo part of it). This is a standard programmer complaint about interruptions.
It's interesting that LLMs may have a similar issue.
We have long term memory and short term.
Context is short term.
The still long and expensive training phase embeds the long term memory.