>Find an answer in a text document that spans multiple non-contiguous paragraphs
>Understand context across a whole book.
to have truly much utility when it comes to context a model doesn't just need to correlate information across some text, it also needs general knowledge and understanding so that it can produce knowledge which is only implicit or not even present in the text itself.
You can make the transformers as large as you want, NLP models still fundamentally suck at answer trivial questions like "If Pete was alive in 2000 and alive in 2050, was he alive in 2025?"
>Understand context across a whole book.
to have truly much utility when it comes to context a model doesn't just need to correlate information across some text, it also needs general knowledge and understanding so that it can produce knowledge which is only implicit or not even present in the text itself.
You can make the transformers as large as you want, NLP models still fundamentally suck at answer trivial questions like "If Pete was alive in 2000 and alive in 2050, was he alive in 2025?"