It's a truly bitter pill to swallow when your whole area of research goes redundant.
I have a bit of background in this field so it's nice to see even people who were at the top of the field raise concerns that I had. That comment about LHC was exactly what I told my professor. That the whole field seems to be moving in a direction where you need a lot of resources to do anything. You can have 10 different ideas on how to improve LLMs but unless you have the resources there is barely anything you can do.
NLP was the main reason I pursued an MS degree but by the end of my course I was not longer interested in it mostly because of this.
> That the whole field seems to be moving in a direction where you need a lot of resources to do anything. You can have 10 different ideas on how to improve LLMs but unless you have the resources there is barely anything you can do.
I think you're confusing problems, or you're not realizing that improving the efficiency of a class of models is a research area on it's own. Look at any field that involves expensive computational work. Model reduction strategies dominate research.
I felt that way maybe an year or two ago. It seemed like the most research were only concerned about building bigger models to beat benchmarks. There was also this prevalent idea that models need to be big and have massive compute. Especially from companies like openai. I was glad that models like deepseek were made. Bought back some hope
I have a bit of background in this field so it's nice to see even people who were at the top of the field raise concerns that I had. That comment about LHC was exactly what I told my professor. That the whole field seems to be moving in a direction where you need a lot of resources to do anything. You can have 10 different ideas on how to improve LLMs but unless you have the resources there is barely anything you can do.
NLP was the main reason I pursued an MS degree but by the end of my course I was not longer interested in it mostly because of this.