This is one of those things where I don’t agree with the argument, but know the person making it knows way more than I do on the subject and has given it way more thought. In these cases it’s usually best to sit back and listen a bit...
The part of the Polgár story that amazes me the most is that all three daughters showed enough interest and engagement in chess for the experiment to work so successfully. Because with my own children, I’ve seen again and again that you can encourage and expose them to certain interests, but they’re their own people - many of the things I exposed them to and tried to get them excited about just weren’t interesting to them.
And that’s completely fine. I was never forceful about it and they have their own deep interests in things that I just never got into or understood. I just find it surprising that in some families, these exceptional skills and interests are so readily passed from one generation to the next.
A bullet-point list of ways you screwed up communicates something entirely different than a long form email filled with flowery "fluff" (as the author puts it).
In fact, if the author feels confident in this theory, I suggest they replace the blog post with this AI-generated bullet-point summary I just made...
I fully agree with the sentiment, and yet marvel at the apparent lack of alternatives for so many business models. I'm mildly surprised, for example, that micro-payments for web content are still not a (widespread) thing.
Consumers hate ads, but they hate paying for things even more apparently.
It's not really clear to me that consumers hate ads. It seems like more of a tech niche opinion; most people are perfectly fine with watching TV commercials (and look forward to them at the Super Bowl) and don't mind using ad-tier streaming plans. Ad blockers are used by a tiny, tiny percentage of total browser users. And so on.
Look at literally any poll on the topic - people hate ads universally, with advertisers not far behind. And ad blocker usage has skyrocketed. It's now up to ~30% of all internet users [1] with younger people starting to approach a majority. And it should go without saying that the only reason that's not near 100% of users is because most people don't know they exist and/or don't feel comfortable installing one.
This link claims that 39% of people over the age of 56 use an ad blocker. Sorry, but color me skeptical. I could but wrong, but I am highly skeptical of that statistic.
This report also doesn't seem to clarify what is meant by "ad blocker," specifically, which makes the whole thing pretty suspect. Looking into the sources they provide, at least one of them doesn't mention ad blockers at all, even though the related image is about people blocking ads. So, yeah, I'm not considering this to be reliable information.
Have you noticed the growing number of sites trying to be increasingly aggressive towards ad-blockers? They're not jumping through these increasingly sophisticated (and expensive!) technical hoops to try to squeeze a few more ad views out of a "tiny, tiny percentage".
The Reg gives 52% of all Americans [1], b2.ai (who sells anti ad block services) gives 45% in North America [2], and so on. Most sources seem to fall in a 30%-50% range, with all sources agreeing that uptake on adblockers has rapidly and dramatically increased. People hate ads.
1. The study is claiming 50% of all Americans based on a sample size of 2,000.
According to a survey of 2,000 Americans conducted by research firm Censuswide, on behalf of Ghostery, a maker of software to block ads and online tracking, 52 percent of Americans now use an ad blocker, up from 34 percent according to 2022 Statista data.
2. Both of them, and indeed all of these studies seem to be funded by ad blocking companies. That alone makes them suspect.
I can buy the idea that the advertising industry is under pressure from built-in restrictions pushed by Apple, et al. But the idea that 52% of the American population even knows what an ad blocker is, much less has one installed, is completely absurd.
From a quick search, that percentage is comparable to the number of people that: own an iPhone, drink coffee everyday, or own a pet.
2000 is a massive sample assuming it's reasonably representative - far larger than you'd see in an average e.g. election poll. Statistics can be counter-intuitive. The surveys are also generally carried out by survey companies, not the first parties.
You can corroborate these data pretty much anywhere and everywhere. For a silly one I dug up here's [1] PewdiePie in 2016 talking about already seeing a 40% adblock rate as reflected by a non-scientific poll but also in his revenue stats, up from 10%-15% in past years. And it's certainly way higher now - obviously though that sample is going to trend young and probably technically above average.
But really, the thing about ad-blocking is that it's the ultimate in viral tech. Anytime I meet somebody who's not using ad-block I tend to introduce them to the Brave browser and the result like 99.9% of the time is 'omg I didn't even know this was possible.' Those are now people who will probably never go without ad-blocking again and some percent of whom will likely then go on to introduce others to it. There is literally no downside to using an ad-blocker and a million upsides. People just simply don't know about them and/or think they're somehow difficult to use.
I'm really not going to keep arguing about this. I know how statistics work. The studies are clearly biased and make absurd conclusions, and I've listed about ten different problems with them - the foremost being that "ad blocker" is never defined anywhere.
The idea that the number of people using an ad blocker is equivalent to the number of people that drink coffee every morning, subscribe to Netflix, or have a pet makes no sense whatsoever, and if that were the case, you'd see vastly more discussion and general awareness of the concept.
There is relatively high general awareness of the concept! Just remember that we all live in bubbles. And the slice of life you see and know is going to have its own little biases that are not nationally reflected. For an easy one in tech, did the majority of people you know (who voted) vote for Trump?
Even ignoring the sibling comment debunking your "tiny, tiny percentage"...
There's a huge difference between extremely-high-production-value Superbowl ads and regular TV ads.
There's another huge difference between regular TV ads and some of the ads you can get on YouTube (example: someone I follow decided, For Science™, to actually watch the entirety of a 5-hour ad they got on YouTube for...I think it was one of the Lego Movies? It consisted entirely of an endless repetition of the same 3-minute-ish song).
There's also a huge difference between what people are willing to tolerate in order to reduce some of the often-absurd financial burden they're being put under, and what people actually like, or are genuinely content with.
"Revealed preference" theory is a bunch of motivated reasoning.
One problem is transaction costs; you need aggregators to sit in the middle, adding up the people paying and to be paid out. Structurally, aggregators are then in a position to extract rents. Which they will do, sooner or later.
There are of course other problems, fraud, money laundering and the like, which extract taxes, and and a whole patchwork quilt of national regulations, which extracts a lot more.
Things like crypto look, feel and smell like money laundering. I don't think there's a technical solution here. The problems are social and regulatory.
I think this argument only makes sense if you believe that AGI and/or unbounded AI agents are "right around the corner". For sure, we will progress in that direction, but when and if we truly get there–who knows?
If you believe, as I do, that these things are a lot further off than some people assume, I think there's plenty of time to build a successful business solving domain-specific workflows in the meantime, and eventually adapting the product as more general technology becomes available.
Let's say 25 years ago you had the idea to build a product that can now be solved more generally with LLMs–let's say a really effective spam filter. Even knowing what you know now, would it have been right at the time to say, "Nah, don't build that business, it will eventually be solved with some new technology?"
I don't think it's that binary. We've had a lot of progress over the last 25 years; much of it in the last two. AGI is not a well defined thing that people easily agree on. So, determining whether we have it or not is actually not that simple.
Mostly people either get bogged down into deep philosophical debates or simply start listing things that AI can and cannot do (and why they believe why that is the case). Some of those things are codified in benchmarks. And of course the list of stuff that AIs can't do is getting stuff removed from it on a regular basis at an accelerating rate. That acceleration is the problem. People don't deal well with adapting to exponentially changing trends.
At some arbitrary point when that list has a certain length, we may or may not have AGI. It really depends on your point of view. But of course, most people score poorly on the same benchmarks we use for testing AIs. There are some specific groups of things where they still do better. But also a lot of AI researchers working on those things.
Consider OpenAI's products as an example. GPT-3 (2020) was a massive step up in reasoning ability from GPT-2 (2019). GPT-3.5 (2022) was another massive step up. GPT-4 (2023) was a big step up, but not quite as big. GPT-4o (2024) was marginally better at reasoning, but mostly an improvement with respect to non-core functionality like images and audio. o1 (2024) is apparently somewhat better at reasoning at the cost of being much slower. But when I tried it on some puzzle-type problems I thought would be on the hard side for GPT-4o, it gave me (confidently) wrong answers every time. 'Orion' was supposed to be released as GPT-5, but was reportedly cancelled for not being good enough. o3 (2025?) did really well on one benchmark at the cost of $10k in compute, or even better at the cost of >$1m – not terribly impressive. We'll see how much better it is than o1 in practical scenarios.
To me that looks like progress is decelerating. Admittedly, OpenAI's releases have gotten more frequent and that has made the differences between each release seem less impressive. But things are decelerating even on a time basis. Where is GPT-5?
>Let's say 25 years ago you had the idea to build a product
I resemble that remark ;)
>that can now be solved more generally with LLMs
Nope, sorry, not yet.
>"Nah, don't build that business, it will eventually be solved with some new technology?"
Actually I did listen to people like that to an extent, and started my business with the express intent of continuing to develop new technologies which would be adjacent to AI when it matured. Just better than I could at my employer where it was already in progress. It took a couple years before I was financially stable enough to consider layering in a neural network, but that was 30 years ago now :\
Wasn't possible to benefit with Windows 95 type of hardware, oh well, didn't expect a miracle anyway.
Heck, it's now been a full 45 years since I first dabbled in a bit of the ML with more kilobytes of desktop memory than most people had ever seen. I figured all that memory should be used for something, like memorizing, why not? Seemed logical. Didn't take long to figure out how much megabytes would help, but they didn't exist yet. And it became apparent that you could only go so far without a specialized computer chip of some kind to replace or augment a microprocessor CPU. What kind, I really had no idea :)
I didn't say they resembled 25-year-old ideas that much anyway ;)
>We've had a lot of progress over the last 25 years; much of it in the last two.
I guess it's understandable this has been making my popcorn more enjoyable than ever ;)
Agreed. There's a difference between developing new AI, and developing applications of existing AI. The OP seems to blur this distinction a bit.
The original "Bitter Lesson" article referenced in the OP is about developing new AI. In that domain, its point makes sense. But for the reasons you describe, it hardly applies at all to applications of AI. I suppose it might apply to some, but they're exceptions.
You think it will be 25 years before we have a drop in replacement for most office jobs?
I think it will be less than 5 years.
You seem to be assuming that the rapid progress in AI will suddenly stop.
I think if you look at the history of compute, that is ridiculous. Making the models bigger or work more is making them smarter.
Even if there is no progress in scaling memristors or any exotic new paradigm, high speed memory organized to localize data in frequently used neural circuits and photonic interconnects surely have multiple orders of magnitude of scaling gains in the next several years.
> You seem to be assuming that the rapid progress in AI will suddenly stop.
And you seem to assume that it will just continue for 5 years. We've already seen the plateau start. OpenAI has tacitly acknowledged that they don't know how to make a next generation model, and have been working on stepwise iteration for almost 2 years now.
Why should we project the rapid growth of 2021–2023 5 years into the future? It seems far more reasonable to project the growth of 2023–2025, which has been fast but not earth-shattering, and then also factor in the second derivative we've seen in that time and assume that it will actually continue to slow from here.
At this point, the lack of progress since April 2023 is really what is shocking.
I just looked on midjourney reddit to make sure I wasn't missing some new great model.
Instead what I notice is the small variations on the themes I have already seen a thousand times a year ago now. Midjourney is so limited in what it can actually produce.
I am really worried that all this is much closer to a parlor trick than AGI.
"simple trick or demonstration that is used especially to entertain or amuse guests"
It all feels more and more like that to me than any kind of progress towards general intelligence.
There's this [0]. But also o1/o3 is that acknowledgment. They're hitting the limits of scaling up models, so they've started scaling compute [1]. That is showing some promise, but it's nowhere near the rate of growth they were hitting while next gen models were buildable.
No, but there's really very little reason to think that that makes the ol' magic robots less shit in any sort of well-defined way. Like, it certainly _looks_ like they've plateaued.
I often suspect that the tech industry's perception of reality is skewed by Moore's Law. Moore's Law is, quibbles aside, basically real, and has of course had a dramatic impact on the tech industry. But there is a tendency to assume that that sort of scaling is _natural_, and the norm, and should just be expected in _everything_. And, er, that is not the case. Moore's Law is _weird_.
> You seem to be assuming that the rapid progress in AI will suddenly stop.
> I think if you look at the history of compute, that is ridiculous. Making the models bigger or work more is making them smarter.
It's better to talk about actual numbers to characterise progress and measure scaling:
"
By scaling I usually mean the specific empirical curve from the 2020 OAI paper. To stay on this curve requires large increases in training data of equivalent quality to what was used to derive the scaling relationships.
"[^2]
"I predicted last summer: 70% chance we fall off the LLM scaling curve because of data limits, in the next step beyond GPT4.
[…]
I would say the most plausible reason is because in order to get, say, another 10x in training data, people have started to resort either to synthetic data, so training data that's actually made up by models, or to lower quality data."[^0]
“There were extraordinary returns over the last three or four years as the Scaling Laws were getting going,” Dr. Hassabis said. “But we are no longer getting the same progress.”[^1]
o1 proved that synthetic data and inference time is a new ramp. There will be more challenges and more innovations. There is a lot of room in hardware, software, model training and model architecture left.
> It's not realistic to make firm quantified predictions any more specific than what I have given.
Then do you actually know what you're talking about or are you handwaving? I'm not trying to be offensive but business plans can't be made based on a lack of predictions.
> We will likely see between 3 and 10000 times improvement in efficiency or IQ or speed of LLM reasoning in the next 5 years
That variance is too large to take you seriously, unfortunately. That's unfortunate because I was really hoping you had an actionable insight for this discussion. :(
If I, for instance, tell my wife I can improve our income by 3x or 1000x but I don't really know, there's no planning that can be done and I'll probably have to sleep on the couch until I figure out what the hell I'm doing.
> business plans can't be made based on a lack of predictions.
They can. It's called "taking a risk". Which is what startups are about, right?
It's hard to give a specific prediction here (I'm leaning towards 10x-1000x in the next 5 years), but there's also no good reason to believe progress will stop, because a) there's many low and mid-hanging fruits to pick, as outlined by GP, and b) because it never did so far, so why would it stop now specifically?
Why did we stop going to the moon and flying commercial supersonic?
Some things that are technologically possible are not economically viable. AI is a marvel but I'm not convinced it will actually plug into economic gains that justify the enormous investment in compute.
Spoken like a young man. I salute you. However, on your journey remember that risk of ruin is what you want to minimize relative to your estimated rewards. That is, not all risks can be afforded. I happen to have a limited budget, perhaps you don't and costs in terms of money and time don't matter for you.
Ruin can set you back years or decades or permanently and then you find yourself on a ycombinator thread hopelessly trying to find someone who can meaningfully quantity and forecast future medium term AI progress so that you can hire them to help your ongoing project. Alas all you get is the comments' section. :-)
> but there's also no good reason to believe progress will stop, because a) there's many low and mid-hanging fruits to pick, as outlined by GP, and b) because it never did so far, so why would it stop now specifically?
Specifically, due to lack of data. Please refer to the earlier comment[^0]: deep learning requires vast amounts of data. Current models have already been trained on the entire internet and corpus of published human knowledge. Models are now being trained on synthetic data and we're running out of that too. This data bottleneck has been widely reported and documented.
> If I, for instance, tell my wife I can improve our income by 3x or 1000x but I don't really know, there's no planning that can be done and I'll probably have to sleep on the couch until I figure out what the hell I'm doing.
For most people, even a mere 3x in the next 5 years is huge, it's 25% per year growth.
3x in 5 years is a reasonable low-ball for hardware improvements alone. Caveat: top-end silicon is now being treated as a strategic asset, so there may be wars over it, driving up prices and/or limiting progress, even on the 5-year horizon.
I'm unclear why your metaphor would have you sleeping on the sofa: If tonight you produce a business idea for which you can be 2σ-confident that it will give you an income 5 years from now in the range [3…1000]x, you can likely get a loan for a substantially bigger house tomorrow than you were able to get yesterday; in the UK that's a change slightly larger than going from the median average full-time salary to the standard member of parliament salary.
(The reason behind this, observed lowering of compute costs, has been used even decades ago to delay investment in compute until the compute was cheaper).
The arguments I've seen elsewhere for order-of-10,000x* cost improvements (which is a proxy for efficiency and speed if not IQ) is based on various different observations cost reductions** since ChatGPT came out — personally, I doubt that the high end of that would come to pass, my guess is those all represent low-hanging fruit that can't be picked twice, but even then I would still expect there to be some opportunity for further gains.
* The original statement had one more digit in it than yours, but this doesn't make much difference to the argument either way
Also office jobs will be adapted to be a better fit to what AI can do, just as manufacturing jobs were adapted so that at least some tasks could be completed by robots.
Not my downvote, just the opposite but I think you can do a lot in an office already if you start early enough . . .
At one time I would have said you should be able to have an efficient office operation using regular typewriters, copiers, filing cabinets, fax machines, etc.
And then you get Office 97, zip through everything and never worry about office work again.
I was pretty extreme having a paperless office when my only product is paperwork, but I got there. And I started my office with typewriters, nice ones too.
Before long Google gets going. Wow. No-ads information superhighway, if this holds it can only get better. And that's without broadband.
But that's besides the point.
Now it might make sense for you to at least be able to run an efficient office on the equivalent of Office 97 to begin with. Then throw in the AI or let it take over and see what you get in terms of output, and in comparison. Microsoft is probably already doing this in an advanced way. I think a factor that can vary over orders of magnitude is how does the machine leverage the abilities and/or tasks of the nominal human "attendant"?
One type of situation would be where a less-capable AI could augment a defined worker more effectively than even a fully automated alternative utilizing 10x more capable AI. There's always some attendant somewhere so you don't get a zero in this equation no matter how close you come.
Could be financial effectiveness or something else, the dividing line could be a moving target for a while.
You could even go full paleo and train the AI on the typewriters and stuff just to see what happens ;)
But would you really be able to get the most out of it without the momentum of many decades of continuous improvement before capturing it at the peak of its abilities?
For me, general intelligence from a computer will be achieved when it knows when it's wrong. You may say that humans also struggle with this, and I'd agree - but I think there's a difference between general intelligence and consciousness, as you said.
Being wrong is one thing, on the other hand knowing that they don't know something is something humans are pretty good at (even if they might not admit to not knowing something and start bullshitting anyways). Current AI predictably fails miserably every single time.
> knowing that they don't know something is something humans are pretty good at (even if they might not admit to not knowing something and start bullshitting anyways)
I'd like to believe this, but I'm not a mind reader and I feel like the last decade has eroded a lot of my trust in the ability of adults to know when they're wrong. I still have hope for children, at least.
I see this post getting trashed in the comments for its overly literal interpretation of personality as a reinforcement learning process, but I think there's some value to it as a mental model of how we operate (which is how the opening sentence describes it).
If you can see past some of the more dubious, overly technical-sounding details and treat it as a metaphor, there is for sure a "behavioral landscape" that we all find ourselves in, filled with local minimal, attractors/basins and steep hills to climb to change our own behaviors.
Thinking about where you are and where you want to be in the behavior landscape can be a useful mental model. Habit changes like exercise and healthy eating, for example, can be really steep hills to climb (and easy to fall back down), but once you get over the hump, you may find yourself in a much better behavioral valley and wonder how you were stuck in the other place for so long.
the essential idea is that personality is malleable, there are concepts in NNs that are analogous to experience we can use to name, deconstruct, orient, and contrast, and as a way to exercise some agency over our own personalities.
you can choose it, and like "the five monkeys experiment"[1] after a while, you don't remember the things you don't believe anymore.
the author used trauma, env change, extreme experiences and psychedelics as examples, but something as simple as reading a book or a comment on a forum can detach us from beliefs and ideas that moored our personality in a local basin. we are the effects of feedback, so change your feedback.
This kind of mental model (some variation of it of course), despite being trashed by the majority, is now becoming the zeitgeist in tech circles. I could have written a very similar blog post (modulo worse writing and more being made fun of), and so could many engineers working on AI. The meta implications of this on the larger society could be interesting.
There's an additional aspect to the dynamics, which is that the social spaces you put yourself in change the landscape to discourage deviancy from the norm. You become like the people you spend time with.
Of course, other kinds of life are possible, which would require other "just right" conditions - but in that case, we'd all be living in that universe, marveling at those coincidences!
Although that's really selling it short - it's so much more than that! But in the context of this conversation, it's a good place to look.