I will find this often-repeated argument compelling only when someone can prove to me that the human mind works in a way that isn't 'combining stuff it learned in the past'.
5 years ago a typical argument against AGI was that computers would never be able to think because "real thinking" involved mastery of language which was something clearly beyond what computers would ever be able to do. The implication was that there was some magic sauce that human brains had that couldn't be replicated in silicon (by us). That 'facility with language' argument has clearly fallen apart over the last 3 years and been replaced with what appears to be a different magic sauce comprised of the phrases 'not really thinking' and the whole 'just repeating what it's heard/parrot' argument.
I don't think LLM's think or will reach AGI through scaling and I'm skeptical we're particularly close to AGI in any form. But I feel like it's a matter of incremental steps. There isn't some magic chasm that needs to be crossed. When we get there I think we will look back and see that 'legitimately thinking' wasn't anything magic. We'll look at AGI and instead of saying "isn't it amazing computers can do this" we'll say "wow, was that all there is to thinking like a human".
> 5 years ago a typical argument against AGI was that computers would never be able to think because "real thinking" involved mastery of language which was something clearly beyond what computers would ever be able to do.
Mastery of words is thinking? In that line of argument then computers have been able to think for decades.
Humans don't think only in words. Our context, memory and thoughts are processed and occur in ways we don't understand, still.
There's a lot of great information out there describing this [0][1]. Continuing to believe these tools are thinking, however, is dangerous. I'd gather it has something to do with logic: you can't see the process and it's non-deterministic so it feels like thinking. ELIZA tricked people. LLMs are no different.
That's the crazy thing. Yes, in fact, it turns out that language encodes and embodies reasoning. All you have to do is pile up enough of it in a high-dimensional space, use gradient descent to model its original structure, and add some feedback in the form of RL. At that point, reasoning is just a database problem, which we currently attack with attention.
No one had the faintest clue. Even now, many people not only don't understand what just happened, but they don't think anything happened at all.
There's no such thing as people without language, except for infants and those who are so mentally incapacitated that the answer is self-evidently "No, they cannot."
Language is the substrate of reason. It doesn't need to be spoken or written, but it's a necessary and (as it turns out) sufficient component of thought.
There are quite a few studies to refute this highly ignorant comment. I'd suggest some reading [0].
From the abstract:
"Is thought possible without language? Individuals with global aphasia, who have almost no ability to understand or produce language, provide a powerful opportunity to find out. Astonishingly, despite their near-total loss of language, these individuals are nonetheless able to add and subtract, solve logic problems, think about another person’s thoughts, appreciate music, and successfully navigate their environments. Further, neuroimaging studies show that healthy adults strongly engage the brain’s language areas when they understand a sentence, but not when they perform other nonlinguistic tasks like arithmetic, storing information in working memory, inhibiting prepotent responses, or listening to music. Taken together, these two complementary lines of evidence provide a clear answer to the classic question: many aspects of thought engage distinct brain regions from, and do not depend on, language."
The resources that the brain is using to think -- whatever resources those are -- are language-based. Otherwise there would be no way to communicate with the test subjects. "Language" doesn't just imply written and spoken text, as these researchers seem to assume.
There’s linguistic evidence that, while language influences thought, it does not determine thought - see the failure of the strong Sapir-Whorf hypothesis. This is one of the most widely studied and robust linguistic results - we actually know for a fact that language does not determine or define thought.
How's the replication rate in that field? Last I heard it was below 50%.
How can you think without tokens of some sort? That's half of the question that has to be answered by the linguists. The other half is that if language isn't necessary for reasoning, what is?
We now know that a conceptually-simple machine absolutely can reason with nothing but language as inputs for pretraining and subsequent reinforcement. We didn't know that before. The linguists (and the fMRI soothsayers) predicted none of this.
Read about linguistic history and make up your own mind, I guess. Or don’t, I don’t care. You’re dismissing a series of highly robust scientific results because they fail to validate your beliefs, which is highly irrational. I'm no longer interested in engaging with you.
I've read plenty of linguistics work on a lay basis. It explains little and predicts even less, so it hasn't exactly encouraged me to delve further into the field. That said, linguistics really has nothing to do with arguments with the Moon-landing deniers in this thread, who are the people you should really be targeting with your advocacy of rationality.
In other words, when I (seem to) dismiss an entire field of study, it's because it doesn't work, not because it does work and I just don't like the results.
> ELIZA, ROFL. How'd ELIZA do at the IMO last year?
What's funny is the failure to grasp any contextual framing of ELIZA. When it came out people were impressed by it's reasoning, it's responses. And in your line of defense it could think because it had mastery of words!
But fast forward the current timeline 30 years. You will have been of the same camp that argued on behalf of ELIZA when the rest of the world was asking, confusingly: how did people think ChatGPT could think?
No one was impressed with ELIZA's "reasoning" except for a few non-specialist test subjects recruited from the general population. Admittedly it was disturbing to see how strongly some of those people latched onto it.
Meanwhile, you didn't answer my question. How'd ELIZA do on the IMO? If you know a way to achieve gold-medal performance at top-level math and programming competitions without thinking, I for one am all ears.
> I will find this often-repeated argument compelling only when someone can prove to me that the human mind works in a way that isn't 'combining stuff it learned in the past'.
Science is distributed. Lots of researchers at lots of different institutions research overlapping topics. That's part of its strength. In the U.S. most basic research is funded by federal grants. And as a results you'll find that research in pretty much any science area you can imagine is funded by federal grants going to multiple different institutions. In this case you're confusing things by bringing in NOAA which is a government agency (part of the Dept of Commerce). NCAR is a non-profit organization and competes for federal grant dollars with researchers at many other institutions (mostly universities). So in that sense there is a strong parallel here to Trump wanting to shut down Harvard (another non-profit organizations at which many different researchers work) and someone saying "doesn't Stanford do research on similar topics?" Yes, there is some conceptual overlap, but in detail there is not. The bigger difference is that Harvard has a big endowment and so can survive (at some level) if the federal grants it has been getting stop flowing. NCAR can't. Also, NCAR happens to have the experts and equipment (supercomputers) to do research that few other organizations can (none really in the U.S.). Harvard probably can't lay claim to that except in very narrow niches....
For perspective the annual budget for NCAR is about half the amount being spend on the new White House ballroom.
Given that it's under scrutiny for regulatory bypass, it's not a purchase and is being reviewed as circumventing those very rules. Might not even happen.
I know, I'm joking: Trump likes Nvidia, but maybe he'll bump the Chinese tax to 30% to approve this deal? In a way I hope he pulls something like that, to punish Huang for his boot shining manipulations.
The multiple meanings of many of the words in this sentence make it really poor at communicating what the site is about. "Endeavour" (with a capital 'E') is a proper name I associate with a space shuttle, and 'stellar' can mean 'having to do with stars'. So a first read for me leads to the conclusion that this site has something to do with space flight. And 'system' could mean almost anything. Maybe this site will let me personalize my own star system? All I can take away is that I'm not sure what this is, but clearly I'm not the target audience. Which I'm fine with.....
Or it doesn't. Because "software as an organic thing" like all analogies is an analogy, not truth. Systems can sit there and run happily for a decade performing the needed function in exactly the way that is needed with no 'rot'. And then maybe the environment changes and you decide to replace it with something new because you decide the time is right. Doesn't always happen. Maybe not even the majority of the time. But in my experience running high-uptime systems over multiple decades it happens. Not having somebody outside forcing you to change because it suits their philosophy or profit strategy is preferrable.
Or more likely the 'whole' accesses the stable bit through some interface. The stable bit can happily keep doing it's job via the interface and the whole can change however it likes knowing that for that particular tasks (which hasn't changed) it can just call the interface.
If a hard drive sometimes fails, why would a raid with multiple hard drives be any more reliable?
"Do task x" and "Is this answer to task x correct?" are two very different prompts and aren't guaranteed to have the same failure modes. They might, but they might not.
RAID only works when failures are independent. E. g. if you bought two drivers from the same faulty batch which die after 1000 power-on hours RAID would not help. With LLM it’s not obvious that errors are not correlated.
> If a hard drive sometimes fails, why would a raid with multiple hard drives be any more reliable?
This is not quite the same situation. It's also the core conceit of self-healing file systems like ZFS. In the case of ZFS it not only stores redundant data but redundant error correction. It allows failures to not only be detected but corrected based on the ground truth (the original data).
In the case of an LLM backstopping an LLM, they both have similar probabilities for errors and no inherent ground truth. They don't necessarily memorize facts in their training data. Even with a RAG the embeddings still aren't memorized.
It gives you a constant probability for uncorrectable bullshit. One of the biggest problems with LLMs is the opportunity for subtle bullshit. People can also introduce subtle errors recalling things but they can be held accountable when that happens. An LLM might be correct nine out of ten times with the same context or only incorrect given a particular context. Even two releases of the same model might not introduce the error the same way. People can even prompt a model to error in a particular way.
At the very largest universities there is a really really wide variety of programs and courses. For example here's a course catalog where a search for 'intro' returns 3500 different courses.
https://classes.osu.edu/#/?q=intro&client=class-search-ui&ca...
You can see the variety, from "Introduction to the Army and Critical Thinking" to "Introductory Meat Science"
This breadth is typical of the very largest universities in the U.S.
Yeah, but just to be clear there's only ~1500 classes offered there. The 3000 comes from many of the classes having multiple components, lab etc.
The number is further reduced by the fact that many of them are the same class with special qualifications to ensure placements. For example intro classes for designated transfer students. So you have the same class but 10 seats or something are only available to certain transfer program students.
The real number looks like somewhere in the ~400-600 range. Which is still very impressive but 3500 different intro subjects would be wildly excessive.
Even in science there is not a 'requirement' that you have a controlled experiment in order to have evidence that a claim is true. Following your argument you can't substantiate that humans are the result of evolution because we can't take two groups of early primates, subject one to evolutionary forces and the other not and see what happens. Instead we can observe a chain of correlations with plausible mechanisms that indicate causation and say it's evidentiary. For example, data that indicates unvaccinated people died at a higher rate and data that indicates people who chose not to vaccinate self-report that the reason they made that choice was based on particular information that they believed. That would be evidence that helps substantiate the theory the information led to deaths. It's not 'proof'. We can't 'prove' that exposure to the information actually led to the decision (because people sometimes misattribute their own decisions) and it would be impractical to imagine we can collect vaccine-decision rationales from a large number of folks pre-death (though someone might have) and you can't attribute a particular death to a particular decision (because vaccines aren't perfectly protective) so you have to do statistics over a large sample. But the causal chain is entirely plausible based on everything I know and there's no reason to believe data around those correlations can't exist. And science isn't about 'proof'. Science is about theories that best explain a set of observations and in particular have predictive power. You almost never run experiments (in the 8th grade science fair sense) in fields like astronomy or geology, but we have strong 'substantiated' theories in those fields nonetheless.
A causal chain being plausible does not justify or substantiate a claim of causation.
I absolutely would say that we can't prove humans are the result of evolution. The theory seems very likely and explains what we have observed, but that's why its a theory and not a fact - its the last hypothesis standing and generally accepted but not proven.
My argument here isn't with whether the causation seemed likely, though we can have that debate if you prefer and we'd have to go deep down the accuracy and reliability of data reporting during the pandemic.
My argument is that we can't make blanket statements that misinformation killed people. Not only is that not a proven (or provable) fact, it skips past what we define as misinformation and ignores what was known at the time in favor of what we know today. Even if the data you to point to shows correlation and possible causation today, we didn't have that information during the pandemic st the time that YouTube was pulling down content for questioning efficacy or safety.
5 years ago a typical argument against AGI was that computers would never be able to think because "real thinking" involved mastery of language which was something clearly beyond what computers would ever be able to do. The implication was that there was some magic sauce that human brains had that couldn't be replicated in silicon (by us). That 'facility with language' argument has clearly fallen apart over the last 3 years and been replaced with what appears to be a different magic sauce comprised of the phrases 'not really thinking' and the whole 'just repeating what it's heard/parrot' argument.
I don't think LLM's think or will reach AGI through scaling and I'm skeptical we're particularly close to AGI in any form. But I feel like it's a matter of incremental steps. There isn't some magic chasm that needs to be crossed. When we get there I think we will look back and see that 'legitimately thinking' wasn't anything magic. We'll look at AGI and instead of saying "isn't it amazing computers can do this" we'll say "wow, was that all there is to thinking like a human".
reply