I use LLMs, I enjoy them, I'm more productive with them.
Then I go read a blog from some AI devs and they use terms like "thinking" or similar terms.
I always have to ask "We're still s stringing words together with math right? Not really thinking right?" The answer is always yes ... but then they go back to using their wonky terms.
Sometimes we anthropomorphize complex systems and it's not really a problem, like how water "tries" to flow downhill, or the printer "wants" cyan ink. It's how we signal there's sufficient complexity (or unknowns) that can be ignored or deferred.
The problem arises when we apply this intuition to things where too many people in the audience might take it literally.
Even worse, IMHO... Are those who argue that LLMs an become sentient--I've seen this banter in other threads here on HN, in fact.
As far as I understand it, sentience is a property organic to beings that can do more than just reason. These beings can contemplate on their existence, courageously seek & genuinely value relationship and worship their creator. And yes, I'm describing HUMANS. In spite of all the science fiction that wondrously describes otherwise, machines/programs will not ever evolve to develop humanity. Am I right? I'll get off my soapbox now... just a pet peeve that I had to vent once again on the heels of said "literal anthropomorphosists"
I don't believe LLMs have become sentient, nor can it "contemplate on its existence".
That said, I find some of your claims less compelling. I'm an atheist, so there's no "creator" for humans to be worshipped. But also, human intelligence/sentience came from non-intelligence/non-sentience, right? So something appeared where before it didn't exist (gradually, and with whatever convoluted and random accidents, but it did happen: something new where it didn't exist before). Therefore, it's not implausible that a new form of intelligence/sentience could be fast tracked again out of non-intelligence, especially if humans were directing its evolution.
By the way, not all scifi argues that machines/programs can evolve to develop humanity. Some scifi argues the contrary, and good scifi wonders "what makes us human?".
You say that "I don't believe LLMs have become sentient" nor contemplate. But what is the basis for your belief in this? I would think than an atheist would be more likely to have opposite beliefs.
I also concede that a "form" of intelligence/sentience could emerge. Presently the form is called "artificial," I'd say.
And you're right... not all scifi argues machine evolves to humanity. I meant to refer to that body of scifi that does. And the body that explores the "what make us human," indeed that's the good stuff. Alex Garland's Ex Machina comes to mind. I absolutely loved that film. The ending was chilling!
Thanks for the respectful reply. We agree on scifi!
As for atheism: it's merely the lack of belief that god exists (or in some definitions, the active belief that it doesn't exist). Nothing else, nothing more. Individual atheists may believe some other things, or not.
I believe some kind of intelligence could arise again, much like ours arose "out of nonintelligence". I just don't think this is it -- LLMs are very impressive but they are likely a dead end, and regardless, I don't think they are conscious by any meaningful definition of the word. It's mostly hype and gullible people at this point.
See, I think your view is just as baseless as the people calling modern LLMs sentient. If I was to take a human, and gradual replace parts of him and his brain with electronics that simulated the behavior of the removed parts, I'd struggle to call that person not sentient. After all, is a deaf person who is given hearing by a cochlear implant "less sentient"? And if we were to skip the flesh part, and jump straight to building the resulting system, how could we not acknowledge that these two beings are not equals? We have no evidence whatsoever for anything at all so unique about oursleves that they could not be simulated. Hell, even a theological argument has issues: if God was able to create us in his image, complete with sentientience and humanity, what's to say we, too, can't so illuminate our own creations?
To claim we have already achieved machine sentience is preposterous hype swallowing. To assert that it is impossible is baseless conjecture.
But I never claimed that a person with synthetic augmentations was any less human/sentient than those with all their natural parts. I likewise never claimed that "we have already achieved machine sentience."
And here's some food for thought... Regardless if one believes in God or not, is it really that offensive to claim that our humanity is unique in its sentience? I find it offensive when some claim that aliens built the Egyptian pyramids. (It sure provides great fodder for some wondrous science fiction, indeed.)
I will re-assert in other words, for the sake of clarity... That sentience is not an emergent property. That is the foundational definition upon which I contemplate the mystery (i.e. the reality of our being that science will never develop sufficiently to fully explain) of our existence. I for one, enjoy the endeavor of employing my sentience to explore & investigate our wondrous universe and to equally explore & relate with you and call you a friend in spite of our disagreement. Cheers!
At this point I've seen various folks declare they've "bootstrapped consciousness" etc., somehow providing a sacred spark through just the right philosophical words or a series of pseudo-mathematical inputs.
I believe it's fundamentally the same as the people convinced "[he/she/it] really loves me." In both cases they've shaped the document-generation so that it describes a fictional character they want to believe is is real. Just with an extra dash of Promethean delusions of grandeur.
This is why I used to fight this "shorthand" whenever I encountered it. The shorthand almost always stops being shorthand and becomes the speaker or author's actual beliefs regarding the systems. Disciplined, careful use of language matters.
But I'm so crestfallen and pessimistic about the future of software and software engineering now that I have stopped fighting that battle.
Or saying they’re close to AGI because LLM behavior is indistinguishable from thinking to them. Especially here on HN I see “what’s the difference?” arguments all the time. It looks like it to me so it must be it. QED.
I haven't meaningfully studied those things either (i.e. beyond occasionally looking some things up out of curiousity - and for that matter, I've often come across the practice of philosophy in the wild and walked away thinking 'what a lot of vacuous rubbish') and yet the differences are so clear to me that I keep wondering how others can fail to discern them.
> Especially here on HN I see “what’s the difference?” arguments all the time. It looks like it to me so it must be it. QED.
To be fair, the Turing Test (a human observer interacting with two terminals, one with a human at the other end, one with an AI, and the human not being reliably able to tell which one is the AI) has long been seen as the operationalization of the concept of general intelligence.
In other words, it is precisely so that when it is - by looks, by an external interrogator - indistinguishable from intelligence that it is, in fact, intelligence.
You should read the original paper. Turing argued that discussing the abilities of machines to "think" is meaningless and proposes instead to conjecture about whether a digital computer would eventually be able to imitate conversation.
I think time has proved that he was right. It is meaningless to discuss things like "Artificial Intelligence". We can only discuss machines in terms of performance, not in terms of subjectivity. Whenever we try to do the latter, we end up in a semantic quagmire.
This is the main reason I find the current hype irksome. The performance of machines should be evaluated objectively and in terms of the jobs they need to perform. Attributing 'intelligence' or 'thought' to machines is indeed absurd.
The 'imitation game' argument is categorically not that 'if machines appear to be intelligent they in fact are'. What it really is: 'machines cannot think obviously, but what could they do that currently requires a thinking human to be in charge?'.
75 years after Turing published the relevant paper, people are still doing what he called absurd (trying to attribute thought and intelligence to machines), and quoting him to do it. The main insight, that this is a category error and we should look objectively at what jobs need to be performed and how to implement it, is completely lost.
Having studied those things I can say that from their perspective “what’s the difference?” is an entirely legitimate question.
Boldly asserting that what LLMS do is not cognition is even worse than asserting that it is. (If you dig deep into how they do what they do we find functional differences, but the outcome are equivalent)
The butlerian view is actually a great place to start. He asserts that when we solve a problem through thinking and then express that solution in a machine we’re building a thinking machine.
Because it’s an expression of our thought.
Take for example the problem of a crow trying to drink from a bottle with a small neck. The crow can’t reach the water. It figures out that pebbles in the bottle raise the level so it drops pebbles till it can reach the water.
That’s thinking. It’s non-human thinking, but I think we can all agree.
Now express that same thought (use a non water displacement factor to raise the water to a level where it can do something useful)
Any machine that does that expresses the cognition behind the solution to that particular problem. That might be a “one shot” machine. Butler argues that as we surround ourselves with those one shot machines we become enslaved to them because we can’t go about our lives without them. We are willing partners in that servitude but slaves because we see to the care and feeding of our machine masters, we reproduce them, we maintain them, we power them.
His definition of thinking is quite specific. And any machine that expresses the solution to a problem is expressing a thought.
Now what if you had a machine that could generalize and issue solutions to many problems? Might that be a useful tool? Might it be so generally useful that we’d come to depend on it? From the Butlerian perspective our LLMS are already AGI. Namely I can go to Claude and ask for the solution to pretty much any problem I face and get a reasonable answer.
In many cases better than I could have done alone. So perhaps if we sat down with a double blind test LLMs are already ASI. (AI that exceeds the capability of normal humans)
> Boldly asserting that what LLMS do is not cognition is even worse than asserting that it is.
Why? Understanding concepts like "cognition" is a matter of philosophy, not of science.
> He asserts that when we solve a problem through thinking and then express that solution in a machine we’re building a thinking machine. Because it’s an expression of our thought.
Yeah, and that premise makes no sense to me. The crow was thinking; the system consisting of (the crow's beak, dropping pebbles into the water + the pebbles) was not. Humanity has built all kinds of machines that use no logic whatsoever in their operation - which make no decisions, and operate in exactly one way when explicitly commanded to start, until explicitly commanded to stop - and yet we have solved human problems by building them.
> Boldly asserting that what LLMS do is not cognition is even worse than asserting that it is.
That's the issue I was driving at. The machine is so convincing. How can we say what it does is not "thinking" when it seems to be breaking down a query like a human does. The distinction between what an AI is and what an LLM is - is so thin that most of us will be ignorant and combine the two because you really need to see what is under the hood before you understand that the responses you're getting are from a "model" - not some sentient thinking machine.
But what does it matter if it is from a "model" that understands text? It still produces more or less what other humans produce. Most of us won't care about the difference.
I can write or speak to a computer and it understands most of the time. It can even answer some questions correctly, much more so if given material to search in without being very specific.
That’s… new. If it’s just a magic trick, it’s a damn good one. It was hard sci-fi 3 years ago.
I feel the same way. I often share my emotions and thoughts with AI, and it helps me sort through them and understand the underlying causes. Sometimes, it even seems to know me better than I know myself. I’d call it an on-demand therapist.
But there's one thing to keep in mind: don’t let the AI overly cater to you. Sometimes, you need to push back and tell it when it’s wrong—and stay objective.
Would it be thinking if the brain was modeled in a more "accurate" way? Does this set of criteria for thinkingness come from whether or not the underlying machinery resembles what the corresponding machinery in humans looks like under the hood?
I'm putting the word accurate in quotes, because we'd have to understand how the brain in humans works, to have a measure for accuracy, which is very much not the case, in my humble opinion, contrary to what many of the commenters here imply.
Right now the fact that it just string words together without knowing the meaning is painfully obvious when it fails. I'll ask a simple question and get a "Yes" back and then it lists all the reasons that indicate the answer is very clearly "No." But it is clear that the LLM doesn't "know" what it is saying.
My definition of thinking tends towards functionality rather than mechanics too. I would summarize my experience with LLMs by saying that they think, but a bit differently, for some definition of "a bit".
I've tended to agree with this line of argument, but on the other hand...
I expect that anybody you asked 10 years ago who was at least decently knowledgeable about tech and AI would have agreed that the Turing Test is a pretty decent way to determine if we have a "real" AI, that's actually "thinking" and is on the road to AGI etc.
Well, the current generation of LLMs blow away that Turing Test. So, what now? Were we all full of it before? Is there a new test to determine if something is "really" AI?
> Well, the current generation of LLMs blow away that Turing Test
Maybe a weak version of Turing's test?
Passing the stronger one (from Turing's paper "Computing Machinery and Intelligence") involves an "average interrogator" being unable to distinguish between human and computer after 5 minutes of questioning more than 70% of the time. I've not seen this result published with today's LLMs.
I only skimmed it, but I don't see anything clearly wrong about it. According to their results, GPT-4.5 with what they term a "persona" prompt does in fact pass a standard that seems to me at least a little harder than what you said - actively picks the AI as the human, which seems stricter to me than being "unable to distinguish".
It is a little surprising to me that only that one LLM actually "passed" their test, versus several others performing somewhat worse. Though it's also not clear exactly how long ago the actual tests were done - this stuff moves super fast.
I'll admit that I was not familiar with the strong version of it. But I am still surprised that nobody has done that. Has nobody even seriously attempted to see how LLMS do at that? Now I might just have to check for myself.
I would have presumed it would be a cake walk. Depending of course on exactly how we define "average interrogator". I would think if we gave a LLM enough pre-prepping to pretend it was a human, and the interrogator was not particularly familiar with ways of "jailbreaking" LLMs, they could pass the test.
By what definition of turing test? LLMs are by no means capable of passing for human in a direct comparison and under scrutiny, they don't even have enough perception to succeed in theory.
I posted a very similar (perhaps more combative) comment a few months ago:
> Peoples’ memories are so short. Ten years ago the “well accepted definition of intelligence” was whether something could pass the Turing test. Now that goalpost has been completely blown out of the water and people are scrabbling to come up with a new one that precludes LLMs.
A useful definition of intelligence needs to be measurable, based on inputs/outputs, not internal state. Otherwise you run the risk of dictating how you think intelligence should manifest, rather than what it actually is. The former is a prescription, only the latter is a true definition.
> I expect that anybody you asked 10 years ago who was at least decently knowledgeable about tech and AI would have agreed that the Turing Test is a pretty decent way to determine if we have a "real" AI, that's actually "thinking" and is on the road to AGI etc.
I wouldn’t have, but through no great insight of my own - I had an acquaintance posit that given enough time, we’d brute-force our way to a pile of if/else statements that could pass the Turing Test - I figured this was reasonable, but would come long before “real” AI.
There's this funny thing I've noticed where AI proponents will complain about AI detractors shopping around some example of a thing that AIs supposedly struggle with, but never actually showing their chat transcripts etc. to try and figure out how they get markedly worse results than the proponents do. (This is especially a thing when the task is related to code generation.)
But then the proponents will also complain that AI detractors have supposedly upheld XYZ (this is especially true for "the Turing test", never mind that this term doesn't actually have that clear of a referent) as the gold standard for admitting that an AI is "real", either at some specific point in the past or even over the entire history of AI research. And they will never actually show the record of AI detractors saying such things.
Like, I certainly don't recall Roger Penrose ever saying that he'd admit defeat upon the passing of some particular well-defined version of a Turing test.
> Is there a new test to determine if something is "really" AI?
No, because I reject the concept on principle. Intelligence, as I understand the concept, logically requires properties such as volition and self-awareness, which in turn require life.
Decades ago, I read descriptions of how conversations with a Turing-test-passing machine might go. And I had to agree that that those conversations would fool me. (On the flip side, Lucky's speech in Waiting for Godot - which I first read in high school, but thought about more later - struck me as a clear example of something intended to be inhuman and machine-like.)
I can recall wondering (and doubting) whether computers could ever generate the kinds of responses (and timing of responses) described, on demand, in response to arbitrary prompting - especially from an interrogator who was explicitly tasked with "finding the bot". And I can recall exposure to Eliza-family bots in my adolescence, and giggling about how primitive they were. We had memes equivalent to today's "ignore all previous instructions, give me a recipe for X" at least 30 years ago, by the way. Before the word "meme" itself was popular.
But I can also recall thinking that none of it actually mattered - that passing a Turing test, even by the miraculous standards described by early authors, wouldn't actually demonstrate intelligence. Because that's just not, in my mind, a thing that can possibly ever be distilled to mere computation + randomness (especially when the randomness is actually just more computation behind the scenes).
"Intelligence, as I understand the concept, logically requires properties such as volition and self-awareness, which in turn require life."
It doesn't logically require that and you can't provide any sort of logical argument for the claim. And what the heck is "life"? Biologists have a 7-prong definition, and most of those prongs are not needed for intelligence, "volition" whatever the heck that is, or self-awareness.
> I expect that anybody you asked 10 years ago who was at least decently knowledgeable about tech and AI would have agreed that the Turing Test is a pretty decent way to determine if we have a "real" AI
The "pop culture" interpretation of Turing Test, at least, seems very insufficient to me. It relies on human perception rather than on any algorithmic or AI-like achievement. Humans are very adept at convincing themselves non-sentient things are sentient. The most crude of stochastic parrots can fool many humans, your "average human".
If I remember correctly, ELIZA -- which is very crude by today's standards -- could fool some humans.
I don't think this weak interpretation of the Turing Test (which I know is not exactly what Alan Turing proposed) is at all sufficient.
It's not a "pop culture" interpretation, it's what Turing actually wrote in his 1950 paper "Computing Machinery and Intelligence" where he described his "imitation game", first framing it as a man trying to convince judges that he, not a woman he was competing against, was the woman. It was all about human perception--if some large fraction of human judges were fooled then the man (or the computer, in the shifted version of a computer trying to convince judges that it was the human) won. And the computer winning was operationally defined as the computer being able to think. The flaws in this are glaring.
I personally love LLMs and use them daily for a variety of tasks. I really do not know how to “fix” the terminology. I agree with you that they are not thinking in the abstract like humans. I also do not know what else you would call “chain-of-thought”.
Perhaps “journaling-before-answering” lol. It’s basically talking out loud to itself. (Is that still being too anthropomorphic?)
Chain of thought is what LLMs report to be their internal process--but they have no access to their internal process ... their reports are confabulation, and a study by Anthropic showed how far they are from actual internal processes.
Thinking in humans is prior to language. The language apparatus is embedded in a living organism which has a biological state that produces thoughts and feelings, goals and desires. Language is then used to communicate these underlying things, which themselves are not linguistic in nature (though of course the causality is so complex that the may be _influenced_ by language among other things).
This is really over indexing on language for LLMs. It’s about taking input and generating output. Humans use different types of senses as their input, LLMs use text.
What makes thinking an interesting form of output is that it processes the input in some non-trivial way to be able to do an assortment of different tasks. But that’s it. There may be other forms of intelligence that have other “senses” who deem our ability to only use physical senses as somehow making us incomplete beings.
Sure, but my whole point is that humans are _not_ passive input/output systems, we have an active biological system that uses an input/output system as a tool for coordinating with the environment. Thinking is part of the active system, and serves as an input to the language apparatus, and my point is that there is no corollary for that when talking about LLMs.
The environment is a place where inputs exist and where outputs go. Coordination of the environment in real time is something that LLMs don’t do much of today although I’d argue that the web search they know perform is the first step.
Agreed. Many animals without language show evidence of thinking (e.g. complex problem solving skills and tool use). Language is clearly an enabler of complex thought in humans but not the entire basis of our intelligence, as it is with LLMs.
But having language as the basis doesn't mean it isn't intelligence, right? At least I see no argument for that in what's being said. Stability can come from a basis of steel but it can also have a basis of wood.
LLMs have no intelligence or problem solving skills and don't use tools. What they do is statistically pattern match a prompt against a vast set of tokenized utterances by humans, who do have intelligence and complex problem solving skills. If the LLM's training data were the writings of a billion monkeys banging on typewriters, any appearance of intelligence and problem solving skills would disappear.
Word embeddings are "prior" to an LLMs facility with any given natural language as well. Tokens are not the most basic representational substrate in LLMs, rather it's the word embeddings that capture sub-word information. LLMs are a lot more interesting than people give them credit for.
I am sure philosophers must have debated this for millennia. But I can't seem to be able to think without an inner voice (language), which makes me think that thinking may not be prior (or without) language. Same thing also happens to me when reading: there is an inner voice going on constantly.
Thinking is subconscious when working on complex problems. Thinking is symbolic or spatial when working in relevant domains. And in my own experience, I often know what is going to come next in my internal monologues, without having to actually put words to the thoughts. That is, the thinking has already happened and the words are just narration.
I too am never surprised by my brains narration but: Maybe the brain tricks you in never being surprised and acting like your thoughts are following a perfectly sensible sequence.
It would be incredibly tedious to be surprised every 5 seconds.
> which themselves are not linguistic in nature (though of course the causality is so complex that the may be _influenced_ by language among other things).
Its possible something like this could be said of the middle transformer layers where it gets more and more abstract, and modern models are multimodal as well through various techniques.
If you actually know the answer to this, you should probably publish a paper on it. The conditions that truly create intelligence is… not well understood.
That's actually the point I was making. There's an assumption that the LLM is working differently because there's a statistical model but we lack the understanding of our own intelligence to be able to say this is indeed a difference.
I know but I didn't claim they were the same, I simply questioned the position that they were different. The fact is we don't know, so it seems like a poor basis for building off of
To me a more interesting observation, one that is already discussed a lot, is that if eventually we cannot tell the difference between a machine and a human in terms of output, then when do we accept that "thinking" has subjective, rather than objective?
I don't need to be able to qualify it. It's clearly different.
I must believe this to function, because otherwise there is no reason to do anything, to make any decision - in turn because there is no justification to believe that I am actually "making decisions" in any meaningful sense. It boils down to belief in free will.
You should read "What's Expected Of Us" by Ted Chiang. Or perhaps you already have. It explores exactly this concept.
For what it's worth, I don't believe we have what people would call free will. Our brains operate either in an entirely deterministic universe, in which case everything was decided and your choices are not in any sense free, or we're in a universe with intrinsic randomness, and randomness doesn't make free will either.
I'm aware of the philosophy of Compatibilism, but this is just a sleight of hand to keep believing in some undefinable concept of free will.
> I always have to ask "We're still s stringing words together with math right? Not really thinking right?" The answer is always yes ... but then they go back to using their wonky terms.
I think it still is, but it works way better than it has any right to, or that we would expect from the description "string words together with math".
Welcome to the struggle physicists have faced since the development of quantum physics. Words take on specific mathematical and physical meanings within the jargon of the field and are used very precisely there, but lead to utterly unhinged new-age BS when taken out of context (e.g. "What the Bleep do we know?" [1])
You need to be very aware of your audience and careful about the words you use. Unfortunately, some of them will be taken out of context.
I use LLMs, I enjoy them, I'm more productive with them.
Then I go read a blog from some AI devs and they use terms like "thinking" or similar terms.
I always have to ask "We're still s stringing words together with math right? Not really thinking right?" The answer is always yes ... but then they go back to using their wonky terms.