Astral Codex Ten

neuroscience

26 min read

neuroscience

Article

neuroscience is a recurring concept in the Astral Codex Ten archive, appearing 3 times across 3 issues between April 14, 2021 and July 19, 2024. The archive places it in contexts such as “This paper permanently fuses artificial intelligence and neuroscience into a single mathematical field”; “borrows much from psychology, neuroscience, and evolutionary biology”; “dominance of Chomskyan linguistics has had downstream effects in adjacent fields like … neuroscience”. It most often appears alongside artificial intelligence, evolutionary biology, ACX.

Metadata

  • Category: Concepts
  • Mention count: 3
  • Issue count: 3
  • First seen: April 14, 2021
  • Last seen: July 19, 2024

Appears In

Source Context

Recovered passages from the original issue text. When the raw archive preserved outbound links inside the source passage, they are listed directly under the quote.

[LINK] Unifying Predictive Coding With Backpropagation
April 14, 2021 · Original source
This paper permanently fuses artificial intelligence and neuroscience into a single mathematical field.
I’m not sure I am as excited as they are; AI and neuroscience have always been a single field in some spiritual sense, and they continue to be very different fields in practice. And I’m not sure what good more neuromorphic computers would be - lsusr suggests “a computer that doesn’t break when you cut it in half”, but it sounds easier to just avoid letting your computer end up in a situation where that might matter.
Your Book Review: Are We Smart Enough To Know How Smart Animals Are?
April 22, 2021 · Original source
Are We Smart Enough gives an overview of the study of animal cognition, its history, past and current controversies, and where the field might go next (as of 2016). "Animal Cognition" is a relatively new subfield that borrows much from psychology, neuroscience, and evolutionary biology, but it is firmly rooted in its parent field of ethology. So, just what is ethology? It is the study animal behavior. Ethologists these days are certainly knowledgeable about evolutionary lineages, cladistics, and animal anatomy, but the heart of the field has always been observing animals interacting with their environments (whether a wild or experimental environment). As we'll see, it's a field with a history at least as tumultuous as psychology, with an early amateurish "wild west", a strict over-correction, and these days a hopefully more productive synthesis.
Okay, so the book doesn't actually talk about predictive processing as such, but since it was about cognition, I couldn't help but think about it. Full disclosure, almosteverything I know about Predictive Processing I learned on Slate Star Codex, so take this with a grain of salt. Given that predictive processing purports to describe what is happening at a really low level, if correct, it almost certainly is how other vertebrate cognition works. This might open up new avenues for experiment and observation that aren't open with humans (no, this isn't a call for vivisection or anything, just building on what has been described above about animals having widely varying cognition that can be tested in different ways). Perhaps even more interesting would be to try to figure out if predictive processing explains the cognition of non-vertebrates like octopodes. I don't really have much further insight on this, but this sounds like yet another place where the fields of (human) psychology/neuroscience and animal cognition could usefully inform each other.
Inline links: almost, everything, know
Your Book Review: How Language Began
July 19, 2024 · Original source
This is what he looks like. According to Wikipedia, the context for this picture is: “Noam Chomsky speaks about humanity's prospects for survival” Since around 1957, Chomsky has dominated linguistics. And this matters because he is kind of a contrarian with weird ideas. Is language for communicating? No, it’s mainly for thinking: (What Kind of Creatures Are We? Ch. 1, pg. 15-16) It is, indeed, virtual dogma that the function of language is communication. ... there is by now quite significant evidence that it is simply false. Doubtless language is sometimes used for communication, as is style of dress, facial expression and stance, and much else. But fundamental properties of language design indicate that a rich tradition is correct in regarding language as essentially an instrument of thought, even if we do not go as far as Humboldt in identifying the two. Should linguists care about the interaction between culture and language? No, that’s essentially stamp-collecting: (Language and Responsibility, Ch. 2, pg. 56-57) Again, a discipline is defined in terms of its object and its results. Sociology is the study of society. As to its results, it seems that there are few things one can say about that, at least at a fairly general level. One finds observations, intuitions, impressions, some valid generalizations perhaps. All very valuable, no doubt, but not at the level of explanatory principles. … Sociolinguistics is, I suppose, a discipline that seeks to apply principles of sociology to the study of language; but I suspect that it can draw little from sociology, and I wonder whether it is likely to contribute much to it. … You can also collect butterflies and make many observations. If you like butterflies, that’s fine; but such work must not be confounded with research, which is concerned to discover explanatory principles of some depth and fails if it has not done so. Did the human capacity for language evolve gradually? No, it suddenly appeared around 50,000 years ago after a freak gene mutation: (Language and Mind, third edition, pg, 183-184) An elementary fact about the language faculty is that it is a system of discrete infinity, rare in the organic world. Any such system is based on a primitive operation that takes objects already constructed, and constructs from them a new object: in the simplest case, the set containing them. Call that operation Merge. Either Merge or some equivalent is a minimal requirement. With Merge available, we instantly have an unbounded system of hierarchically structured expressions. The simplest account of the “Great Leap Forward” in the evolution of humans would be that the brain was rewired, perhaps by some slight mutation, to provide the operation Merge … There are speculations about the evolution of language that postulate a far more complex process … A more parsimonious speculation is that they did not, and that the Great Leap was effectively instantaneous, in a single individual, who was instantly endowed with intellectual capacities far superior to those of others, transmitted to offspring and coming to predominate. At best a reasonable guess, as are all speculations about such matters, but about the simplest one imaginable, and not inconsistent with anything known or plausibly surmised. It is hard to see what account of human evolution would not assume at least this much, in one or another form. I think all of these positions are kind of insane for reasons that we will discuss later. (Side note: Chomsky’s proposal is essentially the hard takeoff theory of human intelligence.) Most consequential of all, perhaps, are the ways Chomsky has influenced (i) what linguists mainly study, and (ii) how they go about studying it. Naively, since language involves many different components—including sound production and comprehension, intonation, gestures, and context, among many others—linguists might want to study all of these. While they do study all of these, Chomsky and his followers view grammar as by far the most important component of humans’ ability to understand and produce language, and accordingly make it their central focus. Roughly speaking, grammar refers to the set of language-specific rules that determine whether a sentence is well-formed. It goes beyond specifying word order (or ‘surface structure’, in Chomskyan terminology) since one needs to know more than just where words are placed in order to modify or extend a given sentence. Consider a pair of sentences Chomsky uses to illustrate this point in Aspects of the Theory of Syntax (pg. 22), his most cited work: (1a) I expected John to be examined by a specialist. (2a) I persuaded John to be examined by a specialist. The words “expected” and “persuaded” appear in the same location in each sentence, but imply different ‘latent’ grammatical structures, or ‘deep structures’. One way to show this is to observe that a particular way of rearranging the words produces a sentence with the same meaning in the first case (1a = 1b), and a different meaning in the second (2a != 2b): (1b) I expected a specialist to examine John. (2b) I persuaded a specialist to examine John. In particular, the target of persuasion is “John” in the case of (2a), and “the specialist” in the case of (2b). A full Chomskyan treatment of sentences like this would involve hierarchical tree diagrams, which permit a precise description of deep structure. You may have encountered the famous sentence: “Colorless green ideas sleep furiously.” It first appeared in Chomsky’s 1957 book Syntactic Structures, and the point is that even nonsense sentences can be grammatically well-formed, and that speakers can quickly assess the grammatical correctness of even nonsense sentences that they’ve never seen before. To Chomsky, this is one of the most important facts to be explained about language. A naive response to Chomsky’s preoccupation with grammar is: doesn’t real language involve a lot of non-grammatical stuff, like stuttering and slips of the tongue and midstream changes of mind? Of course it does, and Chomsky acknowledges this. To address this point, Chomsky has to move the goalposts in two important ways. First, he famously distinguishes competence from performance, and identifies the former as the subject of any serious theory of language: (Aspects of the Theory of Syntax, Ch. 1, pg. 4) The problem for the linguist, as well as for the child learning the language, is to determine from the data of performance the underlying system of rules that has been mastered by the speaker-hearer and that he puts to use in actual performance. Hence, in the technical sense, linguistic theory is mentalistic, since it is concerned with discovering a mental reality underlying actual behavior. Observed use of language or hypothesized dispositions to respond, habits, and so on, may provide evidence as to the nature of this mental reality, but surely cannot constitute the actual subject matter of linguistics, if this is to be a serious discipline. Moreover, he claims that grammar captures most of what we should mean when we talk about speakers’ linguistic competence: (Aspects of the Theory of Syntax, Ch. 1, pg. 24) A grammar can be regarded as a theory of a language; it is descriptively adequate to the extent that it correctly describes the intrinsic competence of the idealized native speaker. Another way Chomsky moves the goalposts is by distinguishing E-languages, like English and Spanish and Japanese, from I-languages, which only exist inside human minds. He claims that serious linguistics should be primarily interested in the latter. In a semi-technical book summarizing Chomsky’s theory of language, Cook and Newson write: (Chomsky’s Universal Grammar: An Introduction, pg. 13) E-language linguistics … aims to collect samples of language and then describe their properties. … I-language linguistics, however, is concerned with what a speaker knows about language and where this knowledge comes from; it treats language as an internal property of the human mind rather than something external … Not only should linguistics primarily be interested in studying I-languages, but to try and study E-languages at all may be a fool’s errand: (Chomsky’s Universal Grammar: An Introduction, pg. 13) Chomsky claims that the history of generative linguistics shows a shift from an E-language to an I-language approach; ‘the shift of focus from the dubious concept of E-language to the significant notion of I-language was a crucial step in early generative grammar’ (Chomsky, 1991b, pg. 10). … Indeed Chomsky is extremely dismissive of E-language approaches: ‘E-language, if it exists at all, is derivative, remote from mechanisms and of no particular empirical significance, perhaps none at all’ (Chomsky, 1991b, pg. 10).1 I Am Not A Linguist (IANAL), but this redefinition of the primary concern of linguistics seems crazy to me. Is studying a language like English as it is actually used really of no particular empirical significance? And this doesn’t seem to be a one-time hyperbole, but a representative claim. Cook and Newson continue: (Chomsky’s Universal Grammar: An Introduction, pg. 14) The opposition between these two approaches in linguistics has been long and acrimonious, neither side conceding the other’s reality. … The E-linguist despises the I-linguist for not looking at the ‘real’ facts; the I-linguist derides the E-linguist for looking at trivia. The I-language versus E-language distinction is as much a difference of research methods and of admissible evidence as it is of long-term goals. So much for what linguists ought to study. How should they study it? The previous quote gives us a clue. Especially in the era before Chomsky (BC), linguists were more interested in description. Linguists were, at least in one view, people who could be dropped anywhere in the world, and emerge with a tentative grammar of the local language six months later. (A notion like this is mentioned early in this video.) Linguists catalog the myriad of strange details about human languages, like the fact that some languages don’t appear to have words for relative directions, or “thank you”, or “yes” and “no”. After Chomsky's domination of the field (AD), there were a lot more theorists. While you could study language by going out into the field and collecting data, this was viewed as not the only, and maybe not even the most important, way to work. Diagrams of sentences proliferated. Chomsky, arguably the most influential linguist of the past hundred years, has never done fieldwork. In summary, to Chomsky and many of the linguists working in his tradition, the scientifically interesting component of language is grammar competence, and real linguistic data only indirectly reflects it. All of this matters because the dominance of Chomskyan linguistics has had downstream effects in adjacent fields like artificial intelligence (AI), evolutionary biology, and neuroscience. Chomsky has long been an opponent of the statistical learning tradition of language modeling, essentially claiming that it does not provide insight about what humans know about languages, and that engineering success probably can’t be achieved without explicitly incorporating important mathematical facts about the underlying structure of language. Chomsky’s ideas have motivated researchers to look for a “language gene” and “language areas” of the brain. Arguably, no one has yet found either—but more on that later. How Chomsky attained this stranglehold on linguistics is an interesting sociological question, but not our main concern in the present work2. The intent here is not to pooh-pooh Chomsky, either; brilliant and hard-working people are often wrong on important questions. Consider that his academic career began in the early 1950s—over 70 years ago!—when our understanding of language, anthropology, biology, neuroscience, and artificial intelligence, among many other things, was substantially more rudimentary. Where are we going with this? All of this is context for understanding the ideas of a certain bomb-throwing terrorist blight on the face of linguistics: Daniel Everett. How Language Began is a book he wrote about, well, what language is and how it began. Everett is the anti-Chomsky. II. THE MISSIONARY We all love classic boy-meets-girl stories. Here’s one: boy meets girl at a rock concert, they fall in love, the boy converts to Christianity for the girl, then the boy and girl move to the Amazon jungle to dedicate the rest of their lives to saving the souls of an isolated hunter-gatherer tribe. Daniel Everett is the boy in this story. The woman he married, Keren Graham, is the daughter of Christian missionaries and had formative experiences living in the Amazon jungle among the Sateré-Mawé people. At seventeen, Everett became a born-again Christian; at eighteen, he and Keren married; and over the next few years, they started a family and prepared to become full-fledged missionaries like Keren’s parents. First, Everett studied “Bible and Foreign Missions” at the Moody Bible Institute in Chicago. After finishing his degree in 1975, the natural next step was to train more specifically to follow in the footsteps of Keren’s parents. In 1976, he and his wife enrolled in the Summer Institute of Linguistics (SIL) to learn translation techniques and more viscerally prepare for life in the jungle: They were sent to Chiapas, Mexico, where Keren stayed in a hut in the jungle with the couple’s children—by this time, there were three—while Everett underwent grueling field training. He endured fifty-mile hikes and survived for several days deep in the jungle with only matches, water, a rope, a machete, and a flashlight. Everett apparently had a gift for language-learning. This led SIL to invite Everett and his wife to work with the Pirahã people (pronounced pee-da-HAN), whose unusual language had thwarted all previous attempts to learn it. In 1977, Everett’s family moved to Brazil, and in December they met the Pirahã for the first time. As an SIL-affiliated missionary, Everett’s explicit goals were to (i) translate the Bible into Pirahã, and (ii) convert as many Pirahã as possible to Christianity. But Everett’s first encounter with the Pirahã was cut short for political reasons: (Don’t Sleep There Are Snakes, Ch. 1, pg. 13-14) In December of 1977 the Brazilian government ordered all missionaries to leave Indian reservations. … Leaving the village under these forced circumstances made me wonder whether I’d ever be able to return. The Summer Institute of Linguistics was concerned too and wanted to find a way around the government’s prohibition against missionaries. So SIL asked me to apply to the graduate linguistics program at the State University of Campinas (UNICAMP), in the state of São Paulo, Brazil. It was hoped that UNICAMP would be able to secure government authorization for me to visit the Pirahãs for a prolonged period, in spite of the general ban against missionaries. … My work at UNICAMP paid off as SIL hoped it would. Everett became a linguist proper sort of by accident, mostly as an excuse to continue his missionary work. But he ended up developing a passion for it. In 1980, he completed Aspects of the Phonology of Pirahã, his master’s thesis. He continued on to get a PhD in linguistics, also from UNICAMP, and in 1983 finished The Pirahã Language and Theory of Syntax, his dissertation. He continued studying the Pirahã and working as an academic linguist after that. In all, Everett spent around ten years of his life living with the Pirahã, spread out over some thirty-odd years. As he notes in Don’t Sleep, There Are Snakes: (Prologue, pg. xvii-xviii) I went to the Pirahãs when I was twenty-six years old. Now I am old enough to receive senior discounts. I gave them my youth. I have contracted malaria many times. I remember several occasions on which the Pirahãs or others threatened my life. I have carried more heavy boxes, bags, and barrels on my back through the jungle than I care to remember. But my grandchildren all know the Pirahãs. My children are who they are in part because of the Pirahãs. And I can look at some of those old men (old like me) who once threatened to kill me and recognize some of the dearest friends I have ever had—men who would now risk their lives for me. Everett interviewing some Pirahã people. (source) Everett did eventually learn their language, and it’s worth taking a step back to appreciate just how hard that task was. No Pirahã spoke Portuguese, apart from some isolated phrases they used for bartering. They didn’t speak any other language at all—just Pirahã. How do you learn another group’s language when you have no languages in common? The technical term is monolingual fieldwork. But this is just a fancy label for some combination of pointing at things, listening, crude imitation, and obsessively transcribing whatever you hear. For years. It doesn’t help that the Pirahã language seems genuinely hard to learn in a few different senses. First, it is probably conventionally difficult for Westerners to learn since it is a tonal language (two tones: high and low) with a small number of phonemes (building block sounds) and a few unusual sounds3. Second, there is no written language. Third, the language has a variety of ‘channels of discourse’, or ways of talking specialized for one or another cultural context. One of these is ‘whistle speech’; Pirahãs can communicate purely in whistles. This feature appears to be extremely useful during hunting trips: (Don’t Sleep, There Are Snakes, Ch. 11, pg. 187-188) My first intense contact with whistle speech came one day when the Pirahãs had given me permission to go hunting with them. After we’d been walking for about an hour, they decided that they weren’t seeing any game because I, with my clunking canteens and machete and congenital clumsiness, was making too much noise. “You stay here and we will be back for you later.” Xaikáibaí said gently but firmly. … As I tried to make the best of my solitary confinement, I heard the men whistling to one another. They were saying, “I’ll go over there; you go that way,” and other such hunting talk. But clearly they were communicating. It was fascinating because it sounded so different from anything I had heard before. The whistle carried long and clear in the jungle. I could immediately see the importance and usefulness of this channel, which I guessed would also be much less likely to scare away game than the lower frequencies of the men’s normal voices. Fourth, important aspects of the language reflect core tenets of Pirahã culture in ways that one might not a priori expect. Everett writes extensively about the ‘immediacy of experience principle’ of Pirahã culture, which he summarizes as the idea that: (Don’t Sleep, There Are Snakes, Ch. 7, pg. 132) Declarative Pirahã utterances contain only assertions related directly to the moment of speech, either experienced by the speaker or witnessed by someone alive during the lifetime of the speaker. One way the language reflects this is that the speaker must specify how they know something by affixing an appropriate suffix to verbs: (Don’t Sleep, There Are Snakes, Ch. 12, pg. 196) Perhaps the most interesting suffixes, however (though these are not unique to Pirahã), are what linguists call evidentials, elements that represent the speaker’s evaluation of his or her knowledge of what he or she is saying. There are three of these in Pirahã: hearsay, observation, and deduction. To see what these do, let’s use an English example. If I ask you, “Did Joe go fishing?” you could answer, “Yes, at least I heard that he did,” or “Yes, I know because I saw him leave,” or “Yes, at least I suppose he did because his boat is gone.” The difference between English and Pirahã is that what English does with a sentence, Pirahã does with a verbal suffix. Everett also convincingly links this cultural principle to the lack of Pirahã number words and creation myths. On the latter topic, Everett recalls the following exchange: (Don’t Sleep, There Are Snakes, Ch. 7, pg. 134) I sat with Kóhoi once and he asked me, after hearing about my god, “What else does your god do?” And I answered, “Well, he made the stars, and he made the earth.” Then I asked, “What do the Pirahãs say?” He answered, “Well, the Pirahãs say that these things were not made.” And all of this is to say nothing of the manifold perils of the jungle: malaria, typhoid fever, dysentery, dangerous snakes, insects, morally gray river traders, and periodic downpours. If Indiana Jones braved these conditions for years, we would consider his stories rousing adventures. Everett did this while also learning one of the most unusual languages in the world. People on the bank of the Maici river. (source) By the way, he did eventually sort of achieve his goal of translating the Bible. Armed with a solid knowledge of Pirahã, he was able to translate the New Testament’s Gospel of Mark. Since the Pirahã have no written language, he provided them with a recorded version, but did not get the reaction he expected: (Don’t Sleep, There Are Snakes, Ch. 17, pg. 267-268) When we returned to the village, I recorded Mark’s gospel in my own voice for the Pirahãs to listen to. I then brought in a wind-up tape recorder to play the recording, and I taught the Pirahãs how to use it, which, surprisingly enough, some of the children did. Keren and I left the village and returned a few weeks later. The people were still listening to the gospel, with children cranking the recorder. I was initially quite excited about this, until it became clear that the only part of the book that they paid attention to was the beheading of John the Baptist. “Wow, they cut off his head. Play that again!” One reaction to hearing the gospel caught Everett even more off-guard: (Don’t Sleep, There Are Snakes, Ch. 17, pg. 269) "The women are afraid of Jesus. We do not want him." "Why not?" I asked, wondering what had triggered this declaration. "Because last night he came to our village and tried to have sex with our women. He chased them around the village, trying to stick his large penis into them." Kaaxaóoi proceeded to show me with his two hands held far apart how long Jesus's penis was—a good three feet. But the Pirahã had an even more serious objection to Jesus: (Don’t Sleep, There Are Snakes, Ch. 17, pg. 265-266) Part of the difficulty of my task began to become clear to me. I communicated more or less correctly to the Pirahãs about my Christian beliefs. The men listening to me understood that there was a man named Hisó, Jesus, and that he wanted others to do what he told them. "The Pirahã men then asked, "Hey Dan, what does Jesus look like? Is he dark like us or light like you?" I said, "Well, I have never actually seen him. He lived a long time ago. But I do have his words." "Well, Dan, how do you have his words if you have never heard him or seen him?" They then made it clear that if I had not actually seen this guy (and not in any metaphorical sense, but literally), they weren't interested in any stories I had to tell about him. Period. This is because, as I now knew, the Pirahãs believe only what they see. Sometimes they also believe in things that someone else has told them, so long as that person has personally witnessed what he or she is reporting. In the end, Everett never converted a single Pirahã. But he did even worse than converting zero people—he lost his own faith after coming to believe that the Pirahã had a good point. After keeping this to himself for many years, he revealed his loss of faith to his family, which led to a divorce and his children breaking contact with him for a number of years afterward. But Everett losing his faith in the God of Abraham was only the beginning. Most importantly for us, he also lost his faith in the God of Linguistics—Noam Chomsky. III. THE WAR In 2005, Everett’s paper “Cultural constraints on grammar and cognition in Pirahã: Another look at the design features of human language” was published in the journal Cultural Anthropology. An outsider might expect an article like this, which made a technical observation about the apparent lack of a property called ‘recursion’ in the Pirahã language, to receive an ‘oh, neat’ sort of response. Languages can be pretty different from one another, after all. Mandarin lacks plurals. Spanish sentences can omit an explicit subject. This is one of those kinds of things. But the article ignited a firestorm of controversy that follows Everett to this day. Praise for Everett and his work on recursion in Pirahã: He became a pure charlatan, although he used to be a good descriptive linguist. That is why, as far as I know, all the serious linguists who work on Brazilian languages ignore him. Noam Chomsky, MIT professor and linguist
Inline links: Wikipedia, hard takeoff, grammar, 1, this video, don’t appear to have words for relative directions, “thank you”, “yes” and “no”, long been an opponent, 2, Daniel Everett, Sateré-Mawé people, prepared, Bible and Foreign Missions, 1975, SIL, viscerally prepare, Pirahã people, https://substackcdn.com/image/fetch/$s_!42Ji!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F5c899185-e1a2-4358-ae6a-e9298f6c7cdb_1600x1002.jpeg, source, 3, number words, https://substackcdn.com/image/fetch/$s_!yx-R!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F77975f2c-ef0f-47d7-9ed9-f22bbe04f78b_1024x768.jpeg, source, Cultural constraints on grammar and cognition in Pirahã: Another look at the design features of human language
The rub is that artificial systems engineered to perform some particular task well are not black boxes; we can look inside them and tinker as we please. Studying the internal representations and computations of such networks has provided neuroscience with crucial insights in recent years, and such approaches are particularly helpful given how costly neuroscience experiments (which might involve, e.g., training animals and expensive recording equipment) can be. Lots of recent computational neuroscience follows this blueprint: build a recurrent neural network to solve a task neuroscientists study, train it somehow, then study its internal representations to generate hypotheses about what the brain might be doing.
Chomsky’s domination of linguistics is probably due to a combination of factors. First, he is indeed brilliant and prolific. Second, Chomsky’s theories promised to ‘unify’ linguistics and make it more like physics and other ‘serious’ sciences; for messy fields like linguistics, I assume this promise is extremely appealing. Third, he helped create and successfully exploited the cognitive zeitgeist that for the first time portrayed the mind as something that can be scientifically studied in the same way that atoms and cells can. Moreover, he was one of the first to make interesting connections between our burgeoning understanding of fields like molecular biology and neuroscience on the one hand, and language on the other. Fourth, Chomsky was not afraid to get into fights, which can be beneficial if you usually win.
Inline links: cognitive zeitgeist, fights