Wednesday, 18 June 2014

Learning to Parrot - modern intelligence as a "Chinese Room" thought experiment


Suppose that I'm locked in a room and given a large batch of Chinese writing...[but] to me, Chinese writing is just so many meaningless squiggles. Now suppose further that after this first batch of Chinese writing I am given a second batch of Chinese script together with a set of rules for correlating the second batch with the first batch. The rules are in English, and I understand these rules as well as any other native speaker of English. They enable me to correlate one set of formal symbols with another set of formal symbols, and all that 'formal' means here is that I can identify the symbols entirely by their shapes. Now suppose also that I am given a third batch of Chinese symbols together with some instructions, again in English, that enable me to correlate elements of this third batch with the first two batches, and these rules instruct me how to give back certain Chinese symbols...from the point of view of somebody outside the room in which I am locked -- my answers to the questions are absolutely indistinguishable from those of native Chinese speakers. Nobody just looking at my answers can tell that I don't speak a word of Chinese.'

John Searle, Behavioural and Brain Sciences, 1980


The nature of modern technology and educational evaluations is such that people typically understand much, much less than they appear - superficially - to understand.

A modern person is in a position much like that described in Searle's Chinese Room thought experiment outlined above.

Whether in school, college, work, the Mass Media or in almost any kind of discourse - a modern person is able to interact on subjects far beyond his comprehension by algorithmically implementing a predetermined set of rules - recognizing inputs from a chart (whether external or internalized), then matching and selecting 'appropriate' predetermined responses, then ordering and setting them out as a kind of mosaic of 'points'.

This activity is, more or less, automatic - and involves no necessary comprehension of the symbolic inputs or outputs - the whole thing is a matter of cycles of recognition, matching and arranging; back and forth between people or groups.


So, a project is assigned on a certain subject. This subject is looked up on the internet. Passages of text, illustrations, graphs are copied, modified, pasted and arranged stylishly in line with explicit guidelines. The work is returned and marked according to a template referencing the guidelines. Several of these projects are accumulated and an educational qualification is awarded. The student becomes a manager, and the same procedure is followed. A task is assigned, information is gathered and arranged and presented - and evaluated, and perhaps implemented - perhaps as bullet points; and if so these implementations will follow the same process: each bullet point leading to an analogous process of recognition, matching and arranging. Even the question "But does it work?" is 'answered' by the same process of gathering and selecting pre-approved forms of data (sales numbers, surveys focus groups...), matching data to the outputs being evaluated, and arranging this into patterns.


In modern 'abstract' discourse, there is never any point at which any actual person evaluates the exchanges to determine whether real understanding is present or absent - because the formal evaluation procedures (whether in school college, work, politics, government or punditry) are themselves typically conducted on exactly the same basis as that which is being evaluated.

A person who really knows the field, may know that there is zero understanding - but from the perspective of formal evaluation procedures, this individual evaluation is merely opinion, rumour, hearsay and anecdote.

What really matters in modern bureaucratic organizations is the formal procedures - recognition, matching and mosaic-building; and these do not require understanding on the part of any of the participants.


So what is really going on behind the mechanical pretence of understanding?

Social interactions; the usual human stuff of gossip, or status competitions, or money-making, or attempted exploitation, or altruistic assistance... or whatever.

So the relevant thought experiment might be somewhat different from the impersonal and contact-less Chinese Room thought experiment - perhaps a better thought experiment might be interacting Parrots.


Imagine a group of parrots which have been taught a set number of English language phrases, and taught when to use these phrases in response to particular other phrases or the presence of key words; and taught rules about how to combine these phrases. These are then evaluated for their linguistic ability by other parrots who are checking whether the stimulus phrases match the proper response phrases according to the rules; and whether the phrases are being uttered in the proper combinations, according to the rules.


So, for these parrots, learning the English language, understanding the English language, is defined as following the proper rules in recognizing, emitting and combining phrases of English.

An intelligent parrot is defined as one that knows a lot of these rules and always follows them.


Throughout, none of the parrots have a clue what these phrases mean (if anything), nor are they in the slightest degree interested; so far as they themselves are concerned, what is really going on is showing-off or deferring, flirting or repulsing, threatening or submitting, and trying to get more food.


And this is a picture of modern 'intellectual' life - in science, medicine, the arts, politics, government, the mass media... the public arena in general.



Ref Steinsson said...

I think this explains certain artists and art trends that use "non-sense." From Lewis Carroll to William S Burroughs, dada/collage, surrealism, etc. It is a resistance to memorization/parroting by way of creating non-sense on the surface.

Anonymous said...

The same is true in all fields. When a student of maths learns the Calculus, he almost never understands the underlying principles, but instead learns several techniques for obtaining the correct answer. Even the majority of teachers will have never learned the subject from first principles or will have forgotten it. Hilary Putnam spoke about a linguistic "division of labour" where half the time we don't really know the true meanings of the words we use, such as "radiographer".
It is a philosophical problem to try to understand what "understanding something" really means. You really only properly understand a subject if you invent it yourself.

Bruce Charlton said...

@martin - Fair point.

Perhaps it would be more accurate to state that understanding is susceptible of degrees, and modern evaluation methods (and information technology) allow a lower degree of understanding to pass for a higher.