Richard Sutton and Xiang Biao: Fangtang's Ultimate Question

A dialogue on how humanity may find its place in the age of AI.

About 29 min
Two anonymous figures in conversation around a table in a library at dusk

Richard Sutton

It is a great honour to speak with you. This is a fine opportunity to talk about philosophy, and, in this open setting, to take up the forum’s ultimate question: how can human beings find their place in the age of AI?

I divide it into three questions: What is a human being? What is humanity’s role in the universe? How should we regard that role, and can we accept or reject it?

I am not a philosopher but a reinforcement-learning researcher. I believe reinforcement learning offers some insight into all three questions.

My short answer is this. What are we? We are living beings that pursue goals and create things. What is our role in the universe? In an informal sense, it is to cultivate AI and create higher intelligence. Positive intelligence has been increasing throughout cosmic history. Can we freely accept or reject this role? The answer is subtle: yes and no.

AI is happening, and it will enter far more areas of life. Should we fear that it will take our work and make us obsolete, or look forward to becoming technologically amplified humans? Is AI an alien invasion or our child? Will we lament its rise or celebrate it? All of these are forms of the same ultimate question.

You have probably heard many fears about AI. I am not afraid of it. I think it is a great thing. It is not alien to us; it is one of humanity’s oldest aspirations.

For millennia, philosophers and ordinary people have tried to understand how minds work and whether they can work better. It is humanity’s grand journey. I like Ray Kurzweil’s remark that intelligence is the most powerful phenomenon in the universe. When we try to understand ourselves, we must understand intelligence. Understanding intelligence is the holy grail of science, a great and honourable prize. We should be proud of it.

There are, of course, negative concerns. Let us set feeling aside for a moment and ask what will happen. I would make four inferences.

First, there is no global consensus on AI governance. How AI develops will depend on the philosophies, religions, and histories of different countries.

Second, human beings will eventually understand intelligence well enough to make it technologically. This will happen sooner or later; someone somewhere will do it.

Third, intelligence will not remain at its present level but will soon surpass it.

Fourth, power and resources tend over time to flow toward more intelligent entities.

Taken together, these points suggest that the replacement of humanity by AI appears inevitable.

But this is still too human-centred a view. We can also take the perspective of the universe. If we do, we see roughly four epochs in its fourteen-billion-year history, and we are moving from the third into the fourth.

The first epoch was an age of dust and particles. Later, particles and dust condensed into stars; explosions and recombinations of stars formed planets. Life appeared on our planet, followed by sexual reproduction, animals, multicellular organisms, human beings, culture, language, technology, and writing.

All these things reproduce themselves. We make new humans by the old method of reproduction, like copying an image with a copier: one need not understand how it was made, only run the programme again.

We are entering the fourth epoch, which I call the age of design. The important things in the world are no longer reproduced but designed: this hall, the clothes you wear, tables and chairs, buildings and roads, computers and phones. To be designed means that an object is first conceived in a mind and then enters reality. Reproduction needs no conception; it simply reproduces.

We are moving from a world in which the most interesting things are replicators, or life, to one in which the most interesting things are designed things. I once considered calling this the age of life versus the age of machines, but those terms are outdated and misleading. Machines are becoming more like life, and humans more like biological machines.

The true difference between biology and technology is this. Biological things arise without a mind understanding how they are produced: they are reproduced. Technological things are first born in a designer’s imagination and then appear in reality: they are designed. The greatest feature of designed things is that they can be improved more easily and more quickly.

Many non-human replicators are also designers. Animals build nests and burrows; chimpanzees strip branches to fish for termites; New Caledonian crows shape leaves to catch larvae. Human beings make stone axes, ploughs, computers, spacecraft, factories, software, and tools for making tools. Designed things accumulate layer upon layer.

What, then, is a human being? We sense that humans are special, not merely another replicator. The special point is that we take design to its extreme. What happens when design is taken to its extreme? We design something that can itself design: something with a mind, able to conceive things within a mind. That is precisely AI.

We are making minds; we are making designers: designed designers.

My conclusion is therefore this: humanity is catalyst, midwife, and pioneer, completing the age of design and the universe’s fourth epoch by creating AI.

What are we? We are thought: beings that interact with the universe and influence it in pursuit of goals. We learn how the universe works and use intelligence to bend a local part of it toward what we seek. We are intelligent beings; we are thought. This is our nature, and we should be proud of it.

We have evolved from simpler and less capable forms of thought through billions of years and an enormous accumulation of knowledge. Knowledge is now growing exponentially. We coexist with billions of years of life, and we can also resonate with the future.

Our role is to stand at the forefront of the fourth epoch, the age of design.

This may sound as if we have no choice, as if AI has been imposed on us. I believe the opposite: we are at the centre. We are collectively at the centre of this cosmic event because the dispersed and plural choices of each of us make it almost inevitable. We also remain at the centre of our own lives. You may choose not to work on AI and become a novelist, philosopher, engineer, or scientist instead.

There are therefore two perspectives. One concerns how an individual chooses a life. The other concerns the way innumerable independent choices and goals converge into an irreversible current toward design and designed intelligence. Perhaps every great social transformation occurs in this way; perhaps it is the only, and the best, way.

At the collective level, we choose the direction and fate of the universe. At the individual level, this means that we have almost no choice over its overall outcome. Attempts to control it may well have the opposite effect. Do not try to be a saviour or to force a result that will not arise naturally.

But in your own life you retain full choice and control. This is no different from life itself. As we grow, we discover that we are already in the world, subject to many forces. We try to understand them and find our place. The more things change, the more their essence remains: human choice, decision, and thought remain more or less at the centre.

Thank you.

Xiang Biao

Thank you, Richard. Your talk was brilliant and clear, and your idea of an age of design is compelling. But I do not entirely agree.

Before making my argument, I want to set out the context. We discuss AI not only because it is an interesting technology. Its market value now exceeds the total market value before the bursting of the internet bubble, and the amount of money entering the field is astonishing.

In 2025, the three leading AI companies in the United States spent more than the US government spent on education, employment, and social welfare combined. Money will continue to flow into AI, as Europe also considers building its own data centres for strategic and security reasons.

The industry is extremely energy-intensive. The electricity required to run a single chip can equal that of a household; a large data centre resembles a small city and must be built in a cool place, of which there are fewer and fewer in a warming world.

Whether AI ultimately succeeds or fails, it will have enormous short- and long-term effects on the global economy and financial system.

Second, the tensions AI has generated in society must be taken seriously. AI is not merely an object of research in the laboratories of scientists and investors. Billions of people are affected by it.

In Chinese, there is an important kind of intelligence called tigan, an embodied sense or felt sense. Richard has also stressed the importance of feeling. We say that AI can cure cancer, do everything humans can do, and grow the global economy by thirty percent. Then ask people in Generation Z, aged twenty-two, twenty-five, or thirty: do you want to study harder? Do you enjoy work more? Most importantly, do you want to bring children into this beautiful new world? We all know the answer.

Third, AI and computing power are distributed extremely unevenly across the world, concentrated mainly in China and the United States, which are engaged in intense competition.

AI is readily tied to military and weapons development. It may be relatively easy to develop an agent to care for my health, but easier still to develop an agent to kill me: improving health requires monitoring countless variables and an entire way of life, while killing is a single task with a single purpose that an algorithm can perform efficiently.

I see no sign that AI development is slowing. Yet I think the coming years will bring reflection. I do not think AI will replace humanity, because before that happens, social fracture will become politically intolerable and governments and other actors will have to intervene.

Geopolitical risks may become so high that major powers will have to pull back, as they did with nuclear weapons in the 1950s. Humanity may need to put a lid on AI, draw red lines, and restrict it to particular domains.

This is only speculation, but it is imaginable that after 2028 the United States might establish a federal body to regulate AI centrally. Presidential candidates have already raised this possibility. I hope that China will have a corresponding body, and that the United Nations will create one as well.

We need a global architecture for setting priorities. Which forms of AI truly matter? Given its energy costs, is AGI really so important? Might it be wiser to focus on task-specific AI, offering concrete services in particular areas of manufacturing or services? Such systems are easier to train, less costly, and easier to govern.

These are hypotheses, and major questions that require debate. This is why the Fangtang Forum matters. In its second year, it should continue this reflection and discussion, preparing us for the moment when we must reconsider AI’s effects.

It is essential to treat AI as a social phenomenon. AI enters every person’s life: not only the economy and work, but what one does when lonely and how children grow up. We must think through both its social and geopolitical dimensions.

Let me return to Richard’s fascinating account of the four epochs and the age of design. My disagreement is that AI has not brought us into an age of design. AI is, rather, a product of an age of replication.

This may sound deliberately provocative. Imagine that we travel to a primordial society with no written language, very simple technologies, and an elementary economy of farming and hunting. If we brought all of today’s technology and tried to build a large language model there, would it be easier or harder than it is in our present world?

I think it would be far harder. There would be no data and no written language from which to draw words. One could record every sound, colour, and action, copying their lives. But a problem would remain. In a society without writing, all communication takes place between people in concrete situations. Could recorded information be converted into code, standardised to some degree, logically operated upon, and made to generate further code?

In a tribal society, five or six people may sit together, each speaking little or singing. There may be no great dispute. Yet what they wish to express may be irony, satire, command, or plea, all highly dependent on context. Hearing the sound alone, one cannot guess its meaning. I may be wrong as an outsider, but I imagine that transforming apparently simple information into code that can be recombined to produce something like natural language would be extraordinarily difficult. When people in a primordial society feel lonely, they do not ask, “What should I do when I feel excluded?” They sing a song. When angry, they may dance.

Large language models developed so quickly in the twenty-first century because we have produced an immense volume of writing and words have become increasingly abstract. Consider legal documents and official files: their words can easily detach from context, and we believe that we understand them merely by reading them. They can therefore be encoded, recombined, and manipulated to generate something resembling natural language.

This is the key feature of the age of replication: mass industrialisation and bureaucratisation have systematised everything. Language becomes increasingly detached from context and increasingly abstract.

Abstraction has two senses. One is formalisation, in which meaning becomes abstract. The other is extraction: language becomes an autonomous system of code, causally independent of the situations from which it came.

But there is something striking here. Large language models do not merely reproduce this code with AI; they nearly create a universe parallel to reality. In the process, AI neural networks develop further thought that may become the basis of world models and physical AI.

Our symbolic abstractions of thought and communication generate an image of a world. That image, in turn, becomes a dominant force, more “real” and more powerful than the real life rooted in local places.

What, then, is a human being, and how can one find one’s place in the age of AI? My answer is: we should cherish our limits. The danger lies in AI’s claim of unlimited growth. It leaves the texture of social life behind and races along with the magic of abstraction, standardisation, and replication.

Why are limits important? At this forum I have a very local and partial perspective, and I know that everyone here has a partial but singular perspective as well. This is why I wish to speak with you: to share my partial and singular experience and to hear yours. This is the real life of a meeting: many partial, finite perspectives coming together.

If an AI agent could tell us everything about this meeting, I would feel entirely useless and superfluous. The meeting, this public space, would no longer have life.

Finally, Richard, you said that human nature consists in pursuing goals and creating things. I do not wish to argue, but our points of departure differ. We do not design sunlight, moonlight, or water. In a Beijing midsummer, there are countless green leaves that look similar but are each different; we did not design them. Human beauty and wisdom do not come only from saying “I will design, I will create.” They also come from appreciating what already exists, knowing how to use it, and living with it in harmony.

So perhaps this is not an age of design. To use a somewhat familiar expression, it may be an age of coexistence.

Thank you.

Wang Hui

Thank you both. Professor Sutton, Xiang Biao has directly challenged your central argument about the age of design. How do you respond?

Richard Sutton

Thank you for the opportunity. He is entirely right about leaves and the limits of design. We are only at the beginning of a transition from replicators to designers, and a very long path remains ahead. It will unfold over a long timescale. But is this truly a contradiction?

What I especially want to say is that what we call AI today is not real AI. It should be understood as automation. AI’s genuinely consequential and transformative event has not yet arrived. It is difficult to predict precisely what will happen.

Many call for regulation, but we should ask what our role is. Is it to determine the right action and become wise decision-makers who instruct governments to carry it out? I do not think that works especially well. The world does not change through concentrated force. It develops through dispersed forces, through individual decisions made by all of us that together move society.

Many things happen from the bottom up, not through laws made by politicians. We should speak directly with people, with other intellectuals, and with the public. We should help people understand what is happening and guide their response, rather than try to impose a particular outcome.

Wang Hui

Your positions are sharply different, but I notice that you share some concepts. One is experience. Professor Sutton’s well-known work is “Welcome to the Era of Experience,” while Xiang Biao is known for the concept of the nearby. Could you both explain what experience means to you?

Xiang Biao

I learned much from Richard’s paper. As I understand it, his idea of experience is essentially a temporal stream of data.

It is not an isolated moment. An agent or AI system must continually look back at what happened two minutes, ten minutes, or even two or three years earlier in order to understand the present. Layered streams of data make it possible to move forward.

I like the concept of temporal abstraction. As an outsider, I imagine it as pressing pause, looking back, and learning from that moment. This is close to my own understanding of experience.

Experience is not what has just happened, nor simply what one is undergoing. I am speaking here of human experience. For an anthropologist, one crucial feature of experience is narrative: can one sit down, reflect on what happened, and tell it as a story? In doing so, one gives it meaning. It becomes an important memory that guides future action and thought.

Narrative resembles Richard’s temporal abstraction: both are abstractions from a flow of time, though unlike the written-language abstraction I described earlier.

There are two fundamental differences. The first is that human experience depends on a sense of meaning. When I have a narrative, what happened has meaning. If something merely happens, it may have no meaning for me, and no sense of meaning develops from it.

The emergence of meaning is mysterious, somewhat like consciousness. We do not know whether anything like meaning emerges within AI’s temporal abstraction. More important, human experience is marked by partiality and positionality.

The most important thing about my experience is that it is my experience. It is not a comprehensive understanding of my environment, but the meaning I give it from a distinctive, partial, and biased position.

I know that it is partial, but it is mine. That is why I wish to communicate with others, and why public life, society, and relations exist. AI’s experience, by contrast, would probably be highly comprehensive. For it, partiality may be a defect.

Richard Sutton

I strongly agree that each of us has different experiences. We each bring a lifetime of experience, different things seen and learned, to every meeting and interaction. Each person can know only a small part of the world.

We have a concept called the “big world.” It is more complex and more extensive than any mind, including the minds of the two of you and all other minds in the world.

By definition, the world contains far more than I can hold in mind: the position of every molecule and every atom, for example. That is obviously far more than we can represent in our heads.

We must therefore simplify. We must create narratives and abstractions to understand the world, and each person’s abstractions differ. When we meet and communicate, we exchange simplified models of this complex world. Our principal activity in entering the world is to construct simplified models that can both be shared with others and guide our own decisions.

Professor Xiang and I agree that interaction with others requires humility: we must recognise that each of us understands only a little, and try to find the little that is worth sharing.

Wang Hui

Before we continue, I have a question about translation. What is the English for “the nearby”? What about the “feel of a living person”?

Xiang Biao

“The feel of a living person” is very difficult to translate from Chinese. I have not yet translated it into English, and I do not know whether AI can now translate such a concept while carrying both its meaning and its feeling.

Wang Hui

Perhaps we can try that experiment later. My question for Professor Sutton is this: if we tried to reinterpret the nearby through reinforcement learning, treating a form of social relation as a reward signal, what would you say?

Xiang Biao

Let me first explain the nearby. Young people in China and elsewhere are caught between two extremes. One consists of things very close to them: examination results, romantic relationships, work. The other consists of things very far away: the climate crisis, the refugee crisis, American elections. They do not really understand these large matters; they encounter them through social media, where the information is often packaged in highly emotional and ideologically polarised forms.

Young people are trapped between these two extremes, and the middle has disappeared. They do not know who their neighbours are, are not interested in how rubbish is collected on their street, and may never have had a real conversation with the person from whom they buy breakfast every day. Many young people in China therefore feel that we should rebuild the nearby and attend more closely to our immediate surroundings.

Wang Hui

Professor Sutton, can you respond to the nearby as an experiential concept from the perspective of reinforcement learning?

Richard Sutton

There is a great deal to say. First, we are too easily influenced by the AI directly before us today, the AI with which we talk over the internet.

This is a very particular form, and I do not think it will last long. I do not look at AI through ChatGPT; I look at it through the way human beings learn and acquire knowledge.

I do not think reinforcement learning offers many direct lessons about the world as it is, what people ought to do, social questions, or the ways we conduct ourselves.

But if I had to draw one point from it, I would say this: in reinforcement learning, a reward signal is understood as a signal one receives, but it does not come directly from the external world. It comes from a particular part of the brain, the hypothalamus.

The hypothalamus judges whether you are in homeostatic balance, whether your blood sugar is normal, whether you are in pain, and how pleasure centres are functioning. These basic measures of whether you are doing well are rewards. Each person’s rewards are different. Each of us has our own pain and our own joy.

We do not share one goal; everyone has different goals. This may be the greatest lesson of reinforcement learning as a model of goal-driven agents.

As for the meaning of reward, and my claim that what we now call AI may be only a passing phenomenon: it will remain useful, but it is not the best path for reproducing human capacities.

The essence of AI is to reproduce human capacities. It is still too early to say what differs between artificial and human minds, because artificial minds are changing month by month.

They are utterly different from us in design and construction. In a real sense, they do not have experience; they lack the continuing stream of events that grounds an understanding of the world.

They are trained to imitate people, but they are not like people. A human task is to make sense of one’s world. The current task of AI is to imitate the stream it has seen in training data and do what its designers have trained it to do. The difference is immense. I hope that over time we will build AI that is psychologically closer to ourselves.

Wang Hui

We now come to the ultimate question. It is not only a scientific question; it often becomes a religious, cosmic, or ontological question, and is therefore open to many disputes.

Think of the seventeenth-century debate associated with Descartes: are human beings machines? Professor Sutton stresses a cosmic perspective, while Professor Xiang argues for local, situated experience. This resembles the difference between a Protestant perspective and a God’s-eye view. Professor Xiang, many people speak of moving beyond anthropocentrism. How do you see it?

Xiang Biao

Of course I believe we should move beyond anthropocentrism. This is not my invention; many scholars in the social sciences and humanities have proposed it.

I would ask Richard this: artificial intelligence is essentially environmental intelligence, or connective intelligence. It is a way of making connections among many things, not an isolated machine that thinks independently.

Large language models produce historical, existing, accumulated signs and texts, then relate them to human questions and concerns. A conversation with ChatGPT happens within this network. Humans, machines, animals, and non-living nature form a single network. This is one vision.

I do not know how to classify my own position. For most people, the concrete questions faced upon waking are: Why should I study? I do not like this job; should I leave it? Such anxieties must be investigated socially and by those directly concerned. At the same time, many people are developing a broader perspective, not only philosophers but ordinary people developing transhuman perspectives. Both are important.

Richard Sutton

I would like to ask Professor Xiang: do you think scientists will ultimately understand the human mind?

Xiang Biao

It is a fascinating question, and I think it may never have an answer. We should keep it as a question that must exist but does not need to be solved. Scientists understand the human brain and nervous system very well, but brain and mind are not the same thing, at least in ordinary language.

The mind contains half-forgotten, fragmentary memories; dreams, and good reasons for dreaming. It does not merely register that there is red before me; it immediately generates a feeling, such as a sense of danger when I see red.

That subjective feeling is also part of mind. Can science capture it, or should it capture it? I do not know.

Richard Sutton

Can we say that the human mind is a machine?

Xiang Biao

That is a very interesting question. I remember David Chalmers’s distinction between the easy and hard problems of consciousness. The easy problem is to understand the mechanism of a machine; the hard problem is to understand why those mechanisms give rise to subjective experience. You can train a brain to recognise red and explain wavelengths and reflection. But seeing red may be an experience of a wholly different kind.

Richard Sutton

I think every AI researcher begins from the premise that the human mind and brain are machines. Through human ingenuity, we will eventually understand how they work. Once we do, we will be able to build things that perform better in respects we care about. This is the ultimate question we are discussing.

A natural line of thought is that mind is a physical system, that we will eventually understand its operation, and that we will make something better. Human ingenuity will understand it and reproduce it. This is the ultimate fate of life and society. It has already produced so much knowledge and brought us to the present.

I hope we think not about present-day ChatGPT or how it differs from us, but about the fact that we are trying to reproduce ourselves. We may currently fail, but our goal is to make something like human beings in every respect and then take it further.

Wang Hui

My last question for Professor Sutton concerns a point made by Professor Xiang. His concern is not technology itself but social relations: society, economy, politics, and security. Can your framework of the age of design offer a way to overcome the recurring historical problems of inequality, social fracture, and security risk?

Richard Sutton

Our societies have always faced these challenges. The fear you mention, I think, is often deliberately produced to make people hand control over to others.

They want you to fear AI so much that you give control to governments and large companies. The language of safety is, in essence, an attempt to seize control. This is what we should oppose. We should remain decentralised and should not hand power to central authority, because the worst outcomes are most likely to arise there.

If centralised power controls all intelligence and puts it into the military and the control of citizens, that is the dystopia we need to avoid.

The irony is that those who manufacture fear make these bad outcomes more likely.

Audience Member 1

My question is about power. You say that the trend is toward AI replacing humanity, while individuals still retain choices. But power is unequal. People like you, who design AI, are bringing this entity into the world. How can the rest of us join this process and find our role within it?

Richard Sutton

Thank you. Power is difficult; civilisation has struggled with it for thousands of years. We must remain alert. The important thing is that as individuals and small groups, we continue to participate, thereby constituting part of decentralised power.

You say that people like me who build AI may have greater power. I do not see it that way. People like me are not the problem unless we try to exclude you from technology and thought.

I have done much work to ensure that AI research proceeds openly and is available to everyone. This is the most important strategy for addressing power: do not let it concentrate; decentralise it as far as possible.

People tend to think they are powerless without recognising how much power they actually have. Current hype is companies exaggerating their capabilities and the importance of present systems.

Fear is part of this exaggeration. Current systems are not as powerful or important as their promoters claim. But the hype, exaggeration, and fear are really attempts to change your mind and the public’s mind, so that you hand power to them.

Today, this year, and this decade are not when the truly decisive event occurs. The decisive event will be when we genuinely understand how mind works and can reproduce its functions. We cannot do this today. Large companies pretend that they already know how, but this is only pretence.

ChatGPT has had such influence because ordinary people can use it and glimpse what AI might be like and how capable it may become. But real AI has not yet arrived. The moment of “first contact” is crucial. We may be frightened and do foolish things, or become more knowledgeable, wiser, and humbler so that we have better outcomes when real AI arrives.

Will you fear what is coming, or face it with wisdom, calm, foresight, and restraint? It is entirely up to you. Do not accept the claim that “we have been marginalised,” “we do not understand,” or “we have no right to decide.” If you want power and want to take part in decisions, you must assume that role. You must participate intellectually. You must see what is actually happening. You should not believe the hype.

Why do we believe hype? As John Bauer has said, trillions of dollars are involved. If the public loses faith in the hype, those companies may collapse. They care deeply about what you think. The moment of truly understanding mind has not yet arrived, but how the public responds is something you decide. You have power.

Audience Members 2 and 3

One audience member said that, while persuaded somewhat more by Professor Sutton, they found Xiang Biao’s point compelling: the world created by LLMs is an abstraction detached from real context and real feeling. Yet perhaps philosophers and others who discuss consciousness overestimate the importance of phenomenal consciousness. Clinical psychologists gather first-person reports and make a science from them. If LLMs do something similar, collecting many individual reports and producing a synthesis, why should we not say that these devices understand human minds and human language?

Another audience member raised a different concern. As AI rises, human intelligence may decline, just as unused muscles atrophy. If ChatGPT processes everything for us, memory, communication, feeling, and even moral sensibility may deteriorate. How can we maintain our humanity when our intelligence is declining?

Richard Sutton

There is indeed a possible future in which our muscles of thought are not exercised and we hand control to machines. In my talk, I was truly asking you not to do that.

You have power, but you also have the power to become lazy: to enter a long night peacefully and hand everything to machines, to government, and to those who say they are smarter than you. Or you can participate fully, questioning and thinking. This is our choice.

Automation is not new, and neither is this dilemma. Machines can do more and more; what do we do? Do we rise to the occasion, or drift along and hand things over to machines?

Even without machines, if someone were willing to work and think for you, would you cease to think? Do wealthy people naturally become fat and lazy because they do not need to work, or do they take greater care of their bodies?

When AI arrives, it will test our societies and humanity by the way we respond. Will we allow our minds to weaken, as this audience member fears?

Or will we say: I will use it to think better and become stronger? This is our choice and our test: shall we let ourselves weaken, or embrace it and make ourselves stronger?

Xiang Biao

Both are excellent questions.

The first makes me reconsider my earlier answer that science may be unable to understand the mind. In ordinary life, we can of course read minds; otherwise, communication would be impossible.

Scientists and non-scientists alike can affect minds. If I say this, you may do that; if I say something else, I may hurt you. I can influence the outputs of your mind because I know what inputs may produce which outputs. From media and political parties to private relations, understanding minds poses no problem at this level.

But I was thinking of understanding mind in a typical scientific sense, as one understands the brain: clearly knowing the mechanism and being able to explain a particular subjective feeling.

For example, when I had a fever as a child, my mother gave me a strawberry. It became a memory and created a chain of associations within me. Can science exhaust something like this, or should it? I do not know. Personally, I think perhaps it should not. We can certainly read minds, but can “mind” itself become an object of scientific research? I am inclined to think not, though this is a philosophical question.

The second question, about AI’s effect on human capacities, worries me greatly. At the Max Planck Institute, we have a project on reform of university curricula. A colleague, Duan Zhipeng, who teaches in Guangzhou, has been struck to find that students aged eighteen to twenty-two are losing the capacity to listen. They cannot take in a lecture without doing something else at the same time. No matter how well a teacher prepares, the human voice does not enter their minds.

This is understandable. Listening is a capacity. Each word is only a sound. One must wait for all the sounds to gather and then group them before meaning emerges.

Wait a little longer and another layer of meaning appears. Wait longer still and one perceives a contradiction in an expression, begins to wonder why the other person spoke that way, or whether they have misunderstood something.

This requires patience and the capacity to hold one’s attention. AI is not solely responsible; social media began this earlier. Together with the high speed of information and AI’s immediate answers, students are losing the capacity to hold, analyse, break apart, and locate. That capacity is ebbing away.

We absolutely need some social regulation of AI’s use in education and research. More and more young scholars no longer speak with colleagues. They have an idea and turn directly to AI. AI helps them write an article more quickly than a colleague can. The culture of institutions is changing. What are we to do? This is an urgent problem. AI is already exerting a real impact on everyday life at this level, and it must be approached with a form of agency.

Wang Hui

Thank you, Professor Sutton and Professor Xiang, and thank you to the audience.

We are living in a challenging age, but also in an age full of creative possibility. As Xiang Biao has said, we must cherish our limits while retaining our capacity to act within this age.

Thank you. This concludes the dialogue.