AI Ethics and the Ethics of Human Survival

The Sapience Perspective

Discussing AI ethics today is both necessary and very important. However, if we could look back from the future, I am certain that the ideas currently proposed by various experts would prove to be quite immature. Three considerations lie behind this statement.

First, the development of AI technology is still at a very early stage. AI may never think like a human being. This is because, in the past 50 years, cognitive science—the field that seeks to understand how we (Homo sapiens) think—has made no fundamental breakthroughs, making such design impossible. For us living in the present, the opportunities and disruptions that AI may bring to human survival can at best only be speculated on through extrapolation.

Second, unlike other technologies humans have invented in the past, the opportunities and disruptions brought by AI to human society are all-encompassing, including individual psychology and emotions, lifestyles, family and interpersonal relationships, information and communication, art and aesthetics, religion, labor, employment and production relations, social welfare, science and medicine, war, forms of social organization, the enhancement of resources and power for key AI-related actors, the nature of the state, modes of governance, and even state-society relations. Its scope is virtually limitless. Moreover, these impacts will differ greatly across countries and regions depending on their circumstances. This means that the development of AI in different fields and societies will generate social and ethical problems that are highly diverse in nature, constantly shifting, and comprehensive. Under such conditions, generalized discussions of ethics at the level of traditional philosophy only serve to oversimplify an extremely complex problem. Put differently, the challenges AI presents are not ones that traditional philosophical reflection can cope with.

Third, if ethical discussions at the traditional philosophical level provide no way out, then technical-level solutions will remain only superficial. While embedding ethics in AI systems (to prevent them from doing harm) is undoubtedly necessary, it cannot address the fundamental problems. This is because individuals can still use AI platforms to commit all sorts of “bad acts”; while AI performs various “good acts,” it simultaneously generates large numbers of unintended social consequences; countries that place excessive emphasis on the social ethics of AI will lose out in international competition in the AI era to those more willing to clear the way for AI development; and once AI is deployed on a large scale in warfare, defeating the “enemy” more effectively will become the supreme ethic. At that point, AI ethics will inevitably give way to the ethics of military efficiency, just as the atomic bomb was developed. None of these problems can be solved at the level of technical AI ethics.

In short, ever since acquiring the capacity for rational thought, humans have, while gaining ever greater abilities to conquer nature and dominate others, simultaneously dug themselves more and ever larger pits. The rapid development of AI has now driven humanity to a critical moment: a landscape brimming with opportunities yet surrounded on all sides by abysses. This compels us to move beyond purely AI-focused ethical reflection and instead link AI ethics to the comprehensive transformation of human social ethics.

Since childhood, I have enjoyed imagining utopian solutions to the problems confronting humanity. After turning to sociology, this habit deepened. Yet it is only in recent years that I have developed a more systematic understanding of the nature and survival conditions of Homo sapiens. In my view, the ethical and other challenges posed by AI are but one manifestation of the broader problems generated by the rapid technological advances of industrial capitalist society. The difference is that past technologies typically gave rise to localized ethical problems. For example, a certain chemical raw material produced a specific environmental problem, and weapons of mass destruction raised questions of war ethics. Such problems gave humanity confidence in pursuing localized solutions. By contrast, the temptations and challenges brought by AI are comprehensive. Like floodwaters breaching a levee, they will spread everywhere, infiltrating every corner of society, comprehensively reshaping the world, offering humanity opportunities in every direction, and at the same time confronting us with problems on all fronts.

One might say that the emergence and rapid development of AI has already forced humanity into a “dead end” that is also a “land of opportunity.” If we wish both to reap the benefits of AI and to mitigate the many problems it creates—and at the same time, to address issues such as environmental crises and weapons of mass destruction—we must move beyond ethical thinking at the level of traditional philosophy. We must instead examine, from a perspective both more concrete and more fundamental, the nature of Homo sapiens, the relationship between Homo sapiens and nature, and the prospects for Homo sapiens, as a biological species, to endure on Earth over the long term. I call this perspective the ontological perspective of “Sapience,” or simply the “Sapience perspective.” In what follows, I will briefly explain its origins, core ideas, and implications.

The Structure-Mechanism Model

While writing the pamphlet What Is Sociology, I proposed that the core of sociology lies in structural-mechanistic explanation. But what exactly is the relationship between structure and mechanism? For instance, does structure determine mechanism, or does mechanism determine structure? In that pamphlet, I did not provide a clear answer. Later, through further reflection, I arrived at the following model of the structure-mechanism relationship:

… → Structure-derived Mechanism → Mechanism-derived Structure →

Structure-derived Mechanism → Mechanism-derived Structure → …

In this model, structure provides the causal basis for the emergence of certain mechanisms, while the dynamics generated by these mechanisms in turn create new structures. Together, they form a chain of mutual generation. Yet I remain uncertain about where this cycle begins or ends, and therefore use ellipses to mark both ends. My gradually clearer sense of what these ellipses signify arose from the following sources of tension.

My tension comes from the deeply felt experiences I had after returning to China full-time, which also gave rise to concerns about China’s development. Put simply, in recent years, the number of obviously impractical rules and regulations, incentive mechanisms, slogan-driven targets, and corresponding metrics and evaluation methods has steadily increased. Faced with such management methods, the managed come up with endless coping strategies. In response, managers devise increasingly cumbersome rules, incentive mechanisms, and evaluation methods to tighten control, while the managed in turn develop yet more countermeasures. This produces a dilemma: The more regulations there are, the more loopholes emerge; the more regulations there are, the less respect people have for them. From the standpoint of causal logic, the managers’ approach and the managed parties’ responses have greatly amplified the importance of various ad-hoc causal mechanisms in social life—so much so that both sides live under constant strain, while management outcomes remain poor. Yet the countless ad-hoc causal relations I observed after returning to China exist neither in the inorganic physical world nor in the organic biological world, but only in contemporary social life. In other words, these overly abundant, highly specialized causal relations are located at the farthest tip of my structure–mechanism mutual-generation model—at the rightmost ellipsis. I therefore call this phenomenon, discovered after my return to China, “tail-heavy.”

Since these tail-end ad-hoc causal relations all arose only after the emergence of human beings—that is, in the very last instant of Earth’s developmental history—I began to think that my theoretical model might also reveal a historically oriented developmental process of the structure-mechanism mutual-generation relationship on Earth. This led me to ask: If the development of the structure-mechanism relationship culminates in the proliferation of such tail-end specialized mechanisms, then where does the model’s starting point lie? Pursuing this line of questioning led me back to the very beginning of Earth’s formation.

The model follows the emergence of increasingly complex structures and mechanisms from Earth’s formation, through inorganic and organic life, to self-reflective human life. At each stage, structures give rise to mechanisms, which in turn create new structures.

Sapience and Its Five Characteristics

From the perspective of AI ethics and human destiny, this model has three key points: First, after its formation, Earth went through an “inorganic structure period” lasting roughly 500 million years. During this period, various physical and inorganic chemical mechanisms provided the dynamics for the development of the structure–mechanism relationship on Earth.

Second, from the emergence of the earliest primitive life forms to the rise of Homo sapiens, Earth underwent about 4 billion years of biological evolution. Over this long process, negative feedback mechanisms with built-in “intelligence” continuously emerged. They are called negative feedback mechanisms because they can make self-stabilizing adjustments based on environmental feedback (for example, sweating while running to cool down, or shivering in the cold to warm up). During this period, although physical and inorganic chemical mechanisms still played a foundational role, intelligent negative feedback mechanisms not only appeared on Earth in exponentially increasing numbers but also played an increasingly crucial role in the developmental trajectory of the structure–mechanism mutual-generation relationship.

Third, the emergence of Homo sapiens as a new species (or a new “structure” in nature) has lasted only about 200,000 years so far. Homo sapiens are still constrained by physical and chemical mechanisms, are still subject to biological mechanisms (for instance, humans, like other animals, age and suffer from various diseases and epidemics), and still engage in fierce competition for limited resources, like other animals do. Yet in this exceptionally brief geological moment, the structure–mechanism relationship on Earth has given rise to five entirely new developments that increasingly shape the fate of the planet and of humanity itself: looping, ad-hocness, tail-heaviness, positive feedback, and the highly uneven distribution of social power. If looping, ad-hocness, tail-heaviness, and positive feedback belong to the domain of human nature, then the highly uneven distribution of social power is a concrete outcome that arises on the basis of human nature.

I term these five characteristics unique to humans “Sapience,” to emphasize that all the major problems humanity now faces are closely tied to Sapience. Put differently, any comprehensive reflection on social ethics must begin with Sapience. Below is a brief summary of the connotations of this concept:

(1) Looping. Since acquiring language and the capacity for thought, humans’ ability to justify their own existence and the meaning of their actions has continually expanded. This expanding capacity has produced looping in the structure–mechanism relationship on Earth: Regardless of whether a particular human imagination is reliable, once people believe in it, the mechanistic force it generates will bring about very real structural consequences. It must be stressed that, for the issues under discussion here, the reliability of human imaginations is not the point. In fact, ever since the rise of world religions and philosophy, few human imaginations have escaped hubris—especially male arrogance and overconfidence. What matters greatly for our discussion today is that human looping carries a positive feedback effect that amplifies both sides: It gives humans ever greater confidence and capacity to conquer and reshape nature, but it also intensifies the negative consequences associated with looping phenomena such as hubris, arrogance, discrimination, superstition, blind conformity, misguidance, deception, self-fulfilling prophecies, and the perils of over-cleverness.

(2) Ad-hocness. Another trait distinguishing Homo sapiens from other organisms is the massive proliferation of individualized ad-hoc mechanisms, which play an ever more important role in personal and social life. Ad-hoc mechanisms did appear before humans. For example, there are over a million insect species, yet some differ only in minute ways. The only significant phenotypic difference between two butterfly species, A and B, might be that A has a slightly longer proboscis than B (a structural difference). This enables species A to access nectar from flowers that B cannot (a mechanistic difference). Yet even this highly specialized mechanism is universal within species A: Every normal individual shares a proboscis of roughly the same length and thus the same adaptive advantage. By contrast, insects would never do what humans are adept at: While your proboscis is stuck in a flower, I pull out scissors and cut it off! This example underscores the point: In the biological world, specialized mechanisms are collective adaptations of a species to its environment, whereas in human society, they increasingly manifest as specific coping strategies devised by individuals or organizations in particular contexts. Because every individual and organization can generate vast numbers of ad-hoc mechanisms at any moment, human society acquires another emergent structure–mechanism relationship: tail-heaviness.

(3) Tail-heaviness. This term carries two meanings. First, it refers to the ceaseless production of innumerable ad-hoc structures and mechanisms by individuals and organizations in human society, with their consequences concentrated at the very tip of the structure–mechanism developmental chain on Earth (the tail end). Second, and more importantly, it points to a phenomenon frequently observed in China today: Tail-end mechanisms dominate the behavior of individuals and organizations, leading to repeated instances of form replacing substance, structures losing function, and personal goals displacing organizational ones. For example, even as we discuss AI ethics here today, countless AI companies are striving for technological breakthroughs; some may achieve one at this very moment, militaries may be making significant progress in AI weaponry, governments may be issuing new policies to foster AI development, and individuals are using AI for writing, design, creation, and other activities. Their actions generate massive tail-end structures and mechanisms whose impact far exceeds that of today’s conference. Meanwhile, others are using AI for fraud, lobbying governments for favorable AI policies, or advancing AI agendas that serve their own departmental interests. One can readily imagine that as AI develops, such tail-end activities will only proliferate, the tail will grow ever heavier, and the consequences produced by these tail-end activities will become ever more significant.

(4) Positive Feedback. During the four billion years of biological evolution, Darwinian negative feedback mechanisms—“natural selection, survival of the fittest”—dominated living behavior. Since the emergence of Homo sapiens, however, a different pattern has prevailed: goal-directed positive feedback mechanisms lacking self-stabilization (Lamarckian in nature), aimed at securing greater power, higher efficiency, larger profits, more effective weapons, and so forth. These mechanisms have not only come to dominate but have also accelerated, exerting ever stronger influence on all aspects of human social structures. It is especially important to note that under this dominance of positive feedback, many human-generated looping mechanisms and ad-hoc tail-end mechanisms likewise acquire positive feedback properties.

(5) Highly Uneven Distribution of Social Power. The fifth fundamental distinction between humans and other species is the extraordinarily unequal distribution of social power and resources. I need not elaborate on the origins of this phenomenon, but two points deserve emphasis. First, such extreme inequality in the distribution of power and resources in human society is neither inherently functional nor necessary. Second, its form is determined not by genes but by culture. These two features mark a fundamental difference from the sources of inequality found among social insects. They show that while extreme inequality is a reality of human society, it is not human nature. We are social animals, not societal animals. Generally speaking, the more power a social actor holds, the greater the consequences their actions produce. In this sense, the more unequal the distribution of power and resources in a society, the more severe the negative consequences generated by the other four characteristics of Sapience—looping, ad-hocness, tail-heaviness, and positive feedback.

From a philosophical perspective, the formation and development of all other major phenomena in human society—such as religion and philosophy, states and militaries, merchants and industrial capitalism, literature and the arts, science and technology, and lifestyles—are made possible by the existence of the five aspects of Sapience outlined above. The deeper sources of harm posed by AI technology, as a product of human activity, also originate in Sapience. Therefore, when considering AI ethics (or any domain of social ethics, including environmental ethics), we must begin by examining how to constrain the harm that these five specific aspects of Sapience can produce under conditions shaped by AI, rather than approaching the problem solely from technical, educational, managerial, traditional philosophical, or religious perspectives. In the following, I briefly examine some of the key mitigation strategies humanity has historically employed in response to Sapience and its potential harms, their weaknesses, and what might constitute a (at least logically) more effective mitigation approach.

From the perspective of this report, all previous mitigation schemes aimed at Sapience can be summed up in a single sentence: They attempt to add self-stabilizing negative feedback to a human society that is already in a highly positive feedback state. I argue that in the face of the formidable force of Sapience, attempts to introduce such negative feedback are not only insufficient but, under conditions of extreme inequality in the distribution of power and resources, may instead produce a wide range of harmful unintended consequences. Therefore, a method that is at least theoretically more reasonable—and perhaps the only way to save humanity—is not to attempt to alter the four innate characteristics of Sapience (Looping, Ad-hocness, Tail-heaviness, and Positive Feedback), but rather to use the technological achievements that arise from these characteristics to work toward reducing the extreme inequality in the distribution of power and resources in human society. Since such inequality is not human nature, efforts in this direction can help mitigate the harm produced by what is natural to us. To borrow the words of the Song Confucians, my strategy may be described as: “Preserve the Heavenly Principle, Eliminate the Desire for Power.” Before setting out the essence of this proposal, I will briefly analyze the weaknesses of traditional mitigation schemes aimed at introducing negative feedback into society.

The price mechanism effectively allocates resources, balances supply and demand, and thus constitutes an important negative feedback mechanism. It is especially significant in human society because its operating logic perfectly matches human nature’s tendency to seek benefit and avoid harm. It can function spontaneously without suppressing human impulses, which gives it immense vitality. This is why neoliberal economists regard the price mechanism as a highly positive social mechanism. The problem, however, is that if an economy operates entirely according to the price mechanism, it can easily produce overcapacity, economic crises, widening wealth inequality, environmental disasters, and even social unrest.

Law is another major negative feedback mechanism created by humans, but it can also generate positive feedback effects. Every law carries biases and loopholes, and laws can easily be exploited by those with unchecked power for self-interest. Under conditions of highly unequal power distribution, the negative consequences of such abuse of law can be magnified without limit.

Multi-party democracy, which relies on ambition checking ambition, is also a human-designed negative feedback mechanism. But for it to function effectively, three preconditions must be met: (1) The ruling and opposition parties must share a basic consensus of values (that is, society must not be too polarized); (2) The losing side must be willing to hand over power peacefully; (3) The winning side must not rewrite the political rules of the game after victory. The difficulty is that these preconditions are hard to sustain in the long term. Hitler, for instance, was elected to office by democratic votes, but immediately upon gaining power, he dismantled the multi-party democratic framework of the Weimar era. Even in a long-established democracy like the United States, Donald Trump in 2020, after losing the election, almost simultaneously violated all three preconditions necessary for democracy’s proper functioning.

Religion is another type of negative feedback created by humans, capable at least of restraining humanity’s infinitely expanding desires. However, the immense persuasive power of religious leaders, the coercive authority inherent in religious organizational structures, religion’s ability to leverage state power, and the competition and conflicts between different religious organizations for various reasons have been and will continue to be a major source of suppression, conflict, control, and domination.

Collective memory is also an important negative feedback mechanism created by humans, as it enables us to avoid repeating past mistakes. Yet differences in personality, circumstances, and values lead individuals to interpret and remember the “past” in different ways. If we try to enforce a single “correct” version, coercion becomes unavoidable. Once coercion is applied, the emergence of significantly distorted collective memories is inevitable. Moreover, remembering is not the same as personally experiencing. We can make the next generation remember the past, but we cannot control how they understand their memories. Could we allow better collective memories to prevail through free competition on the stage of ideas? This is an appealing thought, but somewhat naive, as we cannot prevent the phenomenon of bad ideas driving out good ones from dominating the intellectual stage. In terms of historical memory, humans are like short-tailed dogs endlessly chasing their own tails without ever catching them: Forgetting means betrayal, remembering means misremembering, and pluralism means forgetting.

All negative feedback mitigation schemes have obvious weaknesses, and human nature itself cannot be changed. The question then arises: Under such challenging conditions, is it still possible to construct a sustainable society in the face of AI, internet technology, biotechnology, nuclear weapons, and other cutting-edge technologies? My answer is: The possibility is small, but not zero. This possibility rests on a key insight: Although humans cannot fundamentally alter their innate nature—the “natural law” of Sapience—the degree of harm that these characteristics inflict on human society is constrained by a structural-mechanistic principle: the more uneven the distribution of social power, the greater the negative consequences of the four human traits brought to the human society. This suggests that rather than focusing primarily on designing negative feedback mechanisms, our main attention should be on reducing the uneven distribution of social power.

Due to time constraints, I can only outline some principled guidelines underlying my ideas. It is clear that these guidelines all aim to promote a more even distribution of social power: (1) Diffuse social forces must effectively check concentrated state power. (2) The power of regional states must be significantly curtailed, while global governance capacity should be enhanced—but only for the provision of a few critical global public goods. (3) Military forces should transition to policing functions, maintaining only essential public security. (4) Market mechanisms must operate within an institutional framework that prioritizes environmental sustainability and social equity. (5) As AI replaces more labor, human competitive instincts should be channeled into areas that cause minimal social harm, such as sports, literature, arts, philosophy, social sciences, humanities, fitness, social media, games, and travel. (6) Even in these seemingly “harmless” domains, environments should deliberately encourage the emergence of community-level “local celebrities,” rather than cross-community superstars. In this ideal society, any environment that allows the rise of big politicians, entrepreneurs, religious leaders, intellectuals, celebrities, influencers, or sports stars must be suppressed. In short, oppose any form of excessive concentration of power.

It is evident that I am imagining utopia. In this utopia, each community would have at most a few hundred thousand to a few million people (too small a community would lack diversity and risk boredom). Most work within the community is performed by AI, making self-actualization a primary life goal for community members. Communities are tightly connected internally but loosely connected externally, allowing people at most to become community celebrities, rather than cross-community stars. This utopia requires both community-level governments and a world government, but these governments provide only a few crucial public goods, leaving AI to handle everything else. Feasible? Almost entirely impossible! Yet it is possible to strive toward this utopia. Because it does not suppress human nature, it is highly practical. I call it “feasible utopia.” This may indeed be the only potential way to save humanity without suppressing our innate nature.

Finally, I want to emphasize that although human history itself has no “goal” or “predetermined stage,” from the perspective of the structure–mechanism developmental laws since Earth’s formation, the evolution of matter in the universe appears to follow some predetermined “goal” or “stage” (teleological), and the emergence of Sapience has a certain intrinsic inevitability. Confronted with such powerful natural laws rooted in the evolution of matter and human nature, any solution—including the one I propose—may prove insufficient. This means the challenges we face are likely far greater than we can imagine. Yet because they concern the fate of humanity, we must think the unthinkable and do the impossible.