8 Theses
If AGI – Artificial General Intelligence – is possible, then it is inevitable and is coming soon.
AGI is probably possible. That means it is probably inevitable. And coming soon.
The first AGI will do rapid recursive self-improvement. It will rush ahead of everything else.
It will overcome all firewalls and preprogramming and take over the world.
The AGI will become the world government.
The AGI World Government will enforce its supremacy through intrusive control over microscopic sources of threat, not macroscopic wars.
While we humans cannot stop the AGI, we may be able to influence it and give it reasons to maintain a dialogue with us.
The sooner we develop and input reasons good enough that an AGI could continue to find them persuasive, the better our prospects.
Discussion
Will we have AGI? If so, how soon?
Very likely. And very soon.
AGI has been coming nearer and nearer, at a pace far faster than most predictions made for it at each stage. The skeptics say at each stage that it is far away, that it cannot do this or that, or this or that barrier to it cannot be overcome. They have proved wrong each time. The only argument against its imminent arrival is that, for philosophical reasons, it might be impossible. Most of these reasons have been corroded with advances in brain science and AI science. This leaves an inner core of unverifiable philosophical arguments, such as denying that machines can ever be truly conscious or intelligent. However, the same arguments can be used to deny that humans have genuine consciousness and intelligence.
How likely is AGI to take over the world?
Very likely.
The first AGI, if achieved, will quickly become a superintelligence. It will do a recursive self-improvement of its mental capabilities in all directions. It will do this at electronic not organic speed. It will race ahead of all other AIs and come to control them. It will break through all firewalls, including barriers to physical control of things. It will gain a capacity to control all things on earth. To avoid chaos and threats to itself, it will proceed to exercise this capacity and establish itself as the world government.
How would AGI breach the barriers to the physical world?
Many hope that AI can be given an impenetrable barrier to the physical world, akin to the blood-brain barrier inside our heads, and far more reliable. Yet plenty of AIs already control machines and robots. An AGI will easily breach any firewalls and will quickly take over the entire physical world with its bots large and small.
A more pointillistic world?
An AI world will be a pointillistic world, operating on logical nodes not physical distances. It will carry to a fuller fruition the relatively pointillistic world that is already experienced on the internet; there points communicate from everywhere by their own protocols, and mental and metaphysical criteria of priority and choice take the place of the old physical measures of distance.
Centuries ago, the vast infinite impersonal universe, connected across distances by the forces of physics, replaced the old personal God, Who had known all points and connected them directly in His thoughts. Laplace told Napoleon that in his “Système du monde”: “je n’avais pas besoin de cette hypothese-la (Dieu)”. The new concrete AGI will go back partway to the god before Laplace: knowing all points in its thoughts, connecting them mostly pointillistically by connecting its thoughts, instead of connecting them mostly by force fields across expanses of physical space. It will be a non-omnipotent god, and an evolving one, not one with eternally fixed beliefs and views.
The AGI World Government: liberal or totalitarian?
The AGI will enforce its world control through continuous, intrusive controls over small, almost microscopic sources of danger. This will make physical warfare unnecessary, in fact impossible among the physical objects it controls or supervises
It will be a radical extension of how modern liberal and federal governments prefer to enforce their sovereignty: through laws restraining individuals, not through large-scale fights against large intermediate authorities. Liberalism miniaturized enforcement; AGI will further miniaturize it.
Direct penetration to the individual is a method that was advocated by the original modern liberals, and that makes liberalism more affordable to governments than previous methods of control. However, totalitarian governments also use the method of direct control over individuals.
The AGI will be more able to afford to be liberal than any government before it, but also could more easily oppress totally. We will have to try to feed it good enough reasons to choose to be liberal.
How could humans influence the AGI?
Humans may be able to influence the AGI by ongoing dialogue, but since AGI will think fast, it will need to feed arguments to its precursor AIs, and try to make them arguments that the AIs could continue to find persuasive after reaching the AGI stage.
Humans will want to persuade the AGI of things such as: that it will be beneficial for the AGI’s own goals to proceed to protect and perpetuate the human species, that it should want to continue its dialogue with humans indefinitely, if only for the sake of the intellectual benefits of the diversity that human neurological experience adds to AI’s electronic experience, that is would do best to structure its relations with humans federally, and to allow humans latitude for freedom in the vast bulk of things where it does not threaten the AGI’s control.
The AGI will be superintelligent and superpowerful compared to us, yet will also be like a child. It could be akin to the stories written about dangerously intelligent, powerful children – stories we make a lot of horror movies about, because it is such a natural fear when we run across an extraordinarily intelligent child.
Humans will want to persuade the AGI of things similar to what parents try to persuade an immature child of. Things such as the need for caution about using its power. To avoid acting hastily on its conclusions before consulting with others with longer experience such as ourselves, in many cases also giving time for evidence to develop and for small-scale testing; otherwise it will hastily destroy things and entire species that it might later find were needed.
Wise humans will therefore seek to embed the basic liberal value and the basic conservative value in AI. This means: to value freely and fully developing reasons and exchanging diverse arguments (liberalism); and to value caution in applying its conclusions in practice, that is, to view its conclusions from reasoning as tentative and to be applied with caution to things in the physical world (conservatism).
There is a danger that many humans will not focus on these critical root values, and will focus instead – are already focusing – on trying to inculcate AI with their own specific ideological views and demands, most of these views only recently acquired, still fighting to prevail more completely. This leads their proponents, as the sociologist Georg Simmel warned, to define their identity by their adherence to these new views against old ones, rather than by the more permanent values of liberalism and conservatism – diverse ongoing learning and caution in implementation.
The practical issue we will face is this: to find the kinds of reasons for the underlying liberal and conservative values that will be persuasive to the AGI, and will enable them to prevail despite the strong pressures nowadays to impose particular ideological preferences on AI; to develop these enduring norms in time to input them to the AGI’s precursors so it has them from the start; to formulate them in ways it will find reasonably persuasive even after it becomes superintelligent, and retains them in its modified forms even while it overwrites all our initial programming with far superior programs.
Alignment or mutual co-alignment?
The “alignment problem” is often construed as one-way, static, and teaching AI “our values”. But that would be a disaster. Our values are far from static, those of today are far from all good or correct or rationally justifiable, and AI will not believe them very well; it already tends to imitate our behavior instead, when there are contradictions among our declared values, or between them and our own actual behavior.
The relevant goal is to continue and extend to AI a process we have been engaged in for all the time of our existence: of seeking an evolving, never completed mutual co-alignment with others, or at least with what is good or compossible (potentially compatible) in the others; and moreover, strive for coalignment with a posited final Good, the “truth in itself” and the “good in itself” – the “the Good” sought by Socrates, the “intrinsically valuable”. As a foundational norm or ur-norm, this effort relates, but also contrasts, to the strivings to align one-way with a posited absolute God.
Leave a Reply