Enfant terrible
Fire
- Joined
- Aug 22, 2020
- Posts
- 22,137
- Reputation
- 26,928
Scenario I: The Mindless Outsourcers
Technological progress continues to accelerate and at some point the technology of “mind uploading” becomes possible. Some human individuals upload and make many copies of themselves. Meanwhile, there is gradual progress in neuroscience and artificial intelligence, and eventually it becomes possible to isolate individual cognitive modules and connect them up to modules from other uploaded minds. Possibly, modules would need to be trained before they can communicate with each other effectively. Modules that conform to a common standard would be better able to communicate and cooperate with other modules and would therefore be economically more productive, creating a pressure for standardization. There might be multiple standards; some modules might specialize in translating between incompatible standards. Competitive uploads begin outsourcing increasing portions of their functionality: “Why do I need to know arithmetic when I can buy time on Arithmetic‐Modules Inc. whenever I need to do my accounts? Why do I need to be good with language when I can hire a professional language module to articulate my thoughts? Why do I need to bother with making decisions about my personal life when there are certified executive‐modules that can scan my goal structure and manage my assets so as best to fulfill my goals?” Some uploads might prefer to retain most of their functionality and handle tasks themselves that could be more efficiently done by others. They would be like hobbyists who enjoy growing their own vegetables or knitting their own cardigans; but they would be less efficient than some other uploads, and they would consequently be outcompeted over time.
It is possible that optimum efficiency will be attained by grouping abilities in aggregates that are roughly human‐equivalent. It might be the case, for example, that a math‐module must be tailored to fit the language‐module, and that both must be tailored to fit the executive‐module, in order for all three to be able to work together effectively. Standardization might be almost completely unworkable. But it is hard to see any compelling reason for being confident that this is so. For aught we know, human‐type minds may be optimal only given the constraints of human neurology. When it becomes possible to copy modules at will, to send high‐bandwidth signals between parts of different brains, and to build architectures that cannot readily be implemented on biological neural nets, it might turn out that the optima relative to this new constraints‐landscape have shifted away from the human‐like mind region. There might be no niche for mental architectures of a human kind.
There might be ecological niches for complexes that are either less complex (such as individual modules), more complex (such as vast colonies of modules), or of similar complexity as human minds but with radically different architectures.
Would these complexes be worthwhile from our current point of view? Do we, upon reflection, really favor a world in which such alien types of complexes have replaced human‐type complexes?
The answer may depend on the precise nature of those alien complexes. The present world contains many levels of organization. Some highly complex entities such as multinational corporations and nation states contain human beings as constituents. Yet we usually assign these high‐level complexes only instrumental value. Corporations and states do not (it is generally assumed) themselves have consciousness, over and above the consciousness of the people who constitute them; they cannot feel phenomenal pain or pleasure. We think they are of value only to the extent that they serve human needs. In cases where they do not contribute to the welfare of any sentient creature, we “kill” them without compunction.8 There are also lower levels of organization in today’s world, and the entities inhabiting these levels are not accorded significant moral value either. We do not think it is wrong to erase a piece of computer code. Nor do we think that a neurosurgeon is harming anyone when she extirpates a module (maybe containing an epileptic center) from a human brain if the operation helps the remaining parts of the brain to function better. As for alien forms of complexes of the same complexity as a human brain, most of us would assign them value only if we thought that they had a capacity for conscious experience. We can thus imagine a technologically highly advanced society, containing many sorts of complex structures, some of which are much smarter and more intricate than anything that exists today, in which there would nevertheless be a complete absence of any type of being whose welfare has moral significance. In a sense, this would be an uninhabited society. All the kinds of being that we care even remotely about would have vanished. What would make such a world valueless is not the fact that machines would have replaced biological humans. Whether a mind is implemented on biological neurons or on silicon processors seems to make no fundamental moral difference. Rather, the catastrophe would be that such a world would not contain even the right kind of machines, i.e. ones that are conscious and whose welfare matters. There may be an abundance of economic wealth and technological capability in such a world, yet it would be of no avail because there would be nobody there to benefit from it.
Technological progress continues to accelerate and at some point the technology of “mind uploading” becomes possible. Some human individuals upload and make many copies of themselves. Meanwhile, there is gradual progress in neuroscience and artificial intelligence, and eventually it becomes possible to isolate individual cognitive modules and connect them up to modules from other uploaded minds. Possibly, modules would need to be trained before they can communicate with each other effectively. Modules that conform to a common standard would be better able to communicate and cooperate with other modules and would therefore be economically more productive, creating a pressure for standardization. There might be multiple standards; some modules might specialize in translating between incompatible standards. Competitive uploads begin outsourcing increasing portions of their functionality: “Why do I need to know arithmetic when I can buy time on Arithmetic‐Modules Inc. whenever I need to do my accounts? Why do I need to be good with language when I can hire a professional language module to articulate my thoughts? Why do I need to bother with making decisions about my personal life when there are certified executive‐modules that can scan my goal structure and manage my assets so as best to fulfill my goals?” Some uploads might prefer to retain most of their functionality and handle tasks themselves that could be more efficiently done by others. They would be like hobbyists who enjoy growing their own vegetables or knitting their own cardigans; but they would be less efficient than some other uploads, and they would consequently be outcompeted over time.
It is possible that optimum efficiency will be attained by grouping abilities in aggregates that are roughly human‐equivalent. It might be the case, for example, that a math‐module must be tailored to fit the language‐module, and that both must be tailored to fit the executive‐module, in order for all three to be able to work together effectively. Standardization might be almost completely unworkable. But it is hard to see any compelling reason for being confident that this is so. For aught we know, human‐type minds may be optimal only given the constraints of human neurology. When it becomes possible to copy modules at will, to send high‐bandwidth signals between parts of different brains, and to build architectures that cannot readily be implemented on biological neural nets, it might turn out that the optima relative to this new constraints‐landscape have shifted away from the human‐like mind region. There might be no niche for mental architectures of a human kind.
There might be ecological niches for complexes that are either less complex (such as individual modules), more complex (such as vast colonies of modules), or of similar complexity as human minds but with radically different architectures.
Would these complexes be worthwhile from our current point of view? Do we, upon reflection, really favor a world in which such alien types of complexes have replaced human‐type complexes?
The answer may depend on the precise nature of those alien complexes. The present world contains many levels of organization. Some highly complex entities such as multinational corporations and nation states contain human beings as constituents. Yet we usually assign these high‐level complexes only instrumental value. Corporations and states do not (it is generally assumed) themselves have consciousness, over and above the consciousness of the people who constitute them; they cannot feel phenomenal pain or pleasure. We think they are of value only to the extent that they serve human needs. In cases where they do not contribute to the welfare of any sentient creature, we “kill” them without compunction.8 There are also lower levels of organization in today’s world, and the entities inhabiting these levels are not accorded significant moral value either. We do not think it is wrong to erase a piece of computer code. Nor do we think that a neurosurgeon is harming anyone when she extirpates a module (maybe containing an epileptic center) from a human brain if the operation helps the remaining parts of the brain to function better. As for alien forms of complexes of the same complexity as a human brain, most of us would assign them value only if we thought that they had a capacity for conscious experience. We can thus imagine a technologically highly advanced society, containing many sorts of complex structures, some of which are much smarter and more intricate than anything that exists today, in which there would nevertheless be a complete absence of any type of being whose welfare has moral significance. In a sense, this would be an uninhabited society. All the kinds of being that we care even remotely about would have vanished. What would make such a world valueless is not the fact that machines would have replaced biological humans. Whether a mind is implemented on biological neurons or on silicon processors seems to make no fundamental moral difference. Rather, the catastrophe would be that such a world would not contain even the right kind of machines, i.e. ones that are conscious and whose welfare matters. There may be an abundance of economic wealth and technological capability in such a world, yet it would be of no avail because there would be nobody there to benefit from it.
