(DEEP) MORAL ENHANCEMENT IS RISKY
Complexity Claim:
Moral Traits are likely to be relatively more complex than most other targets of human enhancement.
Distributive nature of moral traits on the brain
Epistemic Complexity
Complex Etiology
Expert Opinion
Moll, Crockett
Natural History: Evolutionary hangovers, population bottlenecks, random mutation
Human History: The existence and spread of a particular cultural belief are the result of a long and complex series of historically contingent events
Fragility Claim: Possible worlds where we perform DME are (on average) more distant from the world we reasonably expect to cause with DME than worlds where we perform, e.g., physical enhancements are from the world we reasonably expect to cause with them.
Moral traits are prone to paradoxical effects
DME is irreversible and self-reinforcing
Complex Etiology
DME leads to more DME
DME increases motivation towards more
D
ME
DME changes
higher-order desires
DME leads to not wanting to revert
DME decreases motivation t
o
revert
Expert Opinion
Douglas
Higher-order desires may not be significantly connected to moral traits
Insofar as some motivations are connected to fundamental aspects of human morality deep moral enhancement is likely to change these motivations
Even if these were not changes on moral traits, selfreinforcing and irreversible changes on higherorder desires would still have moral significance
Etiology full of contingencies imply moral traits have a high susceptibility to such contingencies.
Current state required by moral enhancement is hard to provide
Not adapted to environment any more
Experience some of them as first person decontextualized unconditional emotions
Moral traits have a particularly high proclivity to unexpected disturbances
Increases in individual cooperation lead to decreases in group cooperati
o
n
Emergent properties: there should be more, yet unknown, cases of similar paradoxical effects
On most models, cooperation and parochialism could not have envolved without the other
Policies and practices that promote cooperation inside a group can easily lead to a conflict between groups
Groups that are highly cooperative internally will tend to be the least cooperative with other groups.
Competition between groups leads to increased contribution to the public good within-group and to increased group effectiveness.
Oxytocin increases cooperation but also ingroup favouritism.
Iterative DME makes pacifist Steve be pro human extinction.
DME makes pacifist Steve irreversibly apathetic.
Might be a fallacious SSA. Mild DME now might improve epistemic access to extreme DME later
DME will decrease future persons epistemic access to the desirability of m
o
re ME
We currently value moderate utilitarianism, but future enhanced midl utilitarianism might value extreme utilitarianism.
opposes
rebuts
EXAMPLE
EXAMPLE
EXAMPLE
rebuts
opposes
rebuts
data
Present Remotely
Send the link below via email or IM
Present to your audience
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
Do you really want to delete this prezi?
Neither you, nor the coeditors you shared it with will be able to recover it again.
Make your likes visible on Facebook?
Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.
Deep moral enhancement is risky
No description
by
Tweet


