Introducing 

Prezi AI.

Your new presentation assistant.

Refine, enhance, and tailor your content, source relevant images, and edit visuals quicker than ever before.

Loading…
Transcript

Not adapted to environment any more

On most models, cooperation and parochialism could not have envolved without the other

DME leads to more DME

DME increases motivation towards more DME

(DEEP) MORAL ENHANCEMENT IS RISKY

Current state required by moral enhancement is hard to provide

Moral traits have a particularly high proclivity to unexpected disturbances

Complexity Claim:

Moral Traits are likely to be relatively more complex than most other targets of human enhancement.

Fragility Claim: Possible worlds where we perform DME are (on average) more distant from the world we reasonably expect to cause with DME than worlds where we perform, e.g., physical enhancements are from the world we reasonably expect to cause with them.

DME is irreversible and self-reinforcing

Complex Etiology

Moral traits are prone to paradoxical effects

Epistemic Complexity

Distributive nature of moral traits on the brain

Expert Opinion

Moll, Crockett

Experience some of them as first person decontextualized unconditional emotions

Might be a fallacious SSA. Mild DME now might improve epistemic access to extreme DME later

Natural History: Evolutionary hangovers, population bottlenecks, random mutation

DME leads to not wanting to revert

Human History: The existence and spread of a particular cultural belief are the result of a long and complex series of historically contingent events

Etiology full of contingencies imply moral traits have a high susceptibility to such contingencies.

Expert Opinion

Douglas

Increases in individual cooperation lead to decreases in group cooperation

Emergent properties: there should be more, yet unknown, cases of similar paradoxical effects

Groups that are highly cooperative internally will tend to be the least cooperative with other groups.

Competition between groups leads to increased contribution to the public good within-group and to increased group effectiveness.

Oxytocin increases cooperation but also ingroup favouritism.

data

rebuts

DME decreases motivation to revert

DME will decrease future persons epistemic access to the desirability of more ME

EXAMPLE

Iterative DME makes pacifist Steve be pro human extinction.

EXAMPLE

DME makes pacifist Steve irreversibly apathetic.

EXAMPLE

We currently value moderate utilitarianism, but future enhanced midl utilitarianism might value extreme utilitarianism.

Policies and practices that promote cooperation inside a group can easily lead to a conflict between groups

DME changes

higher-order desires

opposes

Higher-order desires may not be significantly connected to moral traits

rebuts

Even if these were not changes on moral traits, selfreinforcing and irreversible changes on higherorder desires would still have moral significance

Complex Etiology

opposes

rebuts

Insofar as some motivations are connected to fundamental aspects of human morality deep moral enhancement is likely to change these motivations

Learn more about creating dynamic, engaging presentations with Prezi