a) win $2000 with probability 0.4;

b) win $1000 with probability 0.4, or

c) win $500 with probability 0.2, and

your utility can be modeled as an exponential function. The DM is indifferent to the following choice (assess Risk Tolerance R):

1. What is R?

2. What is the EV for this gamble?

3. What is the EU for this gamble?

4. What is the CE for this gamble?

5. What is the RP for this gamble?

6. Calculate U(CE) based on your answer for #3 to check your work.

7. For the following two alternatives, assume exponential utility function with R =900 and calculate:

•EV

•EU

•CU

•U(CE)

8. Which alternative do you recommend?

9. Perform a sensitivity analysis for CE for R = 500 to 2,000. Use a Data Table similar to how we did sensitivity analysis on swing weight.

10. Is the decision sensitive around R = 900?

NPV

Suppose that a test for a particular disease has a very high success rate:

Let D be the event that the patient has the disease, and T be the event that the test returns a positive result.

only 0.1% of the population have that disease: P[D] = 0.001

if a tested patient has the disease, the test accurately reports a 'positive', 99% of the time: P[D| T] = 0.99

if a tested patient does not have the disease, the test accurately reports 'negative', 95% of the time: P[T|DC] = 0.05

We now have all the information required to use Bayes' theorem to calculate the probability of the disease given the test. Using Bayes' theorem the probability of a true positive

Probability that a positive result is a false positive is about (1 −- 0.019) = 0.981. Despite the apparent high accuracy of the test, the incidence of the disease is so low (one in a thousand) that the vast majority of patients who test positive (98 in a hundred) do not have the disease.

Logically organize the spreadsheet

Use a control panel

(Not Req'd for MODA)

Parametrize everything

Type once, then cell reference

Use range names for readability

Use uniform indexing

Plan for Monte Carlo analysis

Manage model configuration

Document Excel file

MODA

Use 2 significant digits (0.52 of 1.0, 5.2 of 10, 52 of 100).

Build in checks

Weights sum to 1

Always score Ideal to check value functions

**DA Process**

**Framing**

Selecting the Appropriate

Decision Process

Lottery Instant Winners:

Best Case: win $20

Worst Case: lose $5

There exists a probabily of winning, p(w), that makes winning a guareenteed amount of money equivalent to playing the game.

Basics: Two homework assignments worth 50 points each. Each assignment will have 5 multiple choice questions. You will answer with the probability that answers A-D are correct; these probabilities must sum to 1.

Why we are doing this:

Develop your ability to use probabilities to assess your uncertainty.

Understand how your state of information effects your decisions.

The best strategy:

Improve your state of information by studying…

Strictly Proper Scoring: Assign your true beliefs!

How Quadratic Strictly Proper Scoring Works:

Qi(r) = 2ri - −r · r ∈ between [0 ,1]

Introductions

Course

Instructional Memo

Class Preparation Points

Probabilistic Grading

My Expectations

Introduction to Decision Analysis

Decision Analysis

Decision

Good Decision

AIAD Opportunities

**Deterministic Models**

**Qualitative Decision Analysis**

**Introductions**

Homework is due the class at start of class

Each day two cadets will be randomly selected to hand in their homework.

Excel Rand function in class spreadsheet will randomly (uniform distribution) choose who turns it in homework for grading.

Selection done with replacement

Students will run the random number generator.

The first cadet that hands in their homework last class will “F9” the next two

Instructor will grade and return the homework

**To Decisions and Decision Analysis**

**Uncertainty**

Don't be too quick to judge. . .

**Modeling Risk**

**Utility Function**

**Risk Attitudes**

**Project Time!**

**Decision Analysis**

**SE385**

**Modeling Uncertainty**

**Subjective Probability Assessment**

The Course. . .

The Other Instructors

The Students. . .

The Instructor. . .

Dr. Greg Parnell

MAJ Pierre Han

Mr. Dave Chennault

My Academic Background:

Education

B.S. from USMA, 1991, Mathematics Major with a Systems Engineering track

M.S. from GA Tech, 2001, Industrial Engineering

M.S. from New Mexico State University, 2007, Mathematics

Ph.D. from New Mexico State University, 2009, Industrial Engineering

Teaching Experience

USMA, Math, 2001-2004; 2009-2011

Core Courses: MA101/MA103/MA104/MA206

and MA491

Program Director for MA100 /101

Math Rocks!

USMA, DSE, 2011 -

Core Engineering Sequence Program Director

SE450, SE350, SE402/403

Husband: Russ

My Army Experiences:

Branched Quartermaster. . . . .Logistics now

7th Infantry Division, Fort Ord, CA 1992-1993

DISCOM Asst S1, Platoon Leader, Accountable Officer

JTF-LA

9th Infantry Regiment, Fort Lewis, WA 1994-1995

Bn S4, Asst SPO, Asst Reg S4

Operation Sea Signal

1st Infantry Division (Mech) Germany 1996-1999

Rear D Cdr, A/SPO, Company Commander

Bosnia, Macedonia

Functional Area 49/ Operaations Research/Systems Ananlyst (ORSA)

TRAC – White Sands Missile Range. WSMR, NM 2004-2006

Wargaming - Scenarios and Data Processing and Analysis

USF-I, Camp Victory and the United States Embassy Baghdad 2010-2011

J5 - Strategic Assessments

Operation New Dawn

LTC Libby Schott

Twins: Robert and Karl

Age 6 - 1st Grade

Daughter: Rhea

Age 13 – 7th grade

AIAD's

16 Nov 12

Lesinski, x5897

Determine if you have space for AIAD (TAC)

Review the AIAD Opportunities available (Jan)

Participate in the Systems AIAD Straw Poll (Jan)

Enter Preferences via AMS (Feb)

Receive Match - Contact Sponsor (Feb)

Complete Travel Admin Survey (Mar)

Complete special requirements (clearance, passport, etc.)

Coordinate Logistics for AIAD (lodging, etc.)

Participate in AIAD

Complete AAR and Travel Voucher (Aug)

Process

Systems AIADs provide the opportunity for our majors to gain valuable, real-world experience by working with military and civilian systems engineers, operations research analysts, and scientists.

Mr. Gene Lesinski, DSE AIAD Coordinator, Mahan 305

eugene.lesinski@usma.edu 938-5897

Web: http://usmasvdfcase6se/AIADs/AIAD_Homepage_lesinski.htm

CIS for Dean’s AIAD Application

Instructors or DACs

How can I find out more?

International Opportunities

SE Majors

Year Group

Capstones

Preferences

Academic Performance

8,9 Jan: Instructors Present AIAD Overview

14-18 Jan: Systems AIAD Straw Poll

22-29 Jan: Cadets Enter Preferences via AMS

22 Feb: USCC Summer Training Schedule

25 Feb: Begin Cadet Matching

8 Mar: USCC Deconfliction Complete

10-17 Mar: Spring Break

28/29 Mar: Systems AIAD Brief

25 May: Graduation

28 May: First AIAD Begins

6 Aug: Last AIAD Ends

General Rules for Matching

Key Dates

AY13 AIADs

Preferred Name

Company

Branch preference

What you did over the break

Sports/clubs/hobbies

What you hope to get out of the class

Lesson 1 Objectives

Read: DAH Chapter 3

Do: HW #2

NRC

Committee

Member

2012

NRC

Committee

Chair

2008

Editor/Author

Dr. Parnell has systems experience in space systems, managing aircraft and missile research and development (R&D) programs, and leading missile systems engineering. He teaches systems engineering, decision and risk analysis, operations research, and engineering management courses. He is a member of International Committee on Systems Engineering (INCOSE), American Society for Engineering Education, Institute for Operations Research and the Management Sciences (INFORMS), and Military Operations Research Society (MORS). He is former President of MORS and the Decision Analysis Society of INFORMS and a fellow of MORS, INFORMS, INCOSE, Society for Decision Professionals, and Lean Systems Society. He serves on several advisory boards and national academy committees. He is a retired Air Force Colonel.

Degrees

BS, Aerospace Engineering (State University of New York at Buffalo)

ME, Industrial & Systems Engineering (University of Florida)

MS, Systems Management (University of Southern California)

PhD, Engineering-Economic Systems (Stanford University)

Project Manager

Minuteman III

Reentry Vehicle

Chief, Missile Systems Engineering

Peacekeeper ICBM

Decision Analysis Advisor

Base Realignment

and Closure 2005

Member,

Technology and

Compliance Panels

NSA Advisory Board

Dr. Gregory S. Parnell

Professor of Systems Engineering

Department of Systems Engineering

**Foundations of Decision Analysis**

Read: DAH Chapter 5

Do: HW # 3

Selecting the Appropriate Decision Process

Next Time

DAH

C & R

PDH

http://usmasvdfcase6se:19387/Courses/AY132/SE385/default.aspx

Instructor,

D/SE, USMA

Graduate

Student

Company

Commander

2-13th AVN (UAS)

USMA ‘02

Rifle PL, Mortar PL,

CO XO, BN S3 Air

2-87 Infantry Battalion

MAJ Han is a 2002 graduate of the United States Military academy, and one of the newest members of the USMA DSE faculty. He most recently came to the department from George Mason University, where he studied Operations Research with a concentration in discrete event simulation, optimization, and military applications. His previous military positions include company commander, Battalion planner, company executive officer, and rifle/mortar platoon leader. He is a member of the Military Operations Research Society and the Omega Rho Honor society.

He is married to the former Megan Weaver, and has two sons, Scott (4) and Brandon (3).

Degrees:

BS, Engineering Management (USMA, ‘02)

MS, Operations Research (George Mason University, ‘12)

MAJ Pierre Han

Instructor, Systems Engineering

Department of Systems Engineering

B Hour

Alex

Andrew

Jon

Reese

Phillip

Mark

Eric

Tyler

Bobby

Ben

Asika

Matt

Jerome

Jonathan

Kevin

Chris

Henry

Cody

Class Preparation Points

r = vector of probabilities

Could be worse!

Probabilistic Grading

* See textbooks for other definitions.

Decision: An irrevocable allocation of resources.

Decision Analysis*: Decision analysis is a philosophy and a social-technical process to create value for decision makers and stakeholders facing difficult decisions involving multiple stakeholders, multiple (possibly conflicting) objectives, complex alternatives, important uncertainties, and significant consequences. Decision analysis is founded on an axiomatic decision theory and uses insights from the study of decision making.

Purpose: Provide insight to decision-makers faced with hard problems.

**Essential Definitions**

“A good decision is an action we take that is logically consistent with the alternatives we perceive, the information we have, and the preferences we feel.”

- Ronald Howard

Stanford University

Preferences: What you want.

Values, Time Preference, Risk Preference.

Information: What you know.

Any relevant models, relationships, or probability assignments that may be important characterizing the connection between decisions and outcomes.

Alternatives: What you can do.

There must be more than one.

Logic

(the seat)

Information

Preferences

Alternatives

Decision Maker

(committed to action)

Frame

**So what is a good decision?**

© 2006 J. Eric Bickel, TAMU

**Probalistic Grading**

Homework Handin

Understand the role of axioms in the development of theory

Understand the five decision analysis rules (axioms) and how they are used in decision analysis

Understand the scope of decision analysis

Understand the benefits of Value-Focused Thinking

Understand the taxonomy of value models

Lesson 2 Objectives

All of probability theory is based on these three simple axioms!

Three major axioms of probability

Sound theory is based on credible axioms.

Howard, R. A. (2007). Chapter 3. The Foundations of Decision Analysis Revisited. In W. Edwards, R. F. Miles , & D. von Winterfeldt (Eds.), Advances in Decision Analysis: From Foundations to Applications (pp. 32-56). Cambridge University Press.

1. The Probability Rule requires that you can fully describe any deal in terms of possibilities and probabilities. A possibility is a clear description of an event that may or may not occur. A set of possibilities (also called an outcome space) is complete if they are mutually exclusive (only one may occur) and collectively exhaustive (one of the set must occur). A probability is a number between 0 and 1 that expresses your degree of belief that a possibility will occur.

Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

Use of the five rules.

2. The Order Rule requires that you can rank any set of prospects in order of preference from best to worst. Indifference (i.e., equal preference) between two prospects is allowed. This rule implies transitivity of preference. If you prefer A to B and you prefer B to C, then you must prefer A to C (i.e., C cannot be both below A and above A in the preference ranking).

3. The Equivalence Rule requires that you can always create an uncertain deal involving two prospects such that you would be indifferent between receiving that deal and receiving a third prospect that is intermediate in your preference ranking between the two prospects in the deal. So, if you prefer A to B and prefer B to C, then the rule requires that there is a probability p such that you are indifferent between 1) a deal that gives you A with probability p and C with probability (1-p) versus 2) receiving B for sure. The probability p is called a preference probability because it is defined by your preferences rather than by your beliefs about the likelihood of any real events. Prospect B is said to be the certain equivalent of the deal involving A and C with the preference probability that you specify.

4. The Substitution Rule requires that your preference for a prospect will not change if an uncertain deal contained in the prospect is replaced by its certain equivalent, or vice versa.

5. The Choice Rule requires that if given the choice between two deals involving the same two prospects but with different probabilities, you must prefer the deal having the higher probability of receiving the more preferred prospect. Suppose that you prefer A to B and that you are offered two different deals. In Deal 1, you would receive either A or B with probabilities 40% and 60%, respectively. In Deal 2, you would receive either A or B with probabilities 25% and 75%, respectively. The rule requires that you must prefer Deal 1 because it offers the higher probability of the more preferred prospect A.

**Probability Rule**

**Order Rule**

**Equivalence Rule**

**Substitution Rule**

**Choice Rule**

Probability Theory

1657

1931

Subjective Probability

(Bayesian View)

On Reasoning in Games of Chance

Christiaan Huygens

Based on the work of Blaise Pascal and Pierre de Fermat

Example

One roll of a die:

Outcome Space: {1,2,3,4,5,6}

Mutually Exclusive - can't roll a 1 and a 2

Exhaustive - only 6 possible outcomes

P(1) = 1/6 . . .

0 <= P(1) <= 1

(bad) Example

Example

Example

Game A:

45% - Bag o'Money

55% - Empty Pockeys

Example

Interchangable!

No difference if you play the game, or take the $5

Game B:

35% - Bag o'Money

65% - Empty Pockeys

And the Winner is. . .

Class Example

Company Party!

Decision:

Outdoors or indoors?

LTC Schott's Solution

Given: P(Sun) = .4

P(Rain) = .6

Equivalence

Apply the five rules.

**Five Rules: Theoretical Foundations of DA**

**Scope of DA**

Modified from Keeney, Ralph L., Value-Focused Thinking: A Path To Creative Decisionmaking, Harvard University Press, Cambridge, MA, 1992, pp. 3-28.

Benefits of VFT

**Taxonomy of DA**

1738

1713

1763

1812

1933

The Art of Conjecture

Exposition of a New Theory on the Measurement of Risk

Essay towards solving a Problem in the Doctrine of Chance

Essay towards solving a Problem in the Doctrine of Chance

Theorie Analytique des Probilities

Truth and Probability

Jacob Bernoulli

Daniel Bernoulli

Rev Thomas Bayes

Pierre-Simon Laplace

Frank Ramsey

Andrey Kolmogorov

Mathematical Decision Theory

1944

Theory of Games and Economic Behavior

Foundations of Statistics

1954

Behavioral Decision Analysis

1954

"The Theory of Decision Making"

1974

"Behavioral Decision Theory"

1961

John von Neumann and Oskar Morgenstern

Leonard J. Savage

Ward Edwards

Amos Tversky and Daniel Kahneman

Ralph L. Kenney and Howard Raiffa

Ron Howard

1981

2007

1968

SODA

MODA

5 Rules

Read: DAH Chapter 6

Do: HW #4

Framing the Decision

(Problem Definition)

Next Time

Compare and contrast the three decision analysis processes described in the three textbooks

Clemen &Reilly’s Decision Process

Systems Decision Process

DA Handbook’s Decision Analysis Process

Task 1. Which of the following types of decision processes are each of the three decision processes?

Analytical

Advocacy

Dialog Decision

Decision Conference

Task 2. Identify the major similarities and differences between (Hint: Use the 6 requirements for a good decision)

Clemen &Reilly’s Decision Process

Systems Decision Process

DA Handbook’s Decision Analysis Process

Board Challenge

Dialog Decision

Analytical Advocacy

Four types of decision processes

Some common decision processes.

“Decisions are easy, it’s only the rationale that is difficult.” -Anonymous

“Nothing is more difficult, and therefore more precious,

than to be able to decide.” -Napoleon

C&R pgs. 3-4

Complexity

Possible courses of action / alternatives?

Possible outcomes?

Likelihood of outcomes?

Eventual consequences?

Uncertainty

It is everywhere…

Which ones are important?

Multiple Values & Objectives (conflicting)

Maximize quality vs. minimize cost

Which ones are important?

Different Perspectives

Who are the decision-makers / stakeholders?

Why are some decisions hard?

Systems decisions can be very complex since they involve many stakeholders, technical risks, large investments, and long time horizons.

Ralph L. Keeney, “Making Better Decision Makers,” Decision Analysis, Volume 1, Number 4, pp.193-204, December 2004

We need decision analysis for our most challenging decisions.

3.1 Understand when we need decision analysis

3.2 Understand the 6 requirements of a good decision

3.3 Understand the four types of decision processes

Analytical

Advocacy

Dialog Decision

Decision Conference

3.3 Compare and contrast three decision analysis processes

Clemen &Reilly’s Decision Process

Systems Decision Process

DA Handbook’s Decision Analysis Process

Objectives

Six requirements are necessary and sufficient to ensure a quality decision process.

Spetzler, C. & Keelin, T. (1992). Decision Quality: Opportunity for Leadership in Total Quality Management. Menlo Park: Strategic Decision Group

The Decision Analysis Creed:

“We work diligently to help you make good decisions and we pray you get good outcomes.”

Good decision quality increases the likelihood of good outcome quality.

Read: DAH # 7

Do: C &R 2.10 and 2.11

Crafting Decision Objectives

Next Time

Decision frame

Vision statement

Issue raising

Stakeholder Issue Identification Matrix

Decision hierarchy

Key Terms

Work in groups of 2 – One decision maker and one decision analysis

Frame the decision maker’s decision to go to graduate school at some future time.

a. Vision statement

b. Issue list

c. Stakeholder issue matrix

c. Decision hierarchy

Class Challenge 2

Work in groups of 3 – One decision maker and two decision analysts

Frame the decision maker’s investment decisions for the next seven - ten years.

a. Vision statement

b. Issue list

c. Stakeholder issue matrix

d. Decision hierarchy

Class Challenge 1

“A pessimist is one who makes difficulties of his opportunities and an optimist is one who makes opportunities of his difficulties.”

Harry S. Truman

“Opportunities multiply as they are seized.”

Sun Tzu

4.1 Understand the importance of framing the decision

4.2 Understand and be able to use the decision framing tools for a personal or professional decision

a. Vision statement

b. Issue list

c. Stakeholder issue matrix

c. Decision hierarchy

4.3 Compare and contrast Problem Definition in the Systems Decision Process with Framing the Decision

Objectives

Issue Identification Matrix

Vision Statement

Decision Hierarchy

Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

Decision framing tools

Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

This is a personal story that our decision analysis colleague Carl Spetzler likes to tell to illustrate the importance of agreeing on the frame for a decision. One morning his wife said to him, “I think it is time to repaint and carpet our house.” Carl looked around and saw that she was right. His response was, “Should we consider doing some remodeling in the kitchen and playroom first? After all, we are going to become empty nesters in six months.” Pretty soon they were talking about hiring an architect to redo their bedroom area. And as the ideas kept growing—along with the dollar signs—they concluded that perhaps they should consider selling the house and buying another that may already have these amenities. After a while, Carl’s wife asked, “How long do you intend to work before retirement?” Soon the question had grown to “So, what are we going to do with the rest of our lives?”

Painting and carpeting would be a two-month and $2,000 project, whereas planning the rest of their lives was a huge question that might take a couple of years to resolve in multiple rounds with large financial and quality-of-life implications. What decision should they focus on?

Importance of a Decision Frame

The decision frame is the critical first step in decision analysis. The decision frame helps us define the decision.

May be some big challenges

May be a challenge

The decision analysis must determine the decision frame.

Framing the decision

Team’s of two

One decision maker

Two decision analyst

Use Keeney’s 10 questions to develop your professional objectives for the next 7 years.

Structure your professional objectives in a value hierarchy (include financial objectives)

Board Challenge Problem

First draft was developed in one day with about 20 stakeholders at an intelligence agency.

Stakeholders all felt there decision issues would be included.

Data center location functional value hierarchy

Keeney, R. (1994). Creativity in decision making with value-focused thinking. Sloan Management Review, 33–34.

Strategic objectives: What are your ultimate or long range objectives?

A wish list: What do you want?

Alternatives: What is a perfect alternative, a terrible alternative, a reasonable alternative?

Problems and shortcomings: What is wrong or right with your organization or enterprise?

Consequences: What has occurred that was good or bad? What might occur that you care about?

Goals, Constraints, and Guidelines: What are your goals or aspirations? What limitations are placed upon you?

Different perspectives: What would your competitor or your constituency be concerned about? What do your stakeholders want? What do your customers want? What do your adversaries want?

Generic fundamental objectives: What objectives do you have for your customers, your employees, your shareholders, yourself?

Structuring objectives: Why is that objective important, how can you achieve it?

Quantifying objectives: How do you measure achievement of this objective?

Non financial objectives.

Keeney’s 10 questions

Decision objectives should be based on shareholder and stakeholder value.

Should private companies focus on shareholder or stakeholder value?

Could BP’s 2010 Deepwater Horizon explosion (triggering the worst offshore oil spill in the country's history) be considered an example of lack of consideration of stakeholder value? Whose values may not have been adequately considered?

How does a company provide value to shareholders and stakeholders?

How does a public organization provide value to its stakeholders?

Shareholder and Stakeholder Value

Management by objective works – if you know the objectives. Ninety percent of the time you don’t. Peter Drucker

“Our age is characterized by the perfection of means and the confusion of goals" A. Einstein

5.1 Understand the difference between shareholder value and stakeholder value.

5.2 Understand financial statements and objectives

5.3 Understand and be able to use Keeney’s 10 questions to generative decision objectives

5.4 Be able to use a value hierarchy to structure objectives

Objectives

Read : DAH Ch 8, C&R (Ch 7, 217-239)

Do : DAH HW # 6

Designing Creative Alternatives

Next Time

Help is logically organize our objectives and identify missing objectives

Improve communications with decision maker

Why do we use a value hierarchy to structure objectives?

Crafting decision objectives

Rt the net cash flow i.e. cash inflow – cash out

Balance Sheet Statement. A balance sheet is developed using standard accounting procedures to report an approximation of the value of a firm (called the net book value) at a point in time.

Income Statement. The income statement describes the changes to the net book value through time.

Cash Flow Statement. A cash flow statement describes the changes in the net cash position through time.

Net Present Value (NPV). To boil a cash flow time pattern down to a one-dimensional value measure, companies usually discount the free cash flow to a “present value” using a discount rate.

Financial statements and objectives

Read: DAH Ch 9 pp. 169-183

Do: HW# 7 - Review your groups Strategy Generation Table and identify the your blocks to creative thinking.

Due: PHW #1 1600 28 Jan 2013

Deterministic Modeling and Sensitivity Analysis -SODA (NPV)

Next Time

http://www.1000ventures.com/business_guide/crosscuttings/creativity_main.html

A useful view of creativity.

6.1 Understand the importance of creative thinking

6.2 Be able to create alternatives using creative thinking

6.3 Be able to use a strategy generation table to identify creative, doable alternatives

6.4 Understand blocks to creative thinking and be able to identify your blocks to creativity

Objectives

Net Present Value

Objective Identification

Objectives Structuring

Value (Objectives) Hierarchy

Working in groups of 3, use the Strategy Generation Process to develop three creative, distinct West Point curriculum alternatives.

Character

Leadership

Decision Making

Technology advances

Breadth vs. Depth

U.S debt

Interconnected international economy

Cyberwarefare

International security challenges

Cultures and language

WMDs and Terrorism

Peer adversaries vs. determined adaptive adversaries

Tailor education to cadet interests

Role of the Core

Accreditation

Cadet choice

Issues?

Hypothetical Dean’s Vision Statement

Redesign the West Point curriculum to better meat the academic goal: Graduates anticipate and respond effectively to the uncertainties of a changing technological, social, political, and economic world.

Classroom Challenge

Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

Small number of wellcrafted

alternatives for evaluation

Highly organized, analytic thinking

Individual and group creativity

Identify key decisions

Provide decision options

Alternatives that span the decision space

Strategy names that convey the key alternative features

Strategy Generation Table

Many creative ideas w/o judgment

Vision (Problem) Statement

Issues

Value Hierarchy

Convergent Phase

Divergent Phase

Strategy Generation Process: Generating good alternatives is a two phase process.

Questions

What is creativity? Are you a creative person?

How do you develop “creative, doable alternatives?

Designing creative alternatives

Intellectual and Expressive Blocks

• Choosing the “correct” problem solving language—we need the ability to go from one problem solving languages to another to include analytical/ mathematical, visual/special, analogy, etc.

• Flexibility versus fluency—we need both, flexibility to generate many alternatives, fluency to generate alternatives that are different in nature.

• Incorrect information—lack of, or bad, information expands rapidly and makes problem solving difficult;

• Inadequate language skills—poor communication

inhibits “selling” good alternatives to others.

Cultural and Environmental Blocks

• Taboos—we eliminate alternatives too quickly in our thought process because they seem culturally incorrect.

• Lack of humor in problem solving—humor can be inspiring.

• Reason and intuition: both sides are essential for creativity.

Left-handed and right-handed - both sides are essential for creativity—left side for order, reason, logic and mathematics, right side for imagination, artistry, and intuition.

• Tradition and change—it is hard to overcome the inertia of tradition

• Supportive versus non-supportive environments—physical, economic, and organizational support are needed to bring ideas into action.

• Autocratic bosses—may make it difficult to push new ideas through.

Emotional Blocks

• Fear of taking a risk—we tend to be afraid of making a mistake or failing.

• No appetite for chaos—we struggle to deal with ambiguity and

uncertainty.

• Judging rather than generating ideas—we tend to judge and analyze too early in the decision-making process.

• Inability to incubate—failing to give ideas time to mature before eliminating

them prematurely.

• Lack of challenge or excessive zeal—lack of motivation will inhibit creativity as will excessive motivation to succeed quickly.

• Reality and fantasy—ignoring one of these critical resources to creativity in problem solving.

Creative Thinking Blocks –How did these effect you? (Homework)

Perceptual Blocks

• Stereotyping—seeing what you expect to see.

• Difficulty in isolating the problem—solving the wrong problem.

• Delimiting the problem too closely—imposing too many constraints upon the problem and its solutions.

• Inability to examine the problem from multiple perspectives.

• Saturation—inability to perceive and retain all the information around us.

• Failure to use all of our senses—we should not neglect any sensory inputs.

Example

1.

2.

3.

I prefer apples over bananas and bananas over oranges.

Therefore I must prefer apples over oranges.

This is based on outcomes - not preferences. Axioms are about preference.

**Objectives and Values**

**Alternatives**

Should you use a DA Process?

**When**

Selecting the Appropriate

Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

Decision Analysis Process

Parnell, G. S., Driscoll, P. J., and Henderson D. L., Editors, Decision Making for Systems Engineering and Management, 2nd Ed, Wiley Series in Systems Engineering, Wiley & Sons Inc., 2011

Systems Decision Process

Clemen and Reilly Process

Modified From Clemen & Reilly, “Making Hard Decisions”

Decision Conference

Urgency

Importance

Difficulty

Spetzler, C., “Building decision competency,” Advances in Decision Analysis – From Foundations to Applications, Edwards, W., Miles, R., and von Winterfeldt, D., Cambridge Press, 2007

Phillips, L. D., “Chapter 19: Decision Conferencing”, Advances in Decision Analysis – From Foundations to Applications, Edwards, W., Miles, R., and von Winterfeldt, D., Cambridge Press, 2007

The approaches have advantages and disadvantages.

**Good Decision?**

What is a

Logic

(the seat)

Information

Preferences

Alternatives

Decision Maker

(committed to action)

Frame

© 2006 J. Eric Bickel, TAMU

Roughneck North American Oil Strategy

Geneptin Personalized Medicine

Data Call Center

Illustrative Examples

This decision looks easy!

Purpose

Perspective

Scope

1. Vision Statement

2. Issue-Raising

3. Categorization of the issues

4. Decision hierarchy

5. Values and Trade-offs

6. Initial Influence Diagram

7. Decision schedules and logistics

Framing Workshop

Elcom’s 32,000-square-foot facility located in Rockleigh, New Jersey

Last Industry Position: Vice President, Operations Elcom Technologies

My Background: Professional

Prior Employers: ADC Telecommunications – 11 years,

Roles: Engineer Manager, Senior Engineer

My Background: Professional

Elcom’s 32,000-square-foot facility located in Rockleigh, New Jersey

Last Industry Position: Vice President, Operations Elcom Technologies

My Background: Professional

Academic Positions: Visiting Professor, DeVry University. EET Program

Education: BS, Electrical Engineering, United States Military Academy, ‘85

MS, Electrical Engineering, University of MN, ’96

Military Branch/Functional Area: Signal/ 25C

Military Education: Signal Officer Basic ‘85. Signal Staff Officer ’86,

COMSEC ’86. Airborne ’83.

Previous Military Assignments: 1-12 INF Signal Officer.

124th Signal Company XO, Platoon Leader. 4ID Division Signal Staff

Professional Associations: IEEE, Association of Old Crows.

Registered Professional Engineer, Electronics and Computer Engineering

My Background: Academic and MIL

Born: Maxwell AFB, Montgomery, AL

Raised: Crosby, TX

Family: Wife, Wendy

Daughter, Calla (15)

Sons: Austin (21) Collin (19)

Home: Waldwick, NJ

C: (201) 280-9032

Me First…

Interests

Dr. Parnell's Solution

* Decision Analysis is based in the five axioms

Options:

Os

Or

Is

Ir

Ranked Options:

Os

Ir

Is

Or

http://www.kantola.com/Ron-Howard-PDPD-143-S.aspx

Examples

What are we doing?

Why are we doing this?

How do we know that we have succeeded?

Givens:

Planning horizon:

5 yrs

Russ retires at 22 YOS

Libby retires at 27 YOS

Focus on investment strategy for our retirement

Decisions:

What should our updated

budget/investment strategy look like?

Does Russ need a second job?

What is our real estate plan (should include what to do with the house(s) in Florida, invest in property in Montana?

Acquisition of "toys" (2nd car, boat, rv, jet skis?)

Addressed later:

Exact allocation of investment dollars

Rolling over TSP, Updating life insurance policies

Purchase details of real estate and/or of "toys"

Vision Statement:

We will decide what is the best investment strategy for our future. We need to do this in support our goal of being financially independent in our retirement. We know we will have succeeded if we are both satisfied and confident in our plan.

What impact will the current financial markets have on our investment?

Where do we want to retire?

Should Russ get a job?

What should we do with our house in Florida?

What we our role be in caring for our parents?

What level of support do we give our kids while they are in college?

What impact does going to O'Neill HS have on Rhea's future?

What lifestyle do we need to fund in retirement?

Should we leave our money in our current TSP or convert it when we retire?

Do we need additional life insurance?

Issues

What activities do we want to enjoy with the kids?

What does the real estate market look like in the next 10 years? in Cape Coral? In Great Falls?

health insurance?

Where should the twins go to High school? Is that the same as where we want to retire to?

Will the GI Bill still be funded when the twins reach college?

Can we count on social security?

Impact of changing tax laws?

LTC Schott's Solution

Can we count on our military retirement?

What will be the cost of living in Great Falls, MT? In Cape Coral, FL?

**MODA**

**SODA**

Single Objective Decision Analysis

Multi-objective Decision Analysis

Focus groups

Surveys

Interviews

chevron

Natural

Constructed

Direct

Proxy

LTC Schott's Soln

Hybrid Solns

Read: DAH Ch 9 pp. 169-183

Do: HW# 8 Complete the design of your spreadsheet and calculate the NPV for 2016 with a price of 30.

Due: PHW #1 1600 28 Jan 2013

Deterministic Modeling and Sensitivity Analysis - SODA II

Next Time

Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

**Decision Analysis Excel Modeling Guidelines**

Develop an influence diagram develop the model structure

7.1 Be able to develop an influence diagram given the decision frame.

7.2 Be able to convert an influence diagram to a deterministic value model in Excel to calculate Net Present Value

7.3 Understand and be able to apply decision analysis modeling guidelines.

Lesson 7 Class objectives

1st worksheet - Control panel

2nd worksheet - Calculations

Design a spreadsheet to model the new product decision

Future decisions

Given our new product

information, determine:

the launch date and

price that will maximize NPV

Decision Hierarchy

This decision

Done deals

New product implementation

design features

production plan

test plan

marketing plan

Corporate

mission & strategy

Corporate profit objectives

Product portfolio including the new product

A company has decided to develop and launch a new product. Develop an influence diagram for two decisions: the product launch year and the selling price of the product. There are several uncertainties. The product features depends on the launch year. The product’s market share is affected the new product features, and by its price. Sales depend on market share and market size. Revenue is calculated as unit sales times unit price. Unit costs and development expenses depend on product features. Expenses are equal to the sum of manufacturing cost (sales times unit cost), marketing expense, development expenses, and annual fixed expenses. Net present value is calculated based on cash flow (revenues – expenses) and discount rate.

Product Launch Decision Model

Appendix, B,, Influence Diagram, Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

We can use an influence diagram to develop the model structure

Read: PDH pp. 395-404, 409-412

Do: HW# 9: Complete the design of your spreadsheet and calculate the NPV for 2016 with a price of 30 and perform a deterministic sensitivity analysis (tornado diagram).

Deterministic Modeling – Multiple Objective Decision Analysis

Next Time

7.1 Be able to develop an influence diagram given the decision frame.

7.2 Be able to convert an influence diagram to a deterministic value model in Excel to calculate Net Present Value

7.3 Understand and be able to apply decision analysis modeling guidelines.

8.1 Be able to perform deterministic sensitivity analysis

Lesson 8 Class objectives

Install RSP

View

Model > Sensitivity > Parameters

Place cursor on NPV (b26)

Parameters > Identify > Show Current Parameters

RSP can be used to develop a tornado diagram

Read: PDH pp. 395-404, 409-412

Do: HW# 10: Complete the design of your spreadsheet and calculate the MODA value of all the UAS Alternatives. Develop a constructed scale that defines all five levels for the weather capability value measure.

Deterministic Modeling – MODA II

Next Time

UAS Functional Value Hierarchy

SE301 Review Question

Is this the logical order of the functions?

UAS Functional Hierarchy

9.1 Be able to convert a functional value hierarchy to a multiple objective decision analysis model in Excel. Understand the role of

a. Value functions

b. Value measure scales

c. Scores

d. Single dimensional value (using ValuePL)

e. Swing weight matrix

f. Weighted value

g. Total Alternative value

h. Value component chart

Lesson 9 Class objectives

Read: PDH pp. 395-404, 409-412

Do: HW# 10: Complete the design of your spreadsheet and calculate the MODA value of all the UAS alternatives. Develop a constructed scale that defines all five levels for the weather capability value measure.

Deterministic Modeling – MODA II

Next Time

Lesson 10 Class Objectives

Influence Diagrams

High level "blue print" for your model

Use forward and backwards modeling

Course Standards

**Sensitivity**

Starting Point: ID

1st worksheet - Control panel

2nd worksheet - Calculations

NPV Example

Assume Product Lifespan goes until 2020

Install RSP

SE301 Review Question

Label each level.

Develop a value measure for each tier 3 box.

UAS Example

UAS Functional Value Hierarchy

UAS MODA Model Design

SE301 Review Questions.

Why do we include the Ideal Alternative?

How can we use this chart to develop a hybrid Alternative?

Value Components Chart

Operational data

Tests

Simulations

Models

Expert opinion

SE301 Review Questions.

What are the five sources of Alternative scores?

What sources would we use for our four Alternatives for each value measure?

Scores

SE301 Review Questions.

What are swing weights?

Can swing weights be assessed before we get the scores?

Why do we normalize the weights?

Swing weight matrix

SE301 Review Questions.

What is the purpose of a value function?

What do value functions measure?

What part of the equation below is shown in the curves to the left?

Value functions

Questions:

How do we create this chart?

How do we use this chart?

What is the “unavailable value”?

What do we have to do to get an alternative with higher value than the hypothetical best?

We define the hypothetical best “hybrid” alternative

Value-Focused Thinking to create hybrid alternatives

SE301 Review Questions.

How is weighted single dimensional value calculated?

Where does the unnormalized weight come from?

What part of the equation below is shown calculations to the left?

Weighted single dimensional value.

SE301 Review Questions.

How is single dimensional value calculated?

What part of the equation below is shown calculations to the left?

Calculation of single dimensional value.

SE301 Review Questions.

How is total value calculated?

What part of the equation below are the calculations above?

Total value calculation.

Homework

Develop a constructed scale that defines all five levels.

SE301 Review Questions.

What type of value measure scale is used for all weather capability?

If we did not use this measure what would be the alternative?

Constructed scales

UAS Example

in Detail

10.1 Be able to create a direct, constructed scale for a value measure by identifying and defining the key distinctions that create value

10.2 Be able to create and interpret a Value Component Chart and use Value-Focused Thinking to create Hybrid alternatives

10.3 Understand the important linkages between the strategy table and the value model

10.4 Be able to perform and interpret a deterministic weight sensitivity analysis

10.5 Understand the concepts of Decision-Focused Transformation and be able to use DFT to compare two alternatives.

We use DFT to

Eliminate the common value and the unavailable value.

Rescale the available value to the DFT value space.

Question: How do we interpret the DFT chart?

Once we have completed our development of hybrid alternatives (using Value-Focused Thinking), we want to compare individual alternatives.

Decision-Focused Transformation (DFT)

What is the same about these two constructed scales?

What is different?

Homework Review : All weather constructed scale.

What is the x-axis?

What is the y-axis?

Why is the Ideal always 10?

How do we interpret this chart?

Swing Weight Sensitivity – UAV Endurance in Min

“No value model survives first contact with the strategies (and vice versa)”

Quantitative

Qualitative

Value Model

Our objective is to develop creative, doable strategies to achieve significant value. We use decision analysis artifacts (decision hierarchy, vision statement, issue raising, and stakeholder issue identification matrix) to develop two integrated products:

dx = decision

a-n = decision options

The strategy generation table must be linked to value model.

Changing the decision option in a decision column should affect one or more value measure scores.

If we score all strategies, and a value measure has no scores, we need to add another column in the table or we may not need the value measure.

If one decision column does not affect any value measure scores we need to delete the column or add value measures.

Strategy Generation Table

Lesson 12

WPR (lessons 1-10)

Lesson 13

Group WPR

Coming Attractions

What is P[ IED | Sensor Detects ]?

A new sensor has been developed to use in UAVs in Afghanistan to detect IEDs on or near a road. The following data is estimated:

Probability of detecting given an IED is on the road is 90%

Probability of false detection is 20%

Probability of IEDs on the road is 15%

Calculate the all conditional and marginal probability distributions using the tree flipping technique.

Class Example - Using Excel

P [ N ] P [ T | N ] = P [N T ] = P [ N | T ] P [ T ]

Solve for all the probabilities by "flipping the probability tree”

What is the probability the individual has the disease given they test positive?

Suppose that a test for a particular disease has a very high success rate:

Only 0.1% of the population have that disease (probability 0.001)

If a tested patient has the disease, the test accurately reports a 'positive', 99% of the time (probability 0.99)

If a tested patient does not have the disease, the test accurately reports 'negative', 95% of the time (probability 0.95).

We need to update our probabilities when we receive new information.

11.1 Review axioms of probability

11.2 Understand the role of Bayes Law in updating our probabilities given new information

11.3 Understand the important applications of Bayes Law

Lesson 11 Objectives

Bayesian Networks are specialized software to apply Bayes Law to large scale problems.

Bayesian network application include “computational biology and bioinformatics, medicine, biomonitoring, document classification, information retrieval, semantic search, image processing, data fusion, decision support systems, engineering, gaming and law” http://en.wikipedia.org/wiki/Bayesian_network#Applications

Medicine

Law

Fault Detection

Intelligence

Surveillance

Damage Assessment

Homeland Security

Etc.

Applications of Bayes Law

P [ N ] P [ T | N ] = P [ N T ] = P [ N | T ] P [ T ]

Solve for all the probabilities by "flipping the probability tree”

http://en.wikipedia.org/wiki/Bayes_theorem

Using Bayes Law to update probability assessment based on new information.

McGrayne, Sharon Bertsch, The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy, 2011

Beginnings to today….

Thomas Bayes born about 1702 in London

Studied logic, theology at University of Edinburgh

Presbyterian minister at Tunbridge Wells, Kent, 35 miles southeast of London

Died at Tunbridge Wells in 1761

Bayes' solution to a problem of "inverse probability" was presented in the Essay Towards Solving a Problem in the Doctrine of Chances (1764), published posthumously in the Philosophical Transactions of the Royal Society of London

Transmitted to the Royal Society by his friend Richard Price who found the unpublished paper after his death

Inverse probability

“The probability approach which endeavors to reason from observed events to the probabilities of the hypotheses which may explain them” (http://stats.oecd.org/glossary/detail.asp?ID=3686)

Uncertainty and Bayes Law

This approach works well for two outcomes but what if there are multiple outcomes?

Suppose we consider 100,000 people.

Since 0.001 have the disease, we would expect 100 have the disease and 99,900 not to have the disease

The test detects 99% of the people with the disease (true positive) (99) and the false positives are 5% (4,955)

Therefore, the probability of a disease given test positive = 99/(99 +4,995) = 0.019 (approx)

Intuitive solution

How do we calculate the probabilities we need?

Information

Performance

P[N | T]

P[T]

The probabilities we need.

The probabilities we have.

P[N]

P[T| N]

Information

Performance

The initial probability data we are provided

The probability data we need in the decision tree

In systems engineering, Bayes Law plays an important role in value of information calculations using decision trees (later class).

Representativeness

The more object X is similar to class Y, the more likely we think X belongs to Y

Availability

Too much weight on vivid, striking, or widely reported events

Too much weight on most recent events

Acceptance of illusory correlation

Anchoring

Initial estimated values affect the final estimates, even after considerable adjustments

Motivational

When making probability judgments, people have incentives to provide estimates that will benefit themselves

Major Human Probability-Assessment Heuristics/Biases

(Tversky and Kahneman, 1974)

Individuals can learn and improve performance

Awareness of heuristics and biases improves subjective probability assessment

Structured DA techniques may help avoid problems

Problem decomposition may make things easier -

More assessments required, but each assessment is easier

Implications of Heuristics and Biases

Heuristics can be viewed as relatively simple rules to make judgments and solve problems.

General rules of thumb

reduce time and effort to make decision

reasonably good estimates

Often lead to systematic biases

Heuristics

Assessment with one probability is too aggregated to be meaningful. We should use decomposition.

8. Indian Point Unclear Reactor. You have been asked to do a risk assessment of the nuclear plant at Indian Point. You recall that stakeholder analysis is important and probability is very important. So you make an appointment with the plant manager, explain the axioms of probability and ask him “What is the probability you will have a melt down?” Is this a recommended approach?

Yes ____ No ____ Why?

Decomposition

a. Your boss insists that you give her ONE number to estimate your sales for the next year. What number do you give her? _____M

b. Next, suppose your boss says, “We expect a very difficult year due to the economy, if your sales estimate is not at least $60M, you will have to lay off some of your salespeople.” What number would you give her? _____M

7. Sales Estimates. Suppose you are a salesman and your belief about next year’s sales in shown in the following probability density function. Of course, this means that there is no probability that sales will be less than 20 or more than 80. In addition, the area integrates to 1.

Incentives to Provide Inaccurate Probabilities

Trying to meet management expectations

Managing your boss’ expectations

Motivational Bias

Potential for motivational bias.

6. Railguns. Railguns are a potential future weapon that use electromagnetic instead of chemical energy. The technology has not been fully developed for military applications. Suppose the leading developer of Railgun’s is Railguns R Us. Recently, the company’s Chief Technology Officer testified to Congress that there was a 95% chance that Railgun weapons would be deployed in the military in 5 years. Should you be concerned about this estimate? Yes ___ No ____ Why?

Motivational Bias

7 letter words ending in “ing” are more available us.

5. Which 7-letter word form is most common?

_ _ _ _ _ n _?

_ _ _ _ i n g ?

Availability

People draw conclusions based on representative characteristics and often ignore relevant facts

3. Dawes did research on Chief Executive Officer (CEO) pet ownership as children. His survey reported a link between childhood pet ownership and career success. 94% of surveyed CEO’s owned a pet (dog, cat, or both) as children and the respondents stated that pet ownership helped them build good character traits of a CEO. Which conditional probability is described here?

a. P[CEO | PET] = 0.94

b. P[PET | CEO] = 0.94

Representativeness

People draw conclusions based on representative characteristics and often ignore relevant facts

1. Which six coin toss sequence is more likely?

a. HTHTTH

b. HHHHHH

2. Suppose you are playing roulette in Las Vegas. After a run of 4 reds in a roulette, what would be your next bet?

Representativeness

Arbitrary data anchored the thinking!

4. Suppose I randomly spin a probability wheel:

Without checking the internet, do you believe that the percentage is

More than % _____

Less than % _____

What do you believe the percentage of African countries are in the United Nations? _____%

Anchoring

Scientific principles of data collection

Documented, peer review

Must adhere to standards:

As probabilities are judgments, there will be controversy, and we must document and justify to show that the experts and the environment were set to reduce bias

As protocol, every assessment should include:

Background

Identification and Recruitment of Experts

Motivating Experts

Structuring and Decomposition

Probability Assessment Training

Probability elicitation and verification

Aggregation of experts’ probability distributions

Protocols: Pulling it all Together

Lesson 15

Read: C & R (Ch. 8 pg.295-311)

Do Problems: 8.1, 8.4, 8.11

Coming Attractions

Centers for Disease Control, www.bt.cdc.gov/Agent/Agentlist.asp

Decomposition Example: DHS Bioterrorism Risk Assessment, 28 bioagents were considered.

14.1 Understand the heuristics and biases we use when we process uncertain information

Representativeness

Availability

Anchoring

Motivational

14.2 Understand decomposition and how it applies to subjective probability assessment

Lesson 14 Objectives

P(Grad) = P(Grad|HS)P(HS) + P(Grad|LS)P(LS) = (.6)(.8)+(.4)(.95) = 0.86

.05

.95

.4

.8

.2

.6

No Grad|HS

Grad|HS

No Grad|LS

Grad|LS

LS

HS

No Grad

Grad

Example:

What is probability I (cadet) will graduate?

with

What is prob. I will graduate if the dean raises academic requirements?

What is prob. I will graduate if the dean reduces academic requirements?

Scenario #1: Conditioning

DHS (Department of Homeland Security). 2006. Bioterrorism Risk Assessment. Biological Threat Characterization Center of the National Biodefense Analysis and Countermeasures Center. Fort Detrick, Md.

The chart is a simplification of the 17-step event-tree (18 step with consequences) that could lead to the deliberate exposure of civilian populations for each of the 28 pathogens.

The biannual DHS Bioterrorism Risk Assessment (BTRA)

used probabilistic risk analysis with event trees.

All of probability theory is based on these three simple axioms!

“When the facts change, I can my opinion. What do you do sir?”

John Maynard Keynes, Economist

Three major axioms of probability

Sound theory is based on credible axioms.

1. If the coin flips are independent, both of are equally likely. However, it is more likely that we get 3 Hs than 6 Hs.

2. The three reds should not influence your bet. This is a very small sample.

Solution

3. P[PET | CEO] = 0.94

Solution

4. What percent of African countries are part of the UN? Answer: (at the time it was 35%)

Wheel rigged to 10 average guess was 25%

Wheel rigged to 65 average guess was 45%

Solution

Solution

Probabilistically, a. must have more words!

(b. is a subset of a.)

Solution

Fear that individual will be held accountable for “achieving the outcome”

Pre-test

Decomposition

Definition – Breaking a probability assessment into smaller and more manageable chunks

Purpose – To make probability assessments easier and to give a clearer view of the uncertainty in a decision problem!

Potentially appropriate in three different scenarios:

Scenario 1: Uncertain event can be conditioned on another event

Scenario 2: Multiple possible causes for uncertain event

Scenario 3: Uncertain event results from a chain of causes

P(Grad) = P(Grad|HS)P(HS) + P(Grad|LS)P(LS) = (.6)(.8)+(.4)(.95) = 0.86

Example:

What is probability I (cadet) will graduate?

with

What is probability I will graduate if the dean raises academic standards (HS)?

What is probability I will graduate if the dean reduces academic standards (LS)?

Scenario #1: Conditioning

Scenario #2: Chain of Causes

Example:

What is probability oil rig explodes?

with

What is probability of critical pipe failing?

Then safety measures failing?

Then no-one noticing in time?

Example:

What is probability you will not graduate?

Scenario 3: Multiple Causes

Solution:

What is probability of a serious injury?

What is probability of failing to meet academic requirements?

What is probability of cheating incident?

. . . etc.

Examples

http://toughmudder.com/worlds-toughest-mudder-series-finals/worlds-toughest-mudder-qualifiers/

http://eb.gmnews.com/news/2012-12-06/Front_Page/Brothers_spend_two_days_Roman_through_obstacles.html

Heads I win, Tails you lose.

Tossing a Coin: Can you use Bayes Law to determine whether or not a coin is fair?

What about a dice?

Tossing die and rolling 1's

Flipping 3 Heads

LTC Schott's Solution

9.Probability. Based on extensive studies, it has been found that 1% of the population has Some Horrible Disease (SHD). A new test for SHD has just been released; this test has been shown to be 98% accurate in clinical trials. If a person takes this test and the results come back positive for SHD, then what are the chances that the person actually has SHD.

a.33%

b.19%

c.82%

d.64%

10.Probability. In 1982 John Hinckley was on trial, accused of having attempted to kill President Reagan. During Hinckley’s trial, Dr. Daniel R. Weinberger told the court that when individuals diagnosed as schizophrenics were given computerized axial tomography (CAT) scan, the scans showed brain atrophy in 30% of the cases compared with only 2% of the scans done on normal people. Hinckley’s defense attorney wanted to introduce as evidence Hinckley’s CAT scan, which showed atrophy. The defense argued that the presences of atrophy strengthened the case that Hinckley suffered from mental illness. Given that approximately 1.5% of the people in the United States suffer from schizophrenia, what is the probability that Hinckley suffered from schizophrenia?

a.33%

b.19%

c.82%

d.64%

Would you recommend admitting the test results as evidence?

Pre-Test Questions

Familiar Questions?

9. 33%

10. 19%

Solution

Homework Handin

15.1 Understand why decision analysis uses probabilities to describe uncertainty

15.2 Distinguish between the frequency and the subjective views of probability

15.3 Understand that people do a poor job of estimating extremes and how structured techniques can improve their ability to estimate probabilities

15.4 Apply discrete methods to assess subjective probability:

Direct assessment

Reference lottery

15.5 Apply continuous methods to assess subjective probability:

Fractile Method

Potential Outcome Method

15.6 Compare fractile and outcome methods for consistency and revise

Lesson 15 Objectives

Provide the 0.01, 0.5, and 0.99 fractiles

What is the length of the world’s longest central span in a suspension bridge?

The length of the Nile River?

The year Thomas Bayes was born (Bayes Law)?

Diameter of the moon?

Year with the highest unemployment rate between 1948-2010?

How good are we at estimating uncertainties?

Find value of p for which expert is indifferent to following choice for the event of the 2013 Army-Navy Football game.

Discrete Method #2:

Reference Lottery

The probability wheel is a commonly used visual model to help with direct assessment.

Probability Wheel

Ask the expert what his/her belief is

Must clearly define the event

Advantages

Easy

Disadvantages

Expert’s may not be comfortable providing probabilities without training

Discrete Method #1:

Direct Assessment

In what order should we assess the three probabilities? Why?

Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Decision Analysis Handbook, Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

The P50 is the 0.50 fractile, the number x, such that P [ X < x ] = 0.50

Documentation is important define the uncertainty and estimate the extremes.

The 0.05 fractile is the number x, such that P [ X < x ] = 0.05

Which technique is more subjective?

Frequency: If we did this many times, what is the long-run frequency of the outcomes of interest?

Relies on past data or the ability to conduct experiments.

What is the probability of “heads” in a sequence of 6 flips?

Unable to provide information for many of our most important problems

Subjective: What is your degree of belief concerning the outcomes of interest?

Relies on subjective interpretation of all relevant information to include data, experiments, simulation results, and expert opinion

What is the market for a new product, e.g., iPhone?

What is the probability a new technology will provide a certain performance level?

Two views of probability

We have

Deterministic value model

Alternatives

Tornado diagram that identifies the most sensitive parameters

We need to use

Probability distributions to quantify uncertainty for the most sensitive parameters

Probability assessment protocols to obtain this information

We must consider

Heuristics and biases

How good the expert is at quantifying uncertainty

Transition from deterministic to probabilistic modeling.

Common verbal phrases are not reliable for decision making.

Need

Subjective probability estimate for continuous random variable

e.g., oil in well, casualties in a conflict

Approach

Find an approximation of CDF

Determine some points on CDF

Method #1: specify cumulative probability (fractile),assess potential outcome

Method #2: specify variable potential outcome, assess cumulative probability (fractile)

Connect the dots

Continuous Assessment Methods

The weather forecaster says it is likely it will rain tomorrow

The Gold Coats say it is likely they will fix your computer in one hour

The weather forecaster says it is very likely it will rain tomorrow

The Gold Coats say it is very likely they will fix your computer in one hour

Should we use words instead of probabilities to describe uncertainty?

Original Estimate

7.5 K gal!

Cleanup technology effectiveness depends on the amount of the spill.

Probability assessment elicited from the manager of the degreaser during the time of operation.

Below ground

Drain

Chemical Separation Facility

Aquifer

Ground Level

Degreaser

Large Dept of Energy Building

DNAPL Plume

Probability

DNAPL Volume (1000 Gallons)

500

450

400

350

300

250

200

150

100

50

0

1

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

Cumulative Probability Distribution of DNAPL Volume

Motivating Example: Paducah KY

Spill Volume Uncertainty

What should we do next?

Comparing the two techniques can provide insights about inconsistencies.

Continuous Method #2

Assess Probabilities for Potential Outcomes -- Example

Assess probabilities for potential outcomes

Pr[X 0] = 0

Pr[X 1,000,000] = 1

Pr[X 100,000] = 0.05

Pr[X 900,000] = 0.95

Pr[X 250,000] = 0.08

Pr[X 750,000] = 0.92

Pr[X 500,000] = 0.50

Connect the dots

Find extreme values

Pr[X 0] = 0

Pr[X 1,000,000] = 1

For each fractile, assess potential outcomes X1.. X5 such that:

Pr[X X1] = 0.1

Pr[X X2] = 0.9

Pr[X X3] = 0.25

Pr[X X4] = 0.75

Pr[X X5] = 0.5

Connect the dots

Continuous Method #2

Fractiles - Example

Lesson from behavioral research: Start at extremes first, and work to 0.5.

Find extreme values

Pr[X 0] = 0

Pr[X 1,000,000] = 1

For each fractile, assess potential outcomes X1… X5 such that:

Pr[X X1] = 0.1

Pr[X X2] = 0.9

Pr[X X3] = 0.25

Pr[X X4] = 0.75

Pr[X X5] = 0.5

Continuous Method #1

Fractiles - Example

Find approximation for CDF

X = barrels of oil in well

Assess a continuous distribution

0

1

0.05

0.95

0.2

0.80

0.50

Lesson from behavioral research: Start at extremes first, and work to 0.5.

Assess probabilities for potential outcomes

Pr[X 0] =

Pr[X 1,000,000] =

Pr[X 100,000] =

Pr[X 900,000] =

Pr[X 250,000] =

Pr[X 750,000] =

Pr[X 500,000] =

Continuous Method #2

Assess Probabilities for Potential Outcomes -- Example

QUESTION:

Write your probability for each of the following event:

Provide the 0.01, 0.5, and 0.99 fractiles - Solutions

What is the length of the world’s longest central span in a suspension bridge? Akashi-Kaikyo Bridge, Japan, 6529 feet, 1991 meters

The length of the Nile River? 6,650 km (4,130 miles) long – longest river in the world

The year Thomas Bayes was born (Bayes Law)? 1702

Diameter of the moon? 2,159 miles or 3,474 km.

Year with the highest unemployment rate between 1948-2010? 1982, Reagan

Solutions

See DA Handbook Chapter 10 for more complete discussion.

In the early 2000’s Dr. Parnell worked on a research project to cleanup a underground DNAPL (bad chemical) caused by two incidents at the DOE nuclear site at Paducah, KY. First, the floor of the degreaser which used the DNAPL to clean equipment with radioactivity had rotted out. Second, the underground pipe designed to transport spilled chemicals to a separation facility had a fork lift hole in the BOTTOM of the pipe. The initial estimate of the spill into the ground was 7,500 gallons. The concern was the that DNAPL was in the aquifer. The DNAPL seeped into the ground until the ground above the pipe caved in. About 20 technologies were considered for the cleanup. The technology’s effectiveness were very dependent on the size of the spill. Several month’s into the study, Dr. Parnell noted that the cleanup site evacuations seem to suggest a much larger spill. The engineer who provided the data stated that he had looked at the records but they were very poor. The above distribution was assessed in a one hour session with the manager of the degreaser. The results of the assessment fundamentally changed the study.

Procedure:

Offer p’s that favor first one side, then the other

Begin with extremely wide brackets and gradually converge on a p that yields indifference

Go slowly; allow DM to think hard

Consider using “probability wheel” to visualize p:

Start with the following questions. Use “probability wheel” to visualize p:

Would you rather have a 90% chance of winning $1,000 or would you rather receive you outcome based on the results of the game. Presumably, they will take p = 0.90

Would you rather have a 10% chance of winning $1,000 or would you rather receive you outcome based on the results of the game. Presumably, they will take outcome of the game.

Narrow the questions until they are indifferent.

Conclude P[Army wins] = p

Decision trees without information (w/o and w/PT)

BRING LAPTOP TO

EVERY CLASS

Read: C & R (Ch. 3 pg. 69-74, Ch. 4 pg. 115-119)

Do: 3.9.a, 3.11, 4.4, 4.8

Next Time

Select an important uncertainty that will impact your future

Assess a continuous probability distribution two ways:

Fractile Method

Potential Outcome Method

Compare fractile and outcome methods for consistency and revise

Board Challenge

Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Decision Analysis Handbook, Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

Use the template

16.1 Assess a continuous probability distribution two ways:

Fractile Method

Potential Outcome Method

16.2 Compare fractile and outcome methods for consistency and revise

Lesson 16 Objectives

**Decision Trees**

17.1 Understand components of decision trees.

17.2 Be able to construct a decision tree from an influence diagram and solve for the optimal solution.

17.3 Understand the strengths and weaknesses of expected value

17.4 Be able to plot and interpret an risk profile using a cumulative probability chart

Deterministic dominance

Stochastic dominance

No dominance

Lesson Objectives

How can we use this information to obtain insights?

Plot the risk profiles and cumulative risk profiles for each decision

2015

2016

2017

Board Problem 1c

Add the probabilities and outcomes to your decision tree and solve for the best decision using the decision tree algorithm.

Board Problem – 1b

BRING COMPUTER

HW#18

Read: Precision Tree Quick Start Tutorial

http://www.palisade.com/QuickStart/EN/PrecisionTree/

(PDF and/or Video)

Do: HW#18 by hand or with Precision Tree

Next Time

Draft the influence diagram

A company must decide to develop a new product, continue to market the current product, or cancel the product line. The current product demand is expected to be the same as last year or worse. There are two new product uncertainties. The first uncertainly is new product performance. The performance may be high or low. The second uncertainty is demand. The new product demand may be large or small.

Board Problem 2a

Expected value is a summary statistic that hides the extreme outcomes. This is why we must consider risk.

Limitations of Expected Value

Expected Value- is “simply the weighted average of the possible outcomes…the weights being the chances with which the outcomes occur.” - Clemen & Reilly pg. 116

Another interpretation is that expected value is what we would expect to happen on average if we were to repeatedly observe the outcome of an uncertainty.

Most of the time, the expected value isn’t even a possible outcome. For example, the expected value of rolling one die is 3.5…

SOLVING FOR EXPECTED VALUE IN A DECISION TREE:

Construct the decision tree.

Add probabilities to the outcomes of the uncertain variables and values to end of the tree based on the path thru the tree.

“Average out and fold back the tree” from right to left.

Calculate expected value of chance nodes.

Choose best alternative at decision nodes.

Label nodes appropriately (expected values and decision path)

Continue until expected value can be assigned to root node.

DEFINITION: A discrete uncertain quantity’s expected value is the probability-weighted average of its possible outcomes. If X can take on any value in the set {x1, x2,…, xn}, then the expected value of X is the sum of x1 through xn, each weighted by the probability of its occurrence. -Clemen & Reilly pg. 259

How to solve a decision tree

Construct a decision tree from the following influence diagram

Board Problem 1a

Statistician drowns crossing the river.

The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty, by Dr. Sam Savage

with Illustrations by Jeff Danziger

http://flawofaverages.com/

The flaw of averages

Convert the influence diagram to a decision tree

using the data in the Board Problem 2 file and solve.

Board Problem 2b

Is there dominance?

If so, what kind, which strategy, and why?

Example C

Now:

We should use expected value to decide if we are risk neutral across the range of possible outcomes (or we are a large company)

If we are risk averse or risk seeking across the range of possible outcomes, we must consider risk.

WOULD YOU MAKE THIS DECISION?

EV = $800

(0.9)($12K) + (0.1)(-$100K) = $800

-$100,000

$12,000

$0

0.1

0.9

Example: Limitation of EV

Is there dominance?

If so, what kind, which alternative, and why?

Example A

Is there dominance?

If so, what kind, which strategy, and why?

Example D

Is there dominance?

If so, what kind, which alternative, and why?

Example B

“For a Max Problem: Alternative R stochastically dominates alternative Q if, for any possible outcome, R has an equal or better chance of resulting in an outcome that is equal or better.”

Stochastic Dominance

For a Max problem: “Alternative P deterministically dominates alternative Q if the worst outcome of P is as good as (or better than) the best outcome of Q.”

Deterministic Dominance

Influence Diagrams

High level "blue print" for your model

Use forward and backwards modeling

Influence Diagrams

High level "blue print" for your model

Use forward and backwards modeling

Decision Trees

Basic Nodes

Example - 3.9a (your homework. . .)

Build from left to right

Expected Value

Example - 4.8 (from your homework. . .)

Board Problem: Product launch. . . iPhone22??

Board Problem 2

Cumulative Risk Profiles

R stochastically dominates Q.

“For a Max Problem: Alternative R stochastically dominates alternative Q if, for any possible outcome, R has an equal or better chance of resulting in an outcome that is equal or better.”

Stochastic Dominance. Down and to the right for a max.

One outcome is the same but the rest of the outcomes are better.

The two plots intersect so there is no dominance.

Alternative B has both the worst outcome and the best. This is a common occurrence. Many times new alternatives have both high potential and high risk.

Deterministic Dominance, B is the preferred alternative since we are seeking the min cost solution.

How reliable is Expected Value?

1. Read: Precision Tree Quick Start Tutorial (PDF and/or Video)

http://www.palisade.com/QuickStart/EN/PrecisionTree/

2. Solve the following problem with a decision tree by hand or with Precision Tree. Construct a decision tree from the following influence diagram and solve for the highest expected value decision.

Homework

Precision Tree

18.1 Be able to formulate and solve a decision analysis problem using Precision Tree software and interpret the results

18.2 Be able to interpret a Precision Tree Risk Profile using a Cumulative Chart

Deterministic dominance

Stochastic dominance

No dominance

18.3 Be able to perform and interpret one way and two sensitivity analysis using Precision Tree

Lesson 18 Objectives

MEET IN THE LAB - MH 203

Lesson 19 - Practice Graded Lab

Bayes Law

Decision Tree analysis with Precision Tree

Lesson 20 – Graded Lab

Bayes Law

Decision Tree analysis with Precision Tree

Next two classes

1. Make sure to uninstall any old versions or textbook versions

before installing this version.

2. The install files are located here =>

\\usmasvdfcase1se\install\DTools601Ind\

3. Right-click on DTS601-cust-Setup.exe then select Run As

Administrator. During setup, accept all defaults.

Install Precision Tree Instructions

One decision with 3 alternatives

One uncertainty with 3 outcomes. For this problem, the outcomes depend on the year but the probabilities do not. In general, the probabilities could depend on the alternative.

The Market Size branch data are the probabilities and the outcomes.

The data at the end of the tree (leaves) are the path probability and the path outcome. For large trees, these will NOT be the same as the last node. NEVER edit the blue calculations.

69 is the expected value of product launch in 2016. The other two EVs are 0.5 and 62.7.

True means that the best expected value is 2016. False is for the alternatives not selected.

Precision tree has a setting to change from max to min if we were using expected cost.

The x axis is the possible outcome and the y axis is the probability of each outcome.

For each alternative, the sum of the probabilities of each of the three possible outcomes in 1.

This chart is easier to understand than the cumulative but harder to get insights for complex problems.

Insights

2015 is dominated

2016 is generally better than 2017 but has a lower possible outcome.

The x axis is the possible outcome and the y axis is the cumulative probability of each outcome.

For each alternative, the sum of the probabilities of each of the three possible outcomes in 1.

This chart is harder to understand than the probability chart but easier to get insights for complex problems.

Insights

2015 is dominated (see dotted line overlay)

2016 is generally better than 2017 but has a lower possible outcome.

One Way Sensitivity

Two Way Sensitivity

Settings

Sensitivity Analysis

Analysis Type: One-Way Sensitivity (default)

Add variables and specify ranges.

Include Results. Specify all.

OK.

Precision Tree One Way Sensitivity

Cell E6. We have to name the cell. We want to vary the probability of high market. We have to also put equations in cells C6 and D6 to insure that the probabilities always add to one.

Cell E4. We only have to name the cell.

How many times did Precision Tree solve the decision tree to generate these graphs?

Perform sensitivity analysis for the probability of high market. Hint: Always name the cell with the sensitivity analysis parameters.

Sensitivity Analysis

Perform sensitivity analysis for the 2016 high market. Hint: Always name the cell with the sensitivity analysis parameters.

Sensitivity Analysis

Assess which of the two parameters is the most sensitive.

Sensitivity Analysis

Interpretation of charts (Warning: Y axis scales are different)

The chart of the left shows to sensitivity of the best EV to P(high market). It does NOT tell you which alternative is the best.

The chart of the right shows the sensitivity of all three alternatives to P(high market). This chart means that our 2016 is the best alternative for all values of P(high market).

Precision Tree solved 11 decision trees to get this data.

Interpretation of charts (Warning: Y axis scales are different)

The chart of the left shows to sensitivity of the best EV to 2016 High Market Outcome. It does NOT tell you which alternative is the best.

The chart of the right shows the sensitivity of all three alternatives 2016 High Market Outcome. This chart means that 2016 is the best alternative unless the 2016 High Market Outcome is less that 85.

Interpretation of charts (Warning: X axis scales are different).

These charts show the same message: the probability of high market is more sensitive than the high market (over the range we selected).

The Tornado Graph shows the range in EV $

The Spider Graph shows the change in input as a percent (X axis) and the impact on EV (Y axis).

Settings

Sensitivity Analysis

Start with the one way variables.

Analysis Type: Change to Two-Way Sensitivity.

Inputs: Select variables for X and Y.

Include Results. Strategy Graph and Strategy Region are the defaults.

OK.

Precision Tree Two Way Sensitivity

How many times did Precision Tree solve the decision tree to generate these graphs?

Assess which of the two parameters is the most sensitive.

Two Way Sensitivity Analysis

Interpretation of charts. These charts show two way sensitivity.

The Sensitivity Graph (2-Way) shows the range in EV $ as both parameters are varied but does not tell you the best alternative.

The Strategy Region (2-Way) Graph shows the best alternative for the two parameter values.

2016 is the best alternative in the blue range and 2017 is the best in the red range.

Precision Tree ran 99 runs to get this data (9 times 11)

**The Value of Information**

**Imperfect Information**

Calculate the Expected Value with Perfect Information and the Value of Perfect Information using Precision Tree influence diagram.

Class problem and HW 22

How do we decide how much to pay for information?

**Perfect Information**

Questions:

What is the purpose of a test/survey vs. marketing? Which would you spend more money on?

Can you give an example of perfect information? perfect control?

On the same decision, which is more valuable perfect information or perfect control?

21.1 Understand the role of technical and market information in engineering management and systems engineering decision making

21.2 Understand and be able to calculate the value of perfect information using Precision Tree

21.3 Understand and be able to calculate the value of control

Lesson 21 Objectives

Value of Imperfect Information

BRING LAPTOP TO

EVERY CLASS

Read: Ch. 12 pg. 502-512

Homework: HW#22

Next Time

EVPC > EVPI > EV

Mathematical relationships between EV, EVII, EVPI and EVPC:

EV of PI = 6

EV of PI = 0.6

EV of PI = 0

Principle. Information only has value if it could cause us to change our decision.

What causes the EV of PI to be positive?

What is the value of information?

Which one is clairvoyant?

Technical Information:

Will the system work as planned?

Market Information:

What will be the market for the systems?

Perfect Information

Draw and Influence Diagram and Decision Tree to represent this situation and calculate the Expected Value

Use Precision Tree to calculate the Expected Value

What is the value of Perfect Control?

Update the Influence Diagram to reflect CPT Thomas' Perfect Information. What is the value of this information?

What does the converted Decision Tree look like? Where do the probabilities come from?

Update the Influence Diagram to reflect Mr. Abdul's Imperfect Information. What are the associated probabilities of this information?

Using the fact that Mr. Abdul is in fact only 70% accurate, update your Influence Diagram. What is the Value of this information?

What does the converted Decision Tree look like? Where do the probabilities come from?

What is

Perfect Control?

What is

Perfect Information?

How do you modify the Precision Tree ID to solve for the Expected Value with Perfect Information on New Product Market?

Calculate the EV of Perfect Information (Market)

New Product

Market

Large

Small

Low

High

Cancel product line

Market current product

Develop new product

New Product

Performance

NPV

Product

Decision

Review: Excel Problem 1c. EV of PI

New Product

Market

Large

Small

Low

High

Cancel product line

Market current product

Develop new product

New Product

Performance

NPV

Product

Decision

22.1 Understand the role of technical and marketing information in engineering management and systems engineering decision making

22.2 Understand the mathematical relationship between

a. EV, Expected Value with EV, EV with Imperfect Information, EV with Perfect Information, and EV with Perfect Control

b. EV of Imperfect Information, EV of Perfect Information, and EV of Perfect Control

22.3 Understand and be able to calculate the value of perfect information and imperfect using Precision Tree

a. Develop a Precision Tree ID from an ID and solve for EV

b. Convert a Precision Tree ID to a symmetric decision tree (Model > Convert to Decision Tree)

c. Simplify the decision tree using Collapse Child Branches

Lesson 22 Objectives

Would you rather have perfect information about New Product Performance or Market?

How do you modify the Precision Tree ID to solve for the Expected Value with Perfect Information on New Product Performance?

Calculate the EV of PI (Performance)

New Product

Market

Large

Small

Low

High

Cancel product line

Market current product

Develop new product

New Product

Performance

Product

Decision

Review: Excel Problem 1b. EV of PI

How do you modify the Precision Tree ID to solve for the Value of Perfect Information on both New Product Performance and Market?

New Product

Market

Large

Small

Low

High

Cancel product line

Market current product

Develop new product

New Product

Performance

NPV

Product

Decision

Excel Problem 1d. EV of PI for

two variables.

Desk Problem 1a

How do you modify the ID in PT to calculate EV with PI?

What is the EV with PI?

What is the EV of PI?

Expected Value of Perfect Information on performance.

Yes

No

Performance

NPV

Launch

Product

23.1 Understand the role of technical and marketing information in engineering management and systems engineering decision making

23.2 Understand the mathematical relationship between

a. EV, EV with Imperfect Information, EV with Perfect Information, and EV with Perfect Control

b. EV of Imperfect Information, EV of Perfect Information, and EV of Perfect Control

23.3 Understand and be able to calculate the value of perfect information and imperfect using Precision Tree

a. Develop a Precision Tree ID from an ID and solve for EV

b. Convert a Precision Tree ID to a symmetric decision tree (Model > Convert to Decision Tree)

c. Simplify the decision tree using Collapse Child Branches

Lesson 23 Objectives

Yes

No

Launch

Product?

Performance

Test

How do you modify the ID in PT to calculate EV with II?

What is the EV with II?

What is the EV of II?

Expected Value of Imperfect Information on performance a performance test.

Yes

No

Performance

NPV

Test?

Calculate the EV for the product launch decision using influence diagrams in PT .

Yes

No

Performance

NPV

Launch

Product

0 ≤ EV of II ≤ EV of PI ≤ EV of C

EV ≤ EV with II ≤ EV with PI ≤ EV with C

0 ≤ 2.7 ≤ 6 ≤ 21

9 ≤ 11.7≤ 15 ≤ 30

Three decision analysis techniques are very useful in guiding the design of development tests and marketing activities

Value of Imperfect Information – Provides an upper bound on the most you would pay for a specific test or marketing analysis

Value of Perfect Information – Provides an upper bound on the most you would pay for a test or marketing analysis

Value of Perfect Control – Provides an upper bound on the most you would pay to achieve the best outcome (e.g., advertizing)

Calculate and compare.

Expected Value OF Perfect Information

Expected Value WITH Perfect Information

Expected Value OF Perfect Control

How much would you pay for these commercials?

Expected Value of Perfect Control = Max Value minus Expected Value

EVC = Max - EV

Solution

EVC = Max Value - EV

EVC = 105 - 69

EVC = 36

What

is the value

of this knowledge?

Value of Perfect Information – Provides an upper bound on the most you would pay for a test or marketing analysis

Value of Perfect Control – Provides an upper bound on the most you would pay to achieve the best outcome (e.g., advertizing)

Expected Value OF Perfect Control

Create a basic influence diagram for the MiO product launch.

Using the data in the Schott Class Data file and Precision tree, make in influence diagram.

Convert your influence diagram to a decision tree.

Based on expected value, what is the optimal launch year and expected NPV?

Solution

Steps in Precision Tree

1. Add Nodes and Outcomes

2. Add Arcs

3. Link to data tables

Desk Problem 1b

Solution

Determine what the probabilities associated with perfect information and fill in the table.

Update your influence diagram to reflect conducting a market survey that will reveal perfect information

Convert your influence diagram to a decision tree

Desk Problem 1c

Based on your updated influence diagram and decision tree, what in the expected value

WITH

perfect information?

What is the expected value

OF

perfect information?

What is the most you would recommend paying for market analysis or a market survey?

Solution:

EV with PI = 69.6

EV = 69

So, the EV of PI is the difference.

EV of PI = EV with PI - EV

= 69.6 - 69

= .6

This is the upper bound of what you should pay for market research.

Desk Problem 1d

Solution

Perform a sensitivity analysis on Market being Low, and a 2017 launch.

What does the curve mean?

Why do companies pay thousands of dollars for surveys and millions of dollars for advertising?

200 points = 20% of your grade

**Monte Carlo Simulation**

Issue List

From Influence Diagram to Decision Tree

3.11a

What is

Imperfect Information?

Value of Imperfect Information – Provides an upper bound on the most you would pay for a

specific

test or marketing analysis

Desk Problem 1a

Solution

Create a basic influence diagram for the updated MiO product launch.

Using the data in the Lsn 22 Data file and Precision tree, make in influence diagram.

Convert your influence diagram to a decision tree.

Based on expected value, what is the optimal launch year and expected NPV?

Steps in Precision Tree

1. Add Nodes and Outcomes

2. Add Arcs

3. Link to data tables

Desk Problem 1 b, c, and d

Update the Influence Diagram to reflect the following:

1. The expected value with perfect information of the perfect product performance

2. The expected value with perfect information of the perfect product market

3. The expected value with both

Desk Problem 1e - Imperfect Information

Update the Influence Diagram to reflect Performance Test predictions.

Expected Value WITH Imperfect information

Expected Value OF Imperfect Information

WPR 2 Part 3 Example

You are a platoon leader on an important and dangerous mission in the Ghazni provenance of Afghanistan. You have just landed by helicopter 2 kilometers from your objective. Your platoon has two ways to reach its objective: (1) traveling off road through a village with an expected time of 30 minutes; (2) traveling on an unimproved road with an expected time of 15 minutes if no IEDs are present, but 60 minutes if there is an IED present because you will have to clear the IED. There is only a 15% chance that the road has an IED. CPT Thomas, a Special Forces Team Leader, can give you perfect information on whether there is an IED on the road. Another option is to consult Mr. Abdul, a pro American tribal elder, who lives nearby and can assess the likelihood of an IED being in the road. Mr. Abdul is not a perfect predictor. If there is an IED, the conditional probability is 0.75 that he will say there is an IED in the road. If there is no IED, the probability is 0.85 that he will say "no IED." You want to minimize the expected time to reach the objective.

**SODA**

**MODA**

Risk Solver Platform

@Risk

Lesson 26 Objectives

26.1 Introduce simulation and the DSE simulation courses

26.2 Understand the theory of Monte Carlo simulation

26.3 Use Risk Solver Platform to perform a Monte Carlo Simulation on an decision model

26.4 Be able to modify a Monte Carlo simulation model to create a “decision tree” like capability to model the value of information

Next Time

Homework 27

Add uncertainty about product features in 2016

**When/why a Simulation Model vs Analytical Model??**

When inputs and processes are uncertain

When mathematical complexity makes it hard to provide “close form” results.

A simulation develops a model to numerically evaluate a system performance over time.

By estimating performance characteristics of the system, the best alternative from a set of alternatives under consideration can be identified.

1. Which are the sources of uncertainty?

2. What do we usually care about?

Figure 2.6, Parnell, G. S., Driscoll, P. J., and Henderson D. L., Editors, Decision Making for Systems Engineering and Management, 2nd Edition, Wiley & Sons Inc., 2011

1. Which are the sources of uncertainty?

Usually the inputs but we can also have uncertainty in the controls and the mechanisms.

2. What do we usually care about?

Uncertainty in the outputs

We begin with a system model with inputs, processes, and outputs.

Due to uncertainty, we assign distributions for the uncertain inputs and, possibly, the uncertain process events.

Monte Carlo simulation* generates inputs from input distributions and process distributions, processes the inputs, and generates distributions of the outputs.

We study the distributions of the outputs obtained by simulating a large number of runs of the model to determine the best system design.

Five steps:

Step 1: Create a parametric model,

y = f(x1, x2, ..., xq).

Step 2: Generate a set of random inputs,

xi1, xi2, ..., xiq.

Step 3: Evaluate the model and store the results as yi.

Step 4: Repeat steps 2 and 3 for i = 1 to n simulations

Step 5: Analyze the results using frequency, cumulative,

summary statistics, confidence intervals, etc.

* Developed by Manhattan Project physicists working on nuclear bomb design.

Revisit initial SODA problem -

Product Launch

Desk Problem 1

Use Excel Risk Solver Platform for Simulation

Name E10, E13, E16, and E19 then do Distributions > Common > Triangular (Low, Base, High)

Delete the index function in the cells

Use cell referencing

Name B29 NPV and designate as output (Results > Output > In Cell)

Look at Model > Simulation on right hand box

Verify in Options > All Options sampling method is Monte Carlo and 1000 trials

Simulate

Add marker for mean, name marker (Statistics > Markers > Type > Mean)

Do F9 a couple times

Copy and paste distribution into Word or Powerpoint

Show options on drop down menu (statistics, chart options, etc)

Paste the Sensitivity into Word or Powerpoint

Deterministic model was 2016, price 35, 64,588

Interpret the two charts

Interpretation

The Mean is approx 60.6 (very close to the 64.6 with deterministic). However, the range can be 30 to 95!

Market size is the major uncertainty.

Blue means increase in variable increases the NPV.

Red means increase in variable decreases the NPV.

Desk Problem 2

The updated model selects the price that maximizes Revenue – Production Cost (a surrogate for NPV).

In this model, the objective function is relatively flat around the optimal price.

In a deterministic model, 35 is still the optimal price.

What will happen when we perform Monte Carlo Simulation?

How can we modify our model to make it more realistic?

Rerun MC Simulation

EM481 Systems Simulation

Lesson 27 Objectives

27.1 Review the use Risk Solver Platform to perform a Monte Carlo Simulation on an decision model

27.2 Be able to perform simulation optimization to optimize the decision variables under uncertainty

Next Time

Homework 28

Use the Product NPV Model v5 130322 MC Lesson 27.

Compare the results of 2015, 2016, and 2017 by looking at the probability density functions and the price decision distribution.

What product launch year to you recommend and why?

Desk Problem 2 Con't

Use Excel Risk Solver Platform for Simulation

Name E5, E6, and E7 then do Distributions > Common > Triangular (Low, Base, High)

Delete the index function in the cells

Use cell referencing

Name B29 NPV and designate as output (Results > Output > In Cell)

Look at Model > Simulation on right hand box

Verify in Options > All Options sampling method is Monte Carlo and 1000 trials

Simulate

Add marker for mean, name marker (Statistics > Markers > Type > Mean)

Do F9 a couple times

Copy and paste distribution into Word or Powerpoint

Show options on drop down menu (statistics, chart options, etc)

Paste the Sensitivity into Word or Powerpoint

Results

Lesson 28 Objectives

28.1 Understand the sources of uncertainty in a MODA model

28.2 Understand how we can use multiple objective decision analysis to model the impact of independent and dependent uncertainties

28.3 Be able to compare uncertain alternative and select the best decision that balances value and risk

28.4 Be able to interpret a probabilistic tornado diagram

Lesson 29, Risk Attitude

C & R (Ch. 13 pg. 527-539)

Problems 13.1, 13.3, 13.4, 13.6

Lesson 31

Submit draft single and multiple objective deterministic decision models (SODA and MODA) in Excel. One file, labeled yourname.xlxs by start of class.

Sources of

Uncertainty??

Additive Value Model

**Desk Problem**

Use the same Monte Carlo simulation approach as SODA

Put uncertain distribution on the independent variables

Scores if independent variables

Probability of detection since all UAVs use different sensors

Non-score independent variable if 1 or more scores depend on an independent variable

Suppose weight, range, and endurance dependent on material strength and Micro Hawk and Micro Eagle use the same new composite material

We are going to use @Risk for the Monte Carlo Simulation

http://www.palisade.com/

Color Cells > Color @Risk Function Cells

Define distributions for independent uncertainties cells F15, F16, J15, J16, and L87

Add outputs and name for cells R32, R33, R34, R35, R37 (Alternative Values)

Application Settings > Reports > Place Reports In > Active Workbook > OK

Iterations > 1,000

Start Simulation

Browse Results > Add Overlay, select cells R32, R33, R34, R35, R37

Select Type of Distribution > Cumulative Ascending

Name the chart title > Comparison of alternatives

What can we conclude about the alternatives?

Explain VFT versus risk management

Put cursor on output cell, e.g, R33

Browse Results > Select tornado graph symbol > Tornado – Change in Output Mean

Interpret the results

How can you manage the risk of the material strength?

What does the tornado diagram look like for the Raven? Why?

Interpret the results. The material strength and P[D] are the major uncertainties. All weather is a minor uncertainty for both.

How can you manage the risk of the material strength? Use proven materials. If you have to do a new development, add a development test to assess strength early in the program.

What does the tornado diagram look like for the Raven? Why? There is no tornado diagram since there are no uncertainties.

Monte Carlo simulation is the same concept for SODA and MODA models

MODA models need to include the probabilistic dependence so distributions are only placed on independent variables

New systems usually have higher EV but more risk than existing systems

Systems engineers and engineering managers must focus on increasing value and managing risk

Summary

SODA vs MODA

SODA Model

**Converting MODA Models to SODA Models**

MODA Model

Converting Scores to $$

PhD or No PhD??

Functional

Hierarchy

Results

Decision

Trees

**Apply to your project**

Graded Lab

**New Timeline!**

**DROP**

Lesson 29 Objectives

29.1 Review the development of a MODA deterministic model

29.2 Understand how to convert MODA scores to dollars using your value preferences

29.3 Be able to develop a SODA deterministic model from the MODA deterministic model scores

29.4 Be able to model uncertainty for both models

33.1 Understand the three basic risk attitudes: risk seeking, risk neutral, and risk averse

33.2 Given an exponential utility function and an assumed R, determine the EV, EU, and CE.

33.3 Use the certain equivalent (CE) for a given situation to calculate R for the exponential utility function

33.4 Be able to calculate the risk premium for an exponential utility function.

34.1 Understand the assumptions required to use the exponential utility function

34.2 Be able to assess the exponential utility function risk tolerance parameter.

34.3 Use an exponential utility function to evaluate alternatives

34.4 Use sensitivity analysis to assess the decision sensitivity to the assessed risk tolerance parameter.

Lesson Objectives

Alternative 2:

a) win $4000 with probability 0.3;

b) win $000 with probability 0.3, or

c) win $500 with probability 0.4,

Solution

**"Desk" Problem**

Risk Tolerance - R

Certain Equivalence

Short answer/Multiple Choice

Topics:

Decision Framing

MODA

Math and insights

Influence Diagrams, Decision Trees, and Risk Profiles

Decision tree calculations with missing info that you fill in

Value of Information

Includes Bayes Law calculations

Risk Attitudes

Exponential utility function calculations

**TEE Topics**

Sat at 1100, 130 cadets

Thurs at 0730, 12 cadets

Bring

Up to 3 sheets of paper with review notes/equations

(WPR1, WPR2, and TEE)

**TEE Admin**

**TEE**

Gerolamo Cardano

Book on Games

of Chance

+ Soft Skills!