### Present Remotely

Send the link below via email or IM

• Invited audience members will follow you as you navigate and present
• People invited to a presentation do not need a Prezi account
• This link expires 10 minutes after you close the presentation

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

# SE385 AY13-2

Decision Analysis, Systems Engineering Course, United States Military Academy, West Point
by

## Libby Schott

on 3 September 2014

Report abuse

#### Transcript of SE385 AY13-2

Problem: Suppose you face the following gamble:
a) win \$2000 with probability 0.4;
b) win \$1000 with probability 0.4, or
c) win \$500 with probability 0.2, and
your utility can be modeled as an exponential function. The DM is indifferent to the following choice (assess Risk Tolerance R):

1. What is R?
2. What is the EV for this gamble?
3. What is the EU for this gamble?
4. What is the CE for this gamble?
5. What is the RP for this gamble?
7. For the following two alternatives, assume exponential utility function with R =900 and calculate:
•EV
•EU
•CU
•U(CE)
8. Which alternative do you recommend?
9. Perform a sensitivity analysis for CE for R = 500 to 2,000. Use a Data Table similar to how we did sensitivity analysis on swing weight.
10. Is the decision sensitive around R = 900?
NPV
Suppose that a test for a particular disease has a very high success rate:
Let D be the event that the patient has the disease, and T be the event that the test returns a positive result.
only 0.1% of the population have that disease:  P[D] = 0.001
if a tested patient has the disease, the test accurately reports a 'positive', 99% of the time:  P[D| T] = 0.99
if a tested patient does not have the disease, the test accurately reports 'negative', 95% of the time:  P[T|DC] = 0.05

We now have all the information required to use Bayes' theorem to calculate the probability of the disease given the test. Using Bayes' theorem the probability of a true positive

Probability that a positive result is a false positive is about (1 −- 0.019) = 0.981. Despite the apparent high accuracy of the test, the incidence of the disease is so low (one in a thousand) that the vast majority of patients who test positive (98 in a hundred) do not have the disease.
Use a control panel
(Not Req'd for MODA)
Parametrize everything
Type once, then cell reference
Use uniform indexing
Plan for Monte Carlo analysis
Manage model configuration
Document Excel file
MODA
Use 2 significant digits (0.52 of 1.0, 5.2 of 10, 52 of 100).
Build in checks
Weights sum to 1
Always score Ideal to check value functions
DA Process
Framing
Selecting the Appropriate
Decision Process
Lottery Instant Winners:
Best Case: win \$20
Worst Case: lose \$5

There exists a probabily of winning, p(w), that makes winning a guareenteed amount of money equivalent to playing the game.
Basics: Two homework assignments worth 50 points each. Each assignment will have 5 multiple choice questions. You will answer with the probability that answers A-D are correct; these probabilities must sum to 1.
Why we are doing this:
The best strategy:
Improve your state of information by studying…
Strictly Proper Scoring: Assign your true beliefs!
How Quadratic Strictly Proper Scoring Works:
Qi(r) = 2ri - −r · r ∈ between [0 ,1]
Introductions

Course
Instructional Memo
Class Preparation Points
My Expectations

Introduction to Decision Analysis
Decision Analysis
Decision
Good Decision

Deterministic Models
Qualitative Decision Analysis
Introductions
Homework is due the class at start of class

Each day two cadets will be randomly selected to hand in their homework.

Excel Rand function in class spreadsheet will randomly (uniform distribution) choose who turns it in homework for grading.

Selection done with replacement

Students will run the random number generator.

The first cadet that hands in their homework last class will “F9” the next two

Instructor will grade and return the homework
To Decisions and Decision Analysis
Uncertainty
Don't be too quick to judge. . .
Modeling Risk
Utility Function
Risk Attitudes
Project Time!
Decision Analysis
SE385
Modeling Uncertainty
Subjective Probability Assessment
The Course. . .
The Other Instructors
The Students. . .
The Instructor. . .
Dr. Greg Parnell
MAJ Pierre Han
Mr. Dave Chennault
Education
B.S. from USMA, 1991, Mathematics Major with a Systems Engineering track
M.S. from GA Tech, 2001, Industrial Engineering
M.S. from New Mexico State University, 2007, Mathematics
Ph.D. from New Mexico State University, 2009, Industrial Engineering

Teaching Experience
USMA, Math, 2001-2004; 2009-2011
Core Courses: MA101/MA103/MA104/MA206
and MA491
Program Director for MA100 /101
Math Rocks!
USMA, DSE, 2011 -
Core Engineering Sequence Program Director
SE450, SE350, SE402/403
Husband: Russ
My Army Experiences:
Branched Quartermaster. . . . .Logistics now
7th Infantry Division, Fort Ord, CA 1992-1993
DISCOM Asst S1, Platoon Leader, Accountable Officer
JTF-LA

9th Infantry Regiment, Fort Lewis, WA 1994-1995
Bn S4, Asst SPO, Asst Reg S4
Operation Sea Signal

1st Infantry Division (Mech) Germany 1996-1999
Rear D Cdr, A/SPO, Company Commander
Bosnia, Macedonia

Functional Area 49/ Operaations Research/Systems Ananlyst (ORSA)
TRAC – White Sands Missile Range. WSMR, NM 2004-2006
Wargaming - Scenarios and Data Processing and Analysis
USF-I, Camp Victory and the United States Embassy Baghdad 2010-2011
J5 - Strategic Assessments
Operation New Dawn
LTC Libby Schott
Twins: Robert and Karl
Daughter: Rhea
16 Nov 12
Lesinski, x5897
Determine if you have space for AIAD (TAC)
Review the AIAD Opportunities available (Jan)
Participate in the Systems AIAD Straw Poll (Jan)
Enter Preferences via AMS (Feb)
Complete special requirements (clearance, passport, etc.)
Coordinate Logistics for AIAD (lodging, etc.)
Complete AAR and Travel Voucher (Aug)
Process
Systems AIADs provide the opportunity for our majors to gain valuable, real-world experience by working with military and civilian systems engineers, operations research analysts, and scientists.
Mr. Gene Lesinski, DSE AIAD Coordinator, Mahan 305
eugene.lesinski@usma.edu 938-5897
Instructors or DACs
How can I find out more?
International Opportunities
SE Majors
Year Group
Capstones
Preferences
8,9 Jan: Instructors Present AIAD Overview
14-18 Jan: Systems AIAD Straw Poll
22-29 Jan: Cadets Enter Preferences via AMS
22 Feb: USCC Summer Training Schedule
8 Mar: USCC Deconfliction Complete
10-17 Mar: Spring Break
General Rules for Matching
Key Dates
Preferred Name
Company
Branch preference
What you did over the break
Sports/clubs/hobbies
What you hope to get out of the class
Lesson 1 Objectives

Do: HW #2
NRC
Committee
Member
2012
NRC
Committee
Chair
2008
Editor/Author
Dr. Parnell has systems experience in space systems, managing aircraft and missile research and development (R&D) programs, and leading missile systems engineering. He teaches systems engineering, decision and risk analysis, operations research, and engineering management courses. He is a member of International Committee on Systems Engineering (INCOSE), American Society for Engineering Education, Institute for Operations Research and the Management Sciences (INFORMS), and Military Operations Research Society (MORS). He is former President of MORS and the Decision Analysis Society of INFORMS and a fellow of MORS, INFORMS, INCOSE, Society for Decision Professionals, and Lean Systems Society. He serves on several advisory boards and national academy committees. He is a retired Air Force Colonel.

Degrees
BS, Aerospace Engineering (State University of New York at Buffalo)
ME, Industrial & Systems Engineering (University of Florida)
MS, Systems Management (University of Southern California)
PhD, Engineering-Economic Systems (Stanford University)
Project Manager
Minuteman III
Reentry Vehicle
Chief, Missile Systems Engineering
Peacekeeper ICBM
Base Realignment
and Closure 2005
Member,
Technology and
Compliance Panels
Dr. Gregory S. Parnell
Professor of Systems Engineering
Department of Systems Engineering
Foundations of Decision Analysis

Do: HW # 3

Selecting the Appropriate Decision Process
Next Time
DAH
C & R
PDH
http://usmasvdfcase6se:19387/Courses/AY132/SE385/default.aspx
Instructor,
D/SE, USMA
Student
Company
Commander
2-13th AVN (UAS)
USMA ‘02
Rifle PL, Mortar PL,
CO XO, BN S3 Air
2-87 Infantry Battalion
MAJ Han is a 2002 graduate of the United States Military academy, and one of the newest members of the USMA DSE faculty. He most recently came to the department from George Mason University, where he studied Operations Research with a concentration in discrete event simulation, optimization, and military applications. His previous military positions include company commander, Battalion planner, company executive officer, and rifle/mortar platoon leader. He is a member of the Military Operations Research Society and the Omega Rho Honor society.

He is married to the former Megan Weaver, and has two sons, Scott (4) and Brandon (3).

Degrees:
BS, Engineering Management (USMA, ‘02)
MS, Operations Research (George Mason University, ‘12)
MAJ Pierre Han
Instructor, Systems Engineering
Department of Systems Engineering
B Hour
Alex
Andrew
Jon
Reese
Phillip
Mark
Eric
Tyler
Bobby
Ben
Asika
Matt
Jerome
Jonathan
Kevin
Chris
Henry
Cody
Class Preparation Points
r = vector of probabilities
Could be worse!
* See textbooks for other definitions.
Decision: An irrevocable allocation of resources.

Decision Analysis*: Decision analysis is a philosophy and a social-technical process to create value for decision makers and stakeholders facing difficult decisions involving multiple stakeholders, multiple (possibly conflicting) objectives, complex alternatives, important uncertainties, and significant consequences. Decision analysis is founded on an axiomatic decision theory and uses insights from the study of decision making.

Purpose: Provide insight to decision-makers faced with hard problems.
Essential Definitions
“A good decision is an action we take that is logically consistent with the alternatives we perceive, the information we have, and the preferences we feel.”
- Ronald Howard
Stanford University
Preferences: What you want.
Values, Time Preference, Risk Preference.
Information: What you know.
Any relevant models, relationships, or probability assignments that may be important characterizing the connection between decisions and outcomes.
Alternatives: What you can do.
There must be more than one.
Logic
(the seat)
Information
Preferences
Alternatives
Decision Maker
(committed to action)
Frame
So what is a good decision?
© 2006 J. Eric Bickel, TAMU
Homework Handin
Understand the role of axioms in the development of theory

Understand the five decision analysis rules (axioms) and how they are used in decision analysis

Understand the scope of decision analysis

Understand the benefits of Value-Focused Thinking

Understand the taxonomy of value models
Lesson 2 Objectives
All of probability theory is based on these three simple axioms!
Three major axioms of probability
Sound theory is based on credible axioms.
Howard, R. A. (2007). Chapter 3. The Foundations of Decision Analysis Revisited. In W. Edwards, R. F. Miles , & D. von Winterfeldt (Eds.), Advances in Decision Analysis: From Foundations to Applications (pp. 32-56). Cambridge University Press.
1. The Probability Rule requires that you can fully describe any deal in terms of possibilities and probabilities. A possibility is a clear description of an event that may or may not occur. A set of possibilities (also called an outcome space) is complete if they are mutually exclusive (only one may occur) and collectively exhaustive (one of the set must occur). A probability is a number between 0 and 1 that expresses your degree of belief that a possibility will occur.
Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013
Use of the five rules.
2. The Order Rule requires that you can rank any set of prospects in order of preference from best to worst. Indifference (i.e., equal preference) between two prospects is allowed. This rule implies transitivity of preference. If you prefer A to B and you prefer B to C, then you must prefer A to C (i.e., C cannot be both below A and above A in the preference ranking).
3. The Equivalence Rule requires that you can always create an uncertain deal involving two prospects such that you would be indifferent between receiving that deal and receiving a third prospect that is intermediate in your preference ranking between the two prospects in the deal. So, if you prefer A to B and prefer B to C, then the rule requires that there is a probability p such that you are indifferent between 1) a deal that gives you A with probability p and C with probability (1-p) versus 2) receiving B for sure. The probability p is called a preference probability because it is defined by your preferences rather than by your beliefs about the likelihood of any real events. Prospect B is said to be the certain equivalent of the deal involving A and C with the preference probability that you specify.
4. The Substitution Rule requires that your preference for a prospect will not change if an uncertain deal contained in the prospect is replaced by its certain equivalent, or vice versa.
5. The Choice Rule requires that if given the choice between two deals involving the same two prospects but with different probabilities, you must prefer the deal having the higher probability of receiving the more preferred prospect. Suppose that you prefer A to B and that you are offered two different deals. In Deal 1, you would receive either A or B with probabilities 40% and 60%, respectively. In Deal 2, you would receive either A or B with probabilities 25% and 75%, respectively. The rule requires that you must prefer Deal 1 because it offers the higher probability of the more preferred prospect A.
Probability Rule
Order Rule
Equivalence Rule
Substitution Rule
Choice Rule
Probability Theory
1657
1931
Subjective Probability
(Bayesian View)
On Reasoning in Games of Chance
Christiaan Huygens
Based on the work of Blaise Pascal and Pierre de Fermat
Example
One roll of a die:
Outcome Space: {1,2,3,4,5,6}
Mutually Exclusive - can't roll a 1 and a 2
Exhaustive - only 6 possible outcomes
P(1) = 1/6 . . .
0 <= P(1) <= 1
Example
Example
Game A:

45% - Bag o'Money
55% - Empty Pockeys
Example
Interchangable!

No difference if you play the game, or take the \$5
Game B:

35% - Bag o'Money
65% - Empty Pockeys
And the Winner is. . .
Class Example
Company Party!
Decision:
Outdoors or indoors?
LTC Schott's Solution
Given: P(Sun) = .4
P(Rain) = .6
Equivalence
Apply the five rules.
Five Rules: Theoretical Foundations of DA
Scope of DA
Modified from Keeney, Ralph L., Value-Focused Thinking: A Path To Creative Decisionmaking, Harvard University Press, Cambridge, MA, 1992, pp. 3-28.
Benefits of VFT
Taxonomy of DA
1738
1713
1763
1812
1933
The Art of Conjecture
Exposition of a New Theory on the Measurement of Risk
Essay towards solving a Problem in the Doctrine of Chance
Essay towards solving a Problem in the Doctrine of Chance
Theorie Analytique des Probilities
Truth and Probability
Jacob Bernoulli
Daniel Bernoulli
Rev Thomas Bayes
Pierre-Simon Laplace
Frank Ramsey
Andrey Kolmogorov
Mathematical Decision Theory
1944
Theory of Games and Economic Behavior
Foundations of Statistics
1954
Behavioral Decision Analysis
1954
"The Theory of Decision Making"
1974
"Behavioral Decision Theory"
1961
John von Neumann and Oskar Morgenstern
Leonard J. Savage
Ward Edwards
Amos Tversky and Daniel Kahneman
Ralph L. Kenney and Howard Raiffa
Ron Howard
1981
2007
1968
SODA
MODA
5 Rules

Do: HW #4

Framing the Decision
(Problem Definition)
Next Time
Compare and contrast the three decision analysis processes described in the three textbooks
Clemen &Reilly’s Decision Process
Systems Decision Process
DA Handbook’s Decision Analysis Process

Task 1. Which of the following types of decision processes are each of the three decision processes?
Analytical
Dialog Decision
Decision Conference

Task 2. Identify the major similarities and differences between (Hint: Use the 6 requirements for a good decision)
Clemen &Reilly’s Decision Process
Systems Decision Process
DA Handbook’s Decision Analysis Process
Board Challenge
Dialog Decision
Four types of decision processes
Some common decision processes.
“Decisions are easy, it’s only the rationale that is difficult.” -Anonymous

“Nothing is more difficult, and therefore more precious,
than to be able to decide.” -Napoleon
C&R pgs. 3-4
Complexity
Possible courses of action / alternatives?
Possible outcomes?
Likelihood of outcomes?
Eventual consequences?
Uncertainty
It is everywhere…
Which ones are important?
Multiple Values & Objectives (conflicting)
Maximize quality vs. minimize cost
Which ones are important?
Different Perspectives
Who are the decision-makers / stakeholders?
Why are some decisions hard?
Systems decisions can be very complex since they involve many stakeholders, technical risks, large investments, and long time horizons.
Ralph L. Keeney, “Making Better Decision Makers,” Decision Analysis, Volume 1, Number 4, pp.193-204, December 2004
We need decision analysis for our most challenging decisions.
3.1 Understand when we need decision analysis

3.2 Understand the 6 requirements of a good decision

3.3 Understand the four types of decision processes
Analytical
Dialog Decision
Decision Conference

3.3 Compare and contrast three decision analysis processes
Clemen &Reilly’s Decision Process
Systems Decision Process
DA Handbook’s Decision Analysis Process
Objectives
Six requirements are necessary and sufficient to ensure a quality decision process.
Spetzler, C. & Keelin, T. (1992). Decision Quality: Opportunity for Leadership in Total Quality Management. Menlo Park: Strategic Decision Group
The Decision Analysis Creed:
“We work diligently to help you make good decisions and we pray you get good outcomes.”
Good decision quality increases the likelihood of good outcome quality.

Do: C &R 2.10 and 2.11
Crafting Decision Objectives
Next Time
Decision frame

Vision statement

Issue raising

Stakeholder Issue Identification Matrix

Decision hierarchy
Key Terms
Work in groups of 2 – One decision maker and one decision analysis

Frame the decision maker’s decision to go to graduate school at some future time.

a. Vision statement
b. Issue list
c. Stakeholder issue matrix
c. Decision hierarchy
Class Challenge 2
Work in groups of 3 – One decision maker and two decision analysts

Frame the decision maker’s investment decisions for the next seven - ten years.

a. Vision statement
b. Issue list
c. Stakeholder issue matrix
d. Decision hierarchy
Class Challenge 1
“A pessimist is one who makes difficulties of his opportunities and an optimist is one who makes opportunities of his difficulties.”
Harry S. Truman

“Opportunities multiply as they are seized.”
Sun Tzu
4.1 Understand the importance of framing the decision

4.2 Understand and be able to use the decision framing tools for a personal or professional decision
a. Vision statement
b. Issue list
c. Stakeholder issue matrix
c. Decision hierarchy

4.3 Compare and contrast Problem Definition in the Systems Decision Process with Framing the Decision
Objectives
Issue Identification Matrix
Vision Statement
Decision Hierarchy
Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013
Decision framing tools
Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013

This is a personal story that our decision analysis colleague Carl Spetzler likes to tell to illustrate the importance of agreeing on the frame for a decision. One morning his wife said to him, “I think it is time to repaint and carpet our house.” Carl looked around and saw that she was right. His response was, “Should we consider doing some remodeling in the kitchen and playroom first? After all, we are going to become empty nesters in six months.” Pretty soon they were talking about hiring an architect to redo their bedroom area. And as the ideas kept growing—along with the dollar signs—they concluded that perhaps they should consider selling the house and buying another that may already have these amenities. After a while, Carl’s wife asked, “How long do you intend to work before retirement?” Soon the question had grown to “So, what are we going to do with the rest of our lives?”

Painting and carpeting would be a two-month and \$2,000 project, whereas planning the rest of their lives was a huge question that might take a couple of years to resolve in multiple rounds with large financial and quality-of-life implications. What decision should they focus on?
Importance of a Decision Frame
The decision frame is the critical first step in decision analysis. The decision frame helps us define the decision.
May be some big challenges
May be a challenge
The decision analysis must determine the decision frame.
Framing the decision
Team’s of two
One decision maker
Two decision analyst

Use Keeney’s 10 questions to develop your professional objectives for the next 7 years.

Structure your professional objectives in a value hierarchy (include financial objectives)
Board Challenge Problem
First draft was developed in one day with about 20 stakeholders at an intelligence agency.
Stakeholders all felt there decision issues would be included.
Data center location functional value hierarchy
Keeney, R. (1994). Creativity in decision making with value-focused thinking. Sloan Management Review, 33–34.
Strategic objectives: What are your ultimate or long range objectives?
A wish list: What do you want?
Alternatives: What is a perfect alternative, a terrible alternative, a reasonable alternative?
Problems and shortcomings: What is wrong or right with your organization or enterprise?
Consequences: What has occurred that was good or bad? What might occur that you care about?
Goals, Constraints, and Guidelines: What are your goals or aspirations? What limitations are placed upon you?
Structuring objectives: Why is that objective important, how can you achieve it?
Quantifying objectives: How do you measure achievement of this objective?
Non financial objectives.
Keeney’s 10 questions
Decision objectives should be based on shareholder and stakeholder value.

Should private companies focus on shareholder or stakeholder value?
Could BP’s 2010 Deepwater Horizon explosion (triggering the worst offshore oil spill in the country's history) be considered an example of lack of consideration of stakeholder value? Whose values may not have been adequately considered?

How does a company provide value to shareholders and stakeholders?

How does a public organization provide value to its stakeholders?
Shareholder and Stakeholder Value
Management by objective works – if you know the objectives. Ninety percent of the time you don’t. Peter Drucker

“Our age is characterized by the perfection of means and the confusion of goals"  A. Einstein
5.1 Understand the difference between shareholder value and stakeholder value.

5.2 Understand financial statements and objectives

5.3 Understand and be able to use Keeney’s 10 questions to generative decision objectives

5.4 Be able to use a value hierarchy to structure objectives
Objectives
Read : DAH Ch 8, C&R (Ch 7, 217-239)

Do : DAH HW # 6
Designing Creative Alternatives
Next Time
Help is logically organize our objectives and identify missing objectives
Improve communications with decision maker
Why do we use a value hierarchy to structure objectives?
Crafting decision objectives
Rt the net cash flow i.e. cash inflow – cash out
Balance Sheet Statement. A balance sheet is developed using standard accounting procedures to report an approximation of the value of a firm (called the net book value) at a point in time.

Income Statement. The income statement describes the changes to the net book value through time.

Cash Flow Statement. A cash flow statement describes the changes in the net cash position through time.

Net Present Value (NPV). To boil a cash flow time pattern down to a one-dimensional value measure, companies usually discount the free cash flow to a “present value” using a discount rate.
Financial statements and objectives
Read: DAH Ch 9 pp. 169-183

Do: HW# 7 - Review your groups Strategy Generation Table and identify the your blocks to creative thinking.

Due: PHW #1 1600 28 Jan 2013
Deterministic Modeling and Sensitivity Analysis -SODA (NPV)
Next Time
A useful view of creativity.

6.1 Understand the importance of creative thinking

6.2 Be able to create alternatives using creative thinking

6.3 Be able to use a strategy generation table to identify creative, doable alternatives

6.4 Understand blocks to creative thinking and be able to identify your blocks to creativity
Objectives
Net Present Value

Objective Identification

Objectives Structuring

Value (Objectives) Hierarchy
Working in groups of 3, use the Strategy Generation Process to develop three creative, distinct West Point curriculum alternatives.
Character
Decision Making
U.S debt
Interconnected international economy
Cyberwarefare
International security challenges
Cultures and language
WMDs and Terrorism
Role of the Core
Accreditation
Issues?
Hypothetical Dean’s Vision Statement

Redesign the West Point curriculum to better meat the academic goal: Graduates anticipate and respond effectively to the uncertainties of a changing technological, social, political, and economic world.
Classroom Challenge
Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013
Small number of wellcrafted
alternatives for evaluation
Highly organized, analytic thinking
Individual and group creativity
Identify key decisions
Provide decision options
Alternatives that span the decision space
Strategy names that convey the key alternative features
Strategy Generation Table
Many creative ideas w/o judgment
Vision (Problem) Statement

Issues

Value Hierarchy
Convergent Phase
Divergent Phase
Strategy Generation Process: Generating good alternatives is a two phase process.
Questions

What is creativity? Are you a creative person?

How do you develop “creative, doable alternatives?
Designing creative alternatives
Intellectual and Expressive Blocks
• Choosing the “correct” problem solving language—we need the ability to go from one problem solving languages to another to include analytical/ mathematical, visual/special, analogy, etc.
• Flexibility versus fluency—we need both, flexibility to generate many alternatives, fluency to generate alternatives that are different in nature.
• Incorrect information—lack of, or bad, information expands rapidly and makes problem solving difficult;
inhibits “selling” good alternatives to others.
Cultural and Environmental Blocks
• Taboos—we eliminate alternatives too quickly in our thought process because they seem culturally incorrect.
• Lack of humor in problem solving—humor can be inspiring.
• Reason and intuition: both sides are essential for creativity.
Left-handed and right-handed - both sides are essential for creativity—left side for order, reason, logic and mathematics, right side for imagination, artistry, and intuition.
• Tradition and change—it is hard to overcome the inertia of tradition
• Supportive versus non-supportive environments—physical, economic, and organizational support are needed to bring ideas into action.
• Autocratic bosses—may make it difficult to push new ideas through.
Emotional Blocks
• Fear of taking a risk—we tend to be afraid of making a mistake or failing.
• No appetite for chaos—we struggle to deal with ambiguity and
uncertainty.
• Judging rather than generating ideas—we tend to judge and analyze too early in the decision-making process.
• Inability to incubate—failing to give ideas time to mature before eliminating
them prematurely.
• Lack of challenge or excessive zeal—lack of motivation will inhibit creativity as will excessive motivation to succeed quickly.
• Reality and fantasy—ignoring one of these critical resources to creativity in problem solving.
Creative Thinking Blocks –How did these effect you? (Homework)
Perceptual Blocks
• Stereotyping—seeing what you expect to see.
• Difficulty in isolating the problem—solving the wrong problem.
• Delimiting the problem too closely—imposing too many constraints upon the problem and its solutions.
• Inability to examine the problem from multiple perspectives.
• Saturation—inability to perceive and retain all the information around us.
• Failure to use all of our senses—we should not neglect any sensory inputs.
Example
1.

2.

3.
I prefer apples over bananas and bananas over oranges.
Therefore I must prefer apples over oranges.
This is based on outcomes - not preferences. Axioms are about preference.
Objectives and Values
Alternatives
Should you use a DA Process?
When
Selecting the Appropriate
Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013
Decision Analysis Process
Parnell, G. S., Driscoll, P. J., and Henderson D. L., Editors, Decision Making for Systems Engineering and Management, 2nd Ed, Wiley Series in Systems Engineering, Wiley & Sons Inc., 2011
Systems Decision Process
Clemen and Reilly Process
Modified From Clemen & Reilly, “Making Hard Decisions”
Decision Conference
Urgency
Importance
Difficulty
Spetzler, C., “Building decision competency,” Advances in Decision Analysis – From Foundations to Applications, Edwards, W., Miles, R., and von Winterfeldt, D., Cambridge Press, 2007
Phillips, L. D., “Chapter 19: Decision Conferencing”, Advances in Decision Analysis – From Foundations to Applications, Edwards, W., Miles, R., and von Winterfeldt, D., Cambridge Press, 2007
Good Decision?
What is a
Logic
(the seat)
Information
Preferences
Alternatives
Decision Maker
(committed to action)
Frame
© 2006 J. Eric Bickel, TAMU
Roughneck North American Oil Strategy
Geneptin Personalized Medicine
Data Call Center
Illustrative Examples
This decision looks easy!
Purpose
Perspective
Scope
1. Vision Statement
2. Issue-Raising
3. Categorization of the issues
4. Decision hierarchy
6. Initial Influence Diagram
7. Decision schedules and logistics
Framing Workshop
Elcom’s 32,000-square-foot facility located in Rockleigh, New Jersey
Last Industry Position: Vice President, Operations Elcom Technologies
My Background: Professional
Prior Employers: ADC Telecommunications – 11 years,
Roles: Engineer Manager, Senior Engineer
My Background: Professional
Elcom’s 32,000-square-foot facility located in Rockleigh, New Jersey
Last Industry Position: Vice President, Operations Elcom Technologies
My Background: Professional
Academic Positions: Visiting Professor, DeVry University. EET Program

Education: BS, Electrical Engineering, United States Military Academy, ‘85
MS, Electrical Engineering, University of MN, ’96

Military Branch/Functional Area: Signal/ 25C

Military Education: Signal Officer Basic ‘85. Signal Staff Officer ’86,
COMSEC ’86. Airborne ’83.

Previous Military Assignments: 1-12 INF Signal Officer.
124th Signal Company XO, Platoon Leader. 4ID Division Signal Staff

Professional Associations: IEEE, Association of Old Crows.
Registered Professional Engineer, Electronics and Computer Engineering
Born: Maxwell AFB, Montgomery, AL
Raised: Crosby, TX

Family: Wife, Wendy
Daughter, Calla (15)
Sons: Austin (21) Collin (19)

Home: Waldwick, NJ
C: (201) 280-9032
Me First…
Interests
Dr. Parnell's Solution
* Decision Analysis is based in the five axioms
Options:
Os
Or
Is
Ir
Ranked Options:
Os
Ir
Is
Or
http://www.kantola.com/Ron-Howard-PDPD-143-S.aspx
Examples
What are we doing?
Why are we doing this?
How do we know that we have succeeded?
Givens:

Planning horizon:
5 yrs
Russ retires at 22 YOS
Libby retires at 27 YOS
Focus on investment strategy for our retirement
Decisions:

What should our updated
budget/investment strategy look like?
Does Russ need a second job?
What is our real estate plan (should include what to do with the house(s) in Florida, invest in property in Montana?
Acquisition of "toys" (2nd car, boat, rv, jet skis?)
Exact allocation of investment dollars
Rolling over TSP, Updating life insurance policies
Purchase details of real estate and/or of "toys"
Vision Statement:

We will decide what is the best investment strategy for our future. We need to do this in support our goal of being financially independent in our retirement. We know we will have succeeded if we are both satisfied and confident in our plan.
What impact will the current financial markets have on our investment?
Where do we want to retire?
Should Russ get a job?
What should we do with our house in Florida?
What we our role be in caring for our parents?
What level of support do we give our kids while they are in college?
What impact does going to O'Neill HS have on Rhea's future?
What lifestyle do we need to fund in retirement?
Should we leave our money in our current TSP or convert it when we retire?
Do we need additional life insurance?
Issues
What activities do we want to enjoy with the kids?
What does the real estate market look like in the next 10 years? in Cape Coral? In Great Falls?
health insurance?
Where should the twins go to High school? Is that the same as where we want to retire to?
Will the GI Bill still be funded when the twins reach college?
Can we count on social security?
Impact of changing tax laws?
LTC Schott's Solution
Can we count on our military retirement?
What will be the cost of living in Great Falls, MT? In Cape Coral, FL?
MODA
SODA
Single Objective Decision Analysis
Multi-objective Decision Analysis
Focus groups
Surveys
Interviews
chevron
Natural
Constructed
Direct
Proxy
LTC Schott's Soln
Hybrid Solns
Read: DAH Ch 9 pp. 169-183

Do: HW# 8 Complete the design of your spreadsheet and calculate the NPV for 2016 with a price of 30.

Due: PHW #1 1600 28 Jan 2013

Deterministic Modeling and Sensitivity Analysis - SODA II
Next Time
Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013
Decision Analysis Excel Modeling Guidelines
Develop an influence diagram develop the model structure
7.1 Be able to develop an influence diagram given the decision frame.

7.2 Be able to convert an influence diagram to a deterministic value model in Excel to calculate Net Present Value

7.3 Understand and be able to apply decision analysis modeling guidelines.
Lesson 7 Class objectives
1st worksheet - Control panel
2nd worksheet - Calculations
Design a spreadsheet to model the new product decision
Future decisions
Given our new product
information, determine:
the launch date and
price that will maximize NPV
Decision Hierarchy
This decision
Done deals
New product implementation
design features
production plan
test plan
marketing plan
Corporate
mission & strategy

Corporate profit objectives

Product portfolio including the new product
A company has decided to develop and launch a new product. Develop an influence diagram for two decisions: the product launch year and the selling price of the product. There are several uncertainties. The product features depends on the launch year. The product’s market share is affected the new product features, and by its price. Sales depend on market share and market size. Revenue is calculated as unit sales times unit price. Unit costs and development expenses depend on product features. Expenses are equal to the sum of manufacturing cost (sales times unit cost), marketing expense, development expenses, and annual fixed expenses. Net present value is calculated based on cash flow (revenues – expenses) and discount rate.
Product Launch Decision Model
Appendix, B,, Influence Diagram, Decision Analysis Handbook, Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013
We can use an influence diagram to develop the model structure

Do: HW# 9: Complete the design of your spreadsheet and calculate the NPV for 2016 with a price of 30 and perform a deterministic sensitivity analysis (tornado diagram).

Deterministic Modeling – Multiple Objective Decision Analysis
Next Time
7.1 Be able to develop an influence diagram given the decision frame.

7.2 Be able to convert an influence diagram to a deterministic value model in Excel to calculate Net Present Value

7.3 Understand and be able to apply decision analysis modeling guidelines.

8.1 Be able to perform deterministic sensitivity analysis
Lesson 8 Class objectives
Install RSP

View
Model > Sensitivity > Parameters

Place cursor on NPV (b26)

Parameters > Identify > Show Current Parameters
RSP can be used to develop a tornado diagram

Do: HW# 10: Complete the design of your spreadsheet and calculate the MODA value of all the UAS Alternatives. Develop a constructed scale that defines all five levels for the weather capability value measure.

Deterministic Modeling – MODA II
Next Time
UAS Functional Value Hierarchy
SE301 Review Question
Is this the logical order of the functions?
UAS Functional Hierarchy

9.1 Be able to convert a functional value hierarchy to a multiple objective decision analysis model in Excel. Understand the role of
a. Value functions
b. Value measure scales
c. Scores
d. Single dimensional value (using ValuePL)
e. Swing weight matrix
f. Weighted value
g. Total Alternative value
h. Value component chart
Lesson 9 Class objectives

Do: HW# 10: Complete the design of your spreadsheet and calculate the MODA value of all the UAS alternatives. Develop a constructed scale that defines all five levels for the weather capability value measure.

Deterministic Modeling – MODA II
Next Time
Lesson 10 Class Objectives
Influence Diagrams
High level "blue print" for your model
Use forward and backwards modeling
Course Standards
Sensitivity
Starting Point: ID
1st worksheet - Control panel
2nd worksheet - Calculations
NPV Example
Assume Product Lifespan goes until 2020
Install RSP
SE301 Review Question
Label each level.
Develop a value measure for each tier 3 box.
UAS Example
UAS Functional Value Hierarchy
UAS MODA Model Design
SE301 Review Questions.
Why do we include the Ideal Alternative?
How can we use this chart to develop a hybrid Alternative?
Value Components Chart
Operational data

Tests

Simulations

Models

Expert opinion
SE301 Review Questions.
What are the five sources of Alternative scores?
What sources would we use for our four Alternatives for each value measure?
Scores
SE301 Review Questions.
What are swing weights?
Can swing weights be assessed before we get the scores?
Why do we normalize the weights?
Swing weight matrix
SE301 Review Questions.
What is the purpose of a value function?
What do value functions measure?
What part of the equation below is shown in the curves to the left?
Value functions
Questions:
How do we create this chart?
How do we use this chart?
What is the “unavailable value”?
What do we have to do to get an alternative with higher value than the hypothetical best?
We define the hypothetical best “hybrid” alternative
Value-Focused Thinking to create hybrid alternatives
SE301 Review Questions.
How is weighted single dimensional value calculated?
Where does the unnormalized weight come from?
What part of the equation below is shown calculations to the left?
Weighted single dimensional value.
SE301 Review Questions.
How is single dimensional value calculated?
What part of the equation below is shown calculations to the left?
Calculation of single dimensional value.
SE301 Review Questions.
How is total value calculated?
What part of the equation below are the calculations above?
Total value calculation.
Homework
Develop a constructed scale that defines all five levels.
SE301 Review Questions.
What type of value measure scale is used for all weather capability?
If we did not use this measure what would be the alternative?
Constructed scales
UAS Example
in Detail
10.1 Be able to create a direct, constructed scale for a value measure by identifying and defining the key distinctions that create value

10.2 Be able to create and interpret a Value Component Chart and use Value-Focused Thinking to create Hybrid alternatives

10.3 Understand the important linkages between the strategy table and the value model

10.4 Be able to perform and interpret a deterministic weight sensitivity analysis

10.5 Understand the concepts of Decision-Focused Transformation and be able to use DFT to compare two alternatives.
We use DFT to
Eliminate the common value and the unavailable value.
Rescale the available value to the DFT value space.
Question: How do we interpret the DFT chart?
Once we have completed our development of hybrid alternatives (using Value-Focused Thinking), we want to compare individual alternatives.
Decision-Focused Transformation (DFT)
What is the same about these two constructed scales?
What is different?
Homework Review : All weather constructed scale.
What is the x-axis?
What is the y-axis?
Why is the Ideal always 10?
How do we interpret this chart?
Swing Weight Sensitivity – UAV Endurance in Min
“No value model survives first contact with the strategies (and vice versa)”
Quantitative
Qualitative
Value Model
Our objective is to develop creative, doable strategies to achieve significant value. We use decision analysis artifacts (decision hierarchy, vision statement, issue raising, and stakeholder issue identification matrix) to develop two integrated products:
dx = decision
a-n = decision options
The strategy generation table must be linked to value model.
Changing the decision option in a decision column should affect one or more value measure scores.
If we score all strategies, and a value measure has no scores, we need to add another column in the table or we may not need the value measure.
If one decision column does not affect any value measure scores we need to delete the column or add value measures.
Strategy Generation Table
Lesson 12

WPR (lessons 1-10)

Lesson 13

Group WPR
Coming Attractions
What is P[ IED | Sensor Detects ]?

A new sensor has been developed to use in UAVs in Afghanistan to detect IEDs on or near a road. The following data is estimated:
Probability of detecting given an IED is on the road is 90%
Probability of false detection is 20%
Probability of IEDs on the road is 15%

Calculate the all conditional and marginal probability distributions using the tree flipping technique.
Class Example - Using Excel
P [ N ] P [ T | N ] = P [N T ] = P [ N | T ] P [ T ]
Solve for all the probabilities by "flipping the probability tree”
What is the probability the individual has the disease given they test positive?
Suppose that a test for a particular disease has a very high success rate:

Only 0.1% of the population have that disease (probability 0.001)
If a tested patient has the disease, the test accurately reports a 'positive', 99% of the time (probability 0.99)
If a tested patient does not have the disease, the test accurately reports 'negative', 95% of the time (probability 0.95).
We need to update our probabilities when we receive new information.

11.1 Review axioms of probability

11.2 Understand the role of Bayes Law in updating our probabilities given new information

11.3 Understand the important applications of Bayes Law
Lesson 11 Objectives
Bayesian Networks are specialized software to apply Bayes Law to large scale problems.
Bayesian network application include “computational biology and bioinformatics, medicine, biomonitoring, document classification, information retrieval, semantic search, image processing, data fusion, decision support systems, engineering, gaming and law” http://en.wikipedia.org/wiki/Bayesian_network#Applications
Medicine
Law
Fault Detection
Intelligence
Surveillance
Damage Assessment
Homeland Security
Etc.
Applications of Bayes Law
P [ N ] P [ T | N ] = P [ N T ] = P [ N | T ] P [ T ]
Solve for all the probabilities by "flipping the probability tree”
http://en.wikipedia.org/wiki/Bayes_theorem
Using Bayes Law to update probability assessment based on new information.
McGrayne, Sharon Bertsch, The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy, 2011
Beginnings to today….
Thomas Bayes born about 1702 in London
Studied logic, theology at University of Edinburgh
Presbyterian minister at Tunbridge Wells, Kent, 35 miles southeast of London
Died at Tunbridge Wells in 1761
Bayes' solution to a problem of "inverse probability" was presented in the Essay Towards Solving a Problem in the Doctrine of Chances (1764), published posthumously in the Philosophical Transactions of the Royal Society of London
Transmitted to the Royal Society by his friend Richard Price who found the unpublished paper after his death
Inverse probability
“The probability approach which endeavors to reason from observed events to the probabilities of the hypotheses which may explain them” (http://stats.oecd.org/glossary/detail.asp?ID=3686)
Uncertainty and Bayes Law
This approach works well for two outcomes but what if there are multiple outcomes?
Suppose we consider 100,000 people.

Since 0.001 have the disease, we would expect 100 have the disease and 99,900 not to have the disease

The test detects 99% of the people with the disease (true positive) (99) and the false positives are 5% (4,955)

Therefore, the probability of a disease given test positive = 99/(99 +4,995) = 0.019 (approx)
Intuitive solution
How do we calculate the probabilities we need?
Information
Performance
P[N | T]
P[T]
The probabilities we need.
The probabilities we have.
P[N]
P[T| N]
Information
Performance
The initial probability data we are provided

The probability data we need in the decision tree
In systems engineering, Bayes Law plays an important role in value of information calculations using decision trees (later class).
Representativeness
The more object X is similar to class Y, the more likely we think X belongs to Y
Availability
Too much weight on vivid, striking, or widely reported events
Too much weight on most recent events
Acceptance of illusory correlation
Anchoring
Initial estimated values affect the final estimates, even after considerable adjustments
Motivational
When making probability judgments, people have incentives to provide estimates that will benefit themselves
Major Human Probability-Assessment Heuristics/Biases
(Tversky and Kahneman, 1974)
Individuals can learn and improve performance

Awareness of heuristics and biases improves subjective probability assessment

Structured DA techniques may help avoid problems

Problem decomposition may make things easier -
More assessments required, but each assessment is easier
Implications of Heuristics and Biases
Heuristics can be viewed as relatively simple rules to make judgments and solve problems.

General rules of thumb
reduce time and effort to make decision
reasonably good estimates

Heuristics
Assessment with one probability is too aggregated to be meaningful. We should use decomposition.
8. Indian Point Unclear Reactor. You have been asked to do a risk assessment of the nuclear plant at Indian Point. You recall that stakeholder analysis is important and probability is very important. So you make an appointment with the plant manager, explain the axioms of probability and ask him “What is the probability you will have a melt down?” Is this a recommended approach?

Yes ____ No ____ Why?
Decomposition
a. Your boss insists that you give her ONE number to estimate your sales for the next year. What number do you give her? _____M

b. Next, suppose your boss says, “We expect a very difficult year due to the economy, if your sales estimate is not at least \$60M, you will have to lay off some of your salespeople.” What number would you give her? _____M
7. Sales Estimates. Suppose you are a salesman and your belief about next year’s sales in shown in the following probability density function. Of course, this means that there is no probability that sales will be less than 20 or more than 80. In addition, the area integrates to 1.
Incentives to Provide Inaccurate Probabilities
Trying to meet management expectations
Motivational Bias
Potential for motivational bias.
6. Railguns. Railguns are a potential future weapon that use electromagnetic instead of chemical energy. The technology has not been fully developed for military applications. Suppose the leading developer of Railgun’s is Railguns R Us. Recently, the company’s Chief Technology Officer testified to Congress that there was a 95% chance that Railgun weapons would be deployed in the military in 5 years. Should you be concerned about this estimate? Yes ___ No ____ Why?
Motivational Bias
7 letter words ending in “ing” are more available us.
5. Which 7-letter word form is most common?

_ _ _ _ _ n _?

_ _ _ _ i n g ?
Availability
People draw conclusions based on representative characteristics and often ignore relevant facts
3. Dawes did research on Chief Executive Officer (CEO) pet ownership as children. His survey reported a link between childhood pet ownership and career success. 94% of surveyed CEO’s owned a pet (dog, cat, or both) as children and the respondents stated that pet ownership helped them build good character traits of a CEO. Which conditional probability is described here?

a. P[CEO | PET] = 0.94
b. P[PET | CEO] = 0.94
Representativeness
People draw conclusions based on representative characteristics and often ignore relevant facts

1. Which six coin toss sequence is more likely?

a. HTHTTH
b. HHHHHH

2. Suppose you are playing roulette in Las Vegas. After a run of 4 reds in a roulette, what would be your next bet?
Representativeness
Arbitrary data anchored the thinking!
4. Suppose I randomly spin a probability wheel:

Without checking the internet, do you believe that the percentage is
More than % _____
Less than % _____

What do you believe the percentage of African countries are in the United Nations? _____%
Anchoring
Scientific principles of data collection
Documented, peer review

As probabilities are judgments, there will be controversy, and we must document and justify to show that the experts and the environment were set to reduce bias

As protocol, every assessment should include:
Background
Identification and Recruitment of Experts
Motivating Experts
Structuring and Decomposition
Probability Assessment Training
Probability elicitation and verification
Aggregation of experts’ probability distributions
Protocols: Pulling it all Together
Lesson 15

Read: C & R (Ch. 8 pg.295-311)

Do Problems: 8.1, 8.4, 8.11
Coming Attractions
Centers for Disease Control, www.bt.cdc.gov/Agent/Agentlist.asp
Decomposition Example: DHS Bioterrorism Risk Assessment, 28 bioagents were considered.
14.1 Understand the heuristics and biases we use when we process uncertain information
Representativeness
Availability
Anchoring
Motivational

14.2 Understand decomposition and how it applies to subjective probability assessment
Lesson 14 Objectives
.05
.95
.4
.8
.2
.6
LS
HS
Example:
with
What is prob. I will graduate if the dean raises academic requirements?
What is prob. I will graduate if the dean reduces academic requirements?
Scenario #1: Conditioning
DHS (Department of Homeland Security). 2006. Bioterrorism Risk Assessment. Biological Threat Characterization Center of the National Biodefense Analysis and Countermeasures Center. Fort Detrick, Md.
The chart is a simplification of the 17-step event-tree (18 step with consequences) that could lead to the deliberate exposure of civilian populations for each of the 28 pathogens.
The biannual DHS Bioterrorism Risk Assessment (BTRA)
used probabilistic risk analysis with event trees.
All of probability theory is based on these three simple axioms!
“When the facts change, I can my opinion. What do you do sir?”

John Maynard Keynes, Economist
Three major axioms of probability
Sound theory is based on credible axioms.
1. If the coin flips are independent, both of are equally likely. However, it is more likely that we get 3 Hs than 6 Hs.
2. The three reds should not influence your bet. This is a very small sample.
Solution
3. P[PET | CEO] = 0.94
Solution
4. What percent of African countries are part of the UN? Answer: (at the time it was 35%)
Wheel rigged to 10 average guess was 25%
Wheel rigged to 65 average guess was 45%
Solution
Solution
Probabilistically, a. must have more words!
(b. is a subset of a.)
Solution
Fear that individual will be held accountable for “achieving the outcome”
Pre-test
Decomposition
Definition – Breaking a probability assessment into smaller and more manageable chunks

Purpose – To make probability assessments easier and to give a clearer view of the uncertainty in a decision problem!

Potentially appropriate in three different scenarios:
Scenario 1: Uncertain event can be conditioned on another event
Scenario 2: Multiple possible causes for uncertain event
Scenario 3: Uncertain event results from a chain of causes
Example:
with
What is probability I will graduate if the dean raises academic standards (HS)?
What is probability I will graduate if the dean reduces academic standards (LS)?
Scenario #1: Conditioning
Scenario #2: Chain of Causes
Example:
What is probability oil rig explodes?
with
What is probability of critical pipe failing?
Then safety measures failing?
Then no-one noticing in time?
Example:
What is probability you will not graduate?
Scenario 3: Multiple Causes
Solution:
What is probability of a serious injury?
What is probability of failing to meet academic requirements?
What is probability of cheating incident?
. . . etc.
Examples
http://toughmudder.com/worlds-toughest-mudder-series-finals/worlds-toughest-mudder-qualifiers/
http://eb.gmnews.com/news/2012-12-06/Front_Page/Brothers_spend_two_days_Roman_through_obstacles.html
Heads I win, Tails you lose.
Tossing a Coin: Can you use Bayes Law to determine whether or not a coin is fair?
Tossing die and rolling 1's
LTC Schott's Solution
9.Probability. Based on extensive studies, it has been found that 1% of the population has Some Horrible Disease (SHD). A new test for SHD has just been released; this test has been shown to be 98% accurate in clinical trials. If a person takes this test and the results come back positive for SHD, then what are the chances that the person actually has SHD.
a.33%
b.19%
c.82%
d.64%

10.Probability. In 1982 John Hinckley was on trial, accused of having attempted to kill President Reagan. During Hinckley’s trial, Dr. Daniel R. Weinberger told the court that when individuals diagnosed as schizophrenics were given computerized axial tomography (CAT) scan, the scans showed brain atrophy in 30% of the cases compared with only 2% of the scans done on normal people. Hinckley’s defense attorney wanted to introduce as evidence Hinckley’s CAT scan, which showed atrophy. The defense argued that the presences of atrophy strengthened the case that Hinckley suffered from mental illness. Given that approximately 1.5% of the people in the United States suffer from schizophrenia, what is the probability that Hinckley suffered from schizophrenia?
a.33%
b.19%
c.82%
d.64%
Would you recommend admitting the test results as evidence?
Pre-Test Questions
Familiar Questions?
9. 33%
10. 19%
Solution
Homework Handin
15.1 Understand why decision analysis uses probabilities to describe uncertainty
15.2 Distinguish between the frequency and the subjective views of probability
15.3 Understand that people do a poor job of estimating extremes and how structured techniques can improve their ability to estimate probabilities
15.4 Apply discrete methods to assess subjective probability:
Direct assessment
Reference lottery
15.5 Apply continuous methods to assess subjective probability:
Fractile Method
Potential Outcome Method
15.6 Compare fractile and outcome methods for consistency and revise
Lesson 15 Objectives
Provide the 0.01, 0.5, and 0.99 fractiles

What is the length of the world’s longest central span in a suspension bridge?

The length of the Nile River?

The year Thomas Bayes was born (Bayes Law)?

Diameter of the moon?

Year with the highest unemployment rate between 1948-2010?
How good are we at estimating uncertainties?
Find value of p for which expert is indifferent to following choice for the event of the 2013 Army-Navy Football game.
Discrete Method #2:
Reference Lottery
The probability wheel is a commonly used visual model to help with direct assessment.
Probability Wheel
Ask the expert what his/her belief is
Must clearly define the event
Easy
Expert’s may not be comfortable providing probabilities without training
Discrete Method #1:
Direct Assessment
In what order should we assess the three probabilities? Why?
Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Decision Analysis Handbook, Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013
The P50 is the 0.50 fractile, the number x, such that P [ X < x ] = 0.50
Documentation is important define the uncertainty and estimate the extremes.
The 0.05 fractile is the number x, such that P [ X < x ] = 0.05
Which technique is more subjective?
Frequency: If we did this many times, what is the long-run frequency of the outcomes of interest?
Relies on past data or the ability to conduct experiments.
What is the probability of “heads” in a sequence of 6 flips?
Unable to provide information for many of our most important problems

Subjective: What is your degree of belief concerning the outcomes of interest?
Relies on subjective interpretation of all relevant information to include data, experiments, simulation results, and expert opinion
What is the market for a new product, e.g., iPhone?
What is the probability a new technology will provide a certain performance level?
Two views of probability
We have
Deterministic value model
Alternatives
Tornado diagram that identifies the most sensitive parameters

We need to use
Probability distributions to quantify uncertainty for the most sensitive parameters
Probability assessment protocols to obtain this information

We must consider
Heuristics and biases
How good the expert is at quantifying uncertainty
Transition from deterministic to probabilistic modeling.
Common verbal phrases are not reliable for decision making.
Need
Subjective probability estimate for continuous random variable
e.g., oil in well, casualties in a conflict

Approach
Find an approximation of CDF
Determine some points on CDF
Method #1: specify cumulative probability (fractile),assess potential outcome
Method #2: specify variable potential outcome, assess cumulative probability (fractile)
Connect the dots
Continuous Assessment Methods

The weather forecaster says it is likely it will rain tomorrow

The Gold Coats say it is likely they will fix your computer in one hour

The weather forecaster says it is very likely it will rain tomorrow

The Gold Coats say it is very likely they will fix your computer in one hour
Should we use words instead of probabilities to describe uncertainty?
Original Estimate
7.5 K gal!
Cleanup technology effectiveness depends on the amount of the spill.
Probability assessment elicited from the manager of the degreaser during the time of operation.
Below ground
Drain
Chemical Separation Facility
Aquifer
Ground Level
Degreaser
Large Dept of Energy Building
DNAPL Plume
Probability
DNAPL Volume (1000 Gallons)
500
450
400
350
300
250
200
150
100
50
0
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
Cumulative Probability Distribution of DNAPL Volume
Spill Volume Uncertainty
What should we do next?
Comparing the two techniques can provide insights about inconsistencies.
Continuous Method #2
Assess Probabilities for Potential Outcomes -- Example
Assess probabilities for potential outcomes
Pr[X  0] = 0
Pr[X  1,000,000] = 1
Pr[X  100,000] = 0.05
Pr[X  900,000] = 0.95
Pr[X  250,000] = 0.08
Pr[X  750,000] = 0.92
Pr[X  500,000] = 0.50
Connect the dots
Find extreme values
Pr[X  0] = 0
Pr[X  1,000,000] = 1
For each fractile, assess potential outcomes X1.. X5 such that:
Pr[X  X1] = 0.1
Pr[X  X2] = 0.9
Pr[X  X3] = 0.25
Pr[X  X4] = 0.75
Pr[X  X5] = 0.5
Connect the dots
Continuous Method #2
Fractiles - Example
Lesson from behavioral research: Start at extremes first, and work to 0.5.
Find extreme values
Pr[X  0] = 0
Pr[X  1,000,000] = 1
For each fractile, assess potential outcomes X1… X5 such that:
Pr[X  X1] = 0.1
Pr[X  X2] = 0.9
Pr[X  X3] = 0.25
Pr[X  X4] = 0.75
Pr[X  X5] = 0.5
Continuous Method #1
Fractiles - Example
Find approximation for CDF
X = barrels of oil in well
Assess a continuous distribution
0
1
0.05
0.95
0.2
0.80
0.50
Lesson from behavioral research: Start at extremes first, and work to 0.5.
Assess probabilities for potential outcomes
Pr[X  0] =
Pr[X  1,000,000] =
Pr[X  100,000] =
Pr[X  900,000] =
Pr[X  250,000] =
Pr[X  750,000] =
Pr[X  500,000] =
Continuous Method #2
Assess Probabilities for Potential Outcomes -- Example
QUESTION:
Write your probability for each of the following event:
Provide the 0.01, 0.5, and 0.99 fractiles - Solutions

What is the length of the world’s longest central span in a suspension bridge? Akashi-Kaikyo Bridge, Japan, 6529 feet, 1991 meters

The length of the Nile River? 6,650 km (4,130 miles) long – longest river in the world

The year Thomas Bayes was born (Bayes Law)? 1702

Diameter of the moon? 2,159 miles or 3,474 km.

Year with the highest unemployment rate between 1948-2010? 1982, Reagan
Solutions
See DA Handbook Chapter 10 for more complete discussion.

In the early 2000’s Dr. Parnell worked on a research project to cleanup a underground DNAPL (bad chemical) caused by two incidents at the DOE nuclear site at Paducah, KY. First, the floor of the degreaser which used the DNAPL to clean equipment with radioactivity had rotted out. Second, the underground pipe designed to transport spilled chemicals to a separation facility had a fork lift hole in the BOTTOM of the pipe. The initial estimate of the spill into the ground was 7,500 gallons. The concern was the that DNAPL was in the aquifer. The DNAPL seeped into the ground until the ground above the pipe caved in. About 20 technologies were considered for the cleanup. The technology’s effectiveness were very dependent on the size of the spill. Several month’s into the study, Dr. Parnell noted that the cleanup site evacuations seem to suggest a much larger spill. The engineer who provided the data stated that he had looked at the records but they were very poor. The above distribution was assessed in a one hour session with the manager of the degreaser. The results of the assessment fundamentally changed the study.
Procedure:
Offer p’s that favor first one side, then the other
Begin with extremely wide brackets and gradually converge on a p that yields indifference
Go slowly; allow DM to think hard
Consider using “probability wheel” to visualize p:

Would you rather have a 90% chance of winning \$1,000 or would you rather receive you outcome based on the results of the game. Presumably, they will take p = 0.90
Would you rather have a 10% chance of winning \$1,000 or would you rather receive you outcome based on the results of the game. Presumably, they will take outcome of the game.
Narrow the questions until they are indifferent.
Conclude P[Army wins] = p

Decision trees without information (w/o and w/PT)
BRING LAPTOP TO
EVERY CLASS

Read: C & R (Ch. 3 pg. 69-74, Ch. 4 pg. 115-119)

Do: 3.9.a, 3.11, 4.4, 4.8
Next Time
Select an important uncertainty that will impact your future

Assess a continuous probability distribution two ways:
Fractile Method
Potential Outcome Method

Compare fractile and outcome methods for consistency and revise
Board Challenge
Parnell, G. S., Bresnick, T. A., Tani, S. N., and Johnson, E. R., Decision Analysis Handbook, Wiley & Sons, Inc. Operations Research/Management Science Handbook Series, 2013
Use the template
16.1 Assess a continuous probability distribution two ways:
Fractile Method
Potential Outcome Method

16.2 Compare fractile and outcome methods for consistency and revise
Lesson 16 Objectives
Decision Trees
17.1 Understand components of decision trees.

17.2 Be able to construct a decision tree from an influence diagram and solve for the optimal solution.

17.3 Understand the strengths and weaknesses of expected value

17.4 Be able to plot and interpret an risk profile using a cumulative probability chart
Deterministic dominance
Stochastic dominance
No dominance
Lesson Objectives
How can we use this information to obtain insights?
Plot the risk profiles and cumulative risk profiles for each decision
2015
2016
2017
Board Problem 1c
Add the probabilities and outcomes to your decision tree and solve for the best decision using the decision tree algorithm.
Board Problem – 1b
BRING COMPUTER
HW#18
Read: Precision Tree Quick Start Tutorial

(PDF and/or Video)

Do: HW#18 by hand or with Precision Tree
Next Time
Draft the influence diagram
A company must decide to develop a new product, continue to market the current product, or cancel the product line. The current product demand is expected to be the same as last year or worse. There are two new product uncertainties. The first uncertainly is new product performance. The performance may be high or low. The second uncertainty is demand. The new product demand may be large or small.
Board Problem 2a
Expected value is a summary statistic that hides the extreme outcomes. This is why we must consider risk.
Limitations of Expected Value
Expected Value- is “simply the weighted average of the possible outcomes…the weights being the chances with which the outcomes occur.” - Clemen & Reilly pg. 116
Another interpretation is that expected value is what we would expect to happen on average if we were to repeatedly observe the outcome of an uncertainty.
Most of the time, the expected value isn’t even a possible outcome. For example, the expected value of rolling one die is 3.5…
SOLVING FOR EXPECTED VALUE IN A DECISION TREE:
Construct the decision tree.
Add probabilities to the outcomes of the uncertain variables and values to end of the tree based on the path thru the tree.
“Average out and fold back the tree” from right to left.
Calculate expected value of chance nodes.
Choose best alternative at decision nodes.
Label nodes appropriately (expected values and decision path)
Continue until expected value can be assigned to root node.
DEFINITION: A discrete uncertain quantity’s expected value is the probability-weighted average of its possible outcomes. If X can take on any value in the set {x1, x2,…, xn}, then the expected value of X is the sum of x1 through xn, each weighted by the probability of its occurrence. -Clemen & Reilly pg. 259
How to solve a decision tree
Construct a decision tree from the following influence diagram
Board Problem 1a
Statistician drowns crossing the river.
The Flaw of Averages: Why We Underestimate Risk in the Face of Uncertainty, by Dr. Sam Savage
with Illustrations by Jeff Danziger
http://flawofaverages.com/
The flaw of averages
Convert the influence diagram to a decision tree
using the data in the Board Problem 2 file and solve.
Board Problem 2b
Is there dominance?
If so, what kind, which strategy, and why?
Example C
Now:
We should use expected value to decide if we are risk neutral across the range of possible outcomes (or we are a large company)

If we are risk averse or risk seeking across the range of possible outcomes, we must consider risk.
WOULD YOU MAKE THIS DECISION?
EV = \$800
(0.9)(\$12K) + (0.1)(-\$100K) = \$800
-\$100,000
\$12,000
\$0
0.1
0.9
Example: Limitation of EV
Is there dominance?
If so, what kind, which alternative, and why?
Example A
Is there dominance?
If so, what kind, which strategy, and why?
Example D
Is there dominance?
If so, what kind, which alternative, and why?
Example B
“For a Max Problem: Alternative R stochastically dominates alternative Q if, for any possible outcome, R has an equal or better chance of resulting in an outcome that is equal or better.”
Stochastic Dominance
For a Max problem: “Alternative P deterministically dominates alternative Q if the worst outcome of P is as good as (or better than) the best outcome of Q.”
Deterministic Dominance
Influence Diagrams
High level "blue print" for your model
Use forward and backwards modeling
Influence Diagrams
High level "blue print" for your model
Use forward and backwards modeling
Decision Trees
Basic Nodes
Example - 3.9a (your homework. . .)
Build from left to right
Expected Value
Example - 4.8 (from your homework. . .)
Board Problem: Product launch. . . iPhone22??
Board Problem 2
Cumulative Risk Profiles
R stochastically dominates Q.
“For a Max Problem: Alternative R stochastically dominates alternative Q if, for any possible outcome, R has an equal or better chance of resulting in an outcome that is equal or better.”
Stochastic Dominance. Down and to the right for a max.

One outcome is the same but the rest of the outcomes are better.
The two plots intersect so there is no dominance.

Alternative B has both the worst outcome and the best. This is a common occurrence. Many times new alternatives have both high potential and high risk.
Deterministic Dominance, B is the preferred alternative since we are seeking the min cost solution.
How reliable is Expected Value?
1. Read: Precision Tree Quick Start Tutorial (PDF and/or Video)

2. Solve the following problem with a decision tree by hand or with Precision Tree. Construct a decision tree from the following influence diagram and solve for the highest expected value decision.
Homework
Precision Tree
18.1 Be able to formulate and solve a decision analysis problem using Precision Tree software and interpret the results

18.2 Be able to interpret a Precision Tree Risk Profile using a Cumulative Chart
Deterministic dominance
Stochastic dominance
No dominance

18.3 Be able to perform and interpret one way and two sensitivity analysis using Precision Tree
Lesson 18 Objectives
MEET IN THE LAB - MH 203
Lesson 19 - Practice Graded Lab
Bayes Law
Decision Tree analysis with Precision Tree

Bayes Law
Decision Tree analysis with Precision Tree
Next two classes

1. Make sure to uninstall any old versions or textbook versions
before installing this version.

2. The install files are located here =>

\\usmasvdfcase1se\install\DTools601Ind\

3. Right-click on DTS601-cust-Setup.exe then select Run As
Administrator. During setup, accept all defaults.
Install Precision Tree Instructions
One decision with 3 alternatives
One uncertainty with 3 outcomes. For this problem, the outcomes depend on the year but the probabilities do not. In general, the probabilities could depend on the alternative.
The Market Size branch data are the probabilities and the outcomes.
The data at the end of the tree (leaves) are the path probability and the path outcome. For large trees, these will NOT be the same as the last node. NEVER edit the blue calculations.
69 is the expected value of product launch in 2016. The other two EVs are 0.5 and 62.7.
True means that the best expected value is 2016. False is for the alternatives not selected.
Precision tree has a setting to change from max to min if we were using expected cost.
The x axis is the possible outcome and the y axis is the probability of each outcome.
For each alternative, the sum of the probabilities of each of the three possible outcomes in 1.
This chart is easier to understand than the cumulative but harder to get insights for complex problems.
Insights
2015 is dominated
2016 is generally better than 2017 but has a lower possible outcome.
The x axis is the possible outcome and the y axis is the cumulative probability of each outcome.
For each alternative, the sum of the probabilities of each of the three possible outcomes in 1.
This chart is harder to understand than the probability chart but easier to get insights for complex problems.
Insights
2015 is dominated (see dotted line overlay)
2016 is generally better than 2017 but has a lower possible outcome.
One Way Sensitivity
Two Way Sensitivity
Settings
Sensitivity Analysis
Analysis Type: One-Way Sensitivity (default)
Include Results. Specify all.
OK.
Precision Tree One Way Sensitivity
Cell E6. We have to name the cell. We want to vary the probability of high market. We have to also put equations in cells C6 and D6 to insure that the probabilities always add to one.
Cell E4. We only have to name the cell.
How many times did Precision Tree solve the decision tree to generate these graphs?
Perform sensitivity analysis for the probability of high market. Hint: Always name the cell with the sensitivity analysis parameters.
Sensitivity Analysis
Perform sensitivity analysis for the 2016 high market. Hint: Always name the cell with the sensitivity analysis parameters.
Sensitivity Analysis
Assess which of the two parameters is the most sensitive.
Sensitivity Analysis
Interpretation of charts (Warning: Y axis scales are different)
The chart of the left shows to sensitivity of the best EV to P(high market). It does NOT tell you which alternative is the best.
The chart of the right shows the sensitivity of all three alternatives to P(high market). This chart means that our 2016 is the best alternative for all values of P(high market).
Precision Tree solved 11 decision trees to get this data.
Interpretation of charts (Warning: Y axis scales are different)
The chart of the left shows to sensitivity of the best EV to 2016 High Market Outcome. It does NOT tell you which alternative is the best.
The chart of the right shows the sensitivity of all three alternatives 2016 High Market Outcome. This chart means that 2016 is the best alternative unless the 2016 High Market Outcome is less that 85.
Interpretation of charts (Warning: X axis scales are different).
These charts show the same message: the probability of high market is more sensitive than the high market (over the range we selected).
The Tornado Graph shows the range in EV \$
The Spider Graph shows the change in input as a percent (X axis) and the impact on EV (Y axis).
Settings
Sensitivity Analysis
Analysis Type: Change to Two-Way Sensitivity.
Inputs: Select variables for X and Y.
Include Results. Strategy Graph and Strategy Region are the defaults.
OK.
Precision Tree Two Way Sensitivity
How many times did Precision Tree solve the decision tree to generate these graphs?
Assess which of the two parameters is the most sensitive.
Two Way Sensitivity Analysis
Interpretation of charts. These charts show two way sensitivity.
The Sensitivity Graph (2-Way) shows the range in EV \$ as both parameters are varied but does not tell you the best alternative.
The Strategy Region (2-Way) Graph shows the best alternative for the two parameter values.
2016 is the best alternative in the blue range and 2017 is the best in the red range.
Precision Tree ran 99 runs to get this data (9 times 11)
The Value of Information
Imperfect Information
Calculate the Expected Value with Perfect Information and the Value of Perfect Information using Precision Tree influence diagram.
Class problem and HW 22
How do we decide how much to pay for information?
Perfect Information
Questions:
What is the purpose of a test/survey vs. marketing? Which would you spend more money on?
Can you give an example of perfect information? perfect control?
On the same decision, which is more valuable perfect information or perfect control?

21.1 Understand the role of technical and market information in engineering management and systems engineering decision making

21.2 Understand and be able to calculate the value of perfect information using Precision Tree

21.3 Understand and be able to calculate the value of control
Lesson 21 Objectives

Value of Imperfect Information
BRING LAPTOP TO
EVERY CLASS

Homework: HW#22
Next Time
EVPC > EVPI > EV
Mathematical relationships between EV, EVII, EVPI and EVPC:
EV of PI = 6
EV of PI = 0.6
EV of PI = 0
Principle. Information only has value if it could cause us to change our decision.
What causes the EV of PI to be positive?
What is the value of information?
Which one is clairvoyant?
Technical Information:
Will the system work as planned?
Market Information:
What will be the market for the systems?
Perfect Information
Draw and Influence Diagram and Decision Tree to represent this situation and calculate the Expected Value
Use Precision Tree to calculate the Expected Value
What is the value of Perfect Control?
Update the Influence Diagram to reflect CPT Thomas' Perfect Information. What is the value of this information?
What does the converted Decision Tree look like? Where do the probabilities come from?
Update the Influence Diagram to reflect Mr. Abdul's Imperfect Information. What are the associated probabilities of this information?
Using the fact that Mr. Abdul is in fact only 70% accurate, update your Influence Diagram. What is the Value of this information?
What does the converted Decision Tree look like? Where do the probabilities come from?
What is
Perfect Control?
What is
Perfect Information?
How do you modify the Precision Tree ID to solve for the Expected Value with Perfect Information on New Product Market?
Calculate the EV of Perfect Information (Market)
New Product
Market
Large
Small
Low
High
Cancel product line
Market current product
Develop new product
New Product
Performance
NPV
Product
Decision
Review: Excel Problem 1c. EV of PI
New Product
Market
Large
Small
Low
High
Cancel product line
Market current product
Develop new product
New Product
Performance
NPV
Product
Decision

22.1 Understand the role of technical and marketing information in engineering management and systems engineering decision making

22.2 Understand the mathematical relationship between
a. EV, Expected Value with EV, EV with Imperfect Information, EV with Perfect Information, and EV with Perfect Control
b. EV of Imperfect Information, EV of Perfect Information, and EV of Perfect Control

22.3 Understand and be able to calculate the value of perfect information and imperfect using Precision Tree
a. Develop a Precision Tree ID from an ID and solve for EV
b. Convert a Precision Tree ID to a symmetric decision tree (Model > Convert to Decision Tree)
c. Simplify the decision tree using Collapse Child Branches
Lesson 22 Objectives
Would you rather have perfect information about New Product Performance or Market?
How do you modify the Precision Tree ID to solve for the Expected Value with Perfect Information on New Product Performance?
Calculate the EV of PI (Performance)
New Product
Market
Large
Small
Low
High
Cancel product line
Market current product
Develop new product
New Product
Performance
Product
Decision
Review: Excel Problem 1b. EV of PI
How do you modify the Precision Tree ID to solve for the Value of Perfect Information on both New Product Performance and Market?
New Product
Market
Large
Small
Low
High
Cancel product line
Market current product
Develop new product
New Product
Performance
NPV
Product
Decision
Excel Problem 1d. EV of PI for
two variables.
Desk Problem 1a
How do you modify the ID in PT to calculate EV with PI?
What is the EV with PI?
What is the EV of PI?
Expected Value of Perfect Information on performance.
Yes
No
Performance
NPV
Launch
Product

23.1 Understand the role of technical and marketing information in engineering management and systems engineering decision making

23.2 Understand the mathematical relationship between
a. EV, EV with Imperfect Information, EV with Perfect Information, and EV with Perfect Control
b. EV of Imperfect Information, EV of Perfect Information, and EV of Perfect Control

23.3 Understand and be able to calculate the value of perfect information and imperfect using Precision Tree
a. Develop a Precision Tree ID from an ID and solve for EV
b. Convert a Precision Tree ID to a symmetric decision tree (Model > Convert to Decision Tree)
c. Simplify the decision tree using Collapse Child Branches
Lesson 23 Objectives
Yes
No
Launch
Product?
Performance
Test
How do you modify the ID in PT to calculate EV with II?
What is the EV with II?
What is the EV of II?
Expected Value of Imperfect Information on performance a performance test.
Yes
No
Performance
NPV
Test?
Calculate the EV for the product launch decision using influence diagrams in PT .
Yes
No
Performance
NPV
Launch
Product
0 ≤ EV of II ≤ EV of PI ≤ EV of C
EV ≤ EV with II ≤ EV with PI ≤ EV with C
0 ≤ 2.7 ≤ 6 ≤ 21
9 ≤ 11.7≤ 15 ≤ 30
Three decision analysis techniques are very useful in guiding the design of development tests and marketing activities
Value of Imperfect Information – Provides an upper bound on the most you would pay for a specific test or marketing analysis
Value of Perfect Information – Provides an upper bound on the most you would pay for a test or marketing analysis
Value of Perfect Control – Provides an upper bound on the most you would pay to achieve the best outcome (e.g., advertizing)
Calculate and compare.
Expected Value OF Perfect Information
Expected Value WITH Perfect Information
Expected Value OF Perfect Control
How much would you pay for these commercials?
Expected Value of Perfect Control = Max Value minus Expected Value
EVC = Max - EV
Solution
EVC = Max Value - EV
EVC = 105 - 69
EVC = 36
What
is the value
of this knowledge?
Value of Perfect Information – Provides an upper bound on the most you would pay for a test or marketing analysis
Value of Perfect Control – Provides an upper bound on the most you would pay to achieve the best outcome (e.g., advertizing)
Expected Value OF Perfect Control
Create a basic influence diagram for the MiO product launch.
Using the data in the Schott Class Data file and Precision tree, make in influence diagram.
Convert your influence diagram to a decision tree.
Based on expected value, what is the optimal launch year and expected NPV?
Solution
Steps in Precision Tree
Desk Problem 1b
Solution
Determine what the probabilities associated with perfect information and fill in the table.
Update your influence diagram to reflect conducting a market survey that will reveal perfect information
Convert your influence diagram to a decision tree
Desk Problem 1c
Based on your updated influence diagram and decision tree, what in the expected value
WITH
perfect information?
What is the expected value
OF
perfect information?
What is the most you would recommend paying for market analysis or a market survey?
Solution:
EV with PI = 69.6
EV = 69
So, the EV of PI is the difference.
EV of PI = EV with PI - EV
= 69.6 - 69
= .6
This is the upper bound of what you should pay for market research.
Desk Problem 1d
Solution
Perform a sensitivity analysis on Market being Low, and a 2017 launch.
What does the curve mean?
Why do companies pay thousands of dollars for surveys and millions of dollars for advertising?
Monte Carlo Simulation
Issue List
From Influence Diagram to Decision Tree
3.11a
What is
Imperfect Information?

Value of Imperfect Information – Provides an upper bound on the most you would pay for a
specific
test or marketing analysis
Desk Problem 1a
Solution
Create a basic influence diagram for the updated MiO product launch.
Using the data in the Lsn 22 Data file and Precision tree, make in influence diagram.
Convert your influence diagram to a decision tree.
Based on expected value, what is the optimal launch year and expected NPV?
Steps in Precision Tree
Desk Problem 1 b, c, and d
Update the Influence Diagram to reflect the following:
1. The expected value with perfect information of the perfect product performance
2. The expected value with perfect information of the perfect product market
3. The expected value with both
Desk Problem 1e - Imperfect Information
Update the Influence Diagram to reflect Performance Test predictions.
Expected Value WITH Imperfect information

Expected Value OF Imperfect Information
WPR 2 Part 3 Example
You are a platoon leader on an important and dangerous mission in the Ghazni provenance of Afghanistan. You have just landed by helicopter 2 kilometers from your objective. Your platoon has two ways to reach its objective: (1) traveling off road through a village with an expected time of 30 minutes; (2) traveling on an unimproved road with an expected time of 15 minutes if no IEDs are present, but 60 minutes if there is an IED present because you will have to clear the IED. There is only a 15% chance that the road has an IED. CPT Thomas, a Special Forces Team Leader, can give you perfect information on whether there is an IED on the road. Another option is to consult Mr. Abdul, a pro American tribal elder, who lives nearby and can assess the likelihood of an IED being in the road. Mr. Abdul is not a perfect predictor. If there is an IED, the conditional probability is 0.75 that he will say there is an IED in the road. If there is no IED, the probability is 0.85 that he will say "no IED." You want to minimize the expected time to reach the objective.
SODA
MODA
Risk Solver Platform
@Risk
Lesson 26 Objectives
26.1 Introduce simulation and the DSE simulation courses

26.2 Understand the theory of Monte Carlo simulation

26.3 Use Risk Solver Platform to perform a Monte Carlo Simulation on an decision model

26.4 Be able to modify a Monte Carlo simulation model to create a “decision tree” like capability to model the value of information
Next Time
Homework 27
When/why a Simulation Model vs Analytical Model??
When inputs and processes are uncertain
When mathematical complexity makes it hard to provide “close form” results.
A simulation develops a model to numerically evaluate a system performance over time.
By estimating performance characteristics of the system, the best alternative from a set of alternatives under consideration can be identified.
1. Which are the sources of uncertainty?
2. What do we usually care about?
Figure 2.6, Parnell, G. S., Driscoll, P. J., and Henderson D. L., Editors, Decision Making for Systems Engineering and Management, 2nd Edition, Wiley & Sons Inc., 2011
1. Which are the sources of uncertainty?
Usually the inputs but we can also have uncertainty in the controls and the mechanisms.
2. What do we usually care about?
Uncertainty in the outputs
We begin with a system model with inputs, processes, and outputs.
Due to uncertainty, we assign distributions for the uncertain inputs and, possibly, the uncertain process events.
Monte Carlo simulation* generates inputs from input distributions and process distributions, processes the inputs, and generates distributions of the outputs.

We study the distributions of the outputs obtained by simulating a large number of runs of the model to determine the best system design.
Five steps:
Step 1: Create a parametric model,
y = f(x1, x2, ..., xq).

Step 2: Generate a set of random inputs,
xi1, xi2, ..., xiq.

Step 3: Evaluate the model and store the results as yi.

Step 4: Repeat steps 2 and 3 for i = 1 to n simulations

Step 5: Analyze the results using frequency, cumulative,
summary statistics, confidence intervals, etc.
* Developed by Manhattan Project physicists working on nuclear bomb design.
Revisit initial SODA problem -
Product Launch
Desk Problem 1
Use Excel Risk Solver Platform for Simulation
Name E10, E13, E16, and E19 then do Distributions > Common > Triangular (Low, Base, High)
Delete the index function in the cells
Use cell referencing
Name B29 NPV and designate as output (Results > Output > In Cell)
Look at Model > Simulation on right hand box
Verify in Options > All Options sampling method is Monte Carlo and 1000 trials
Simulate
Add marker for mean, name marker (Statistics > Markers > Type > Mean)
Do F9 a couple times
Copy and paste distribution into Word or Powerpoint
Show options on drop down menu (statistics, chart options, etc)
Paste the Sensitivity into Word or Powerpoint
Deterministic model was 2016, price 35, 64,588
Interpret the two charts
Interpretation
The Mean is approx 60.6 (very close to the 64.6 with deterministic). However, the range can be 30 to 95!
Market size is the major uncertainty.
Blue means increase in variable increases the NPV.
Red means increase in variable decreases the NPV.
Desk Problem 2
The updated model selects the price that maximizes Revenue – Production Cost (a surrogate for NPV).
In this model, the objective function is relatively flat around the optimal price.
In a deterministic model, 35 is still the optimal price.
What will happen when we perform Monte Carlo Simulation?
How can we modify our model to make it more realistic?
Rerun MC Simulation
EM481 Systems Simulation
Lesson 27 Objectives
27.1 Review the use Risk Solver Platform to perform a Monte Carlo Simulation on an decision model

27.2 Be able to perform simulation optimization to optimize the decision variables under uncertainty
Next Time
Homework 28
Use the Product NPV Model v5 130322 MC Lesson 27.
Compare the results of 2015, 2016, and 2017 by looking at the probability density functions and the price decision distribution.
What product launch year to you recommend and why?
Desk Problem 2 Con't
Use Excel Risk Solver Platform for Simulation
Name E5, E6, and E7 then do Distributions > Common > Triangular (Low, Base, High)
Delete the index function in the cells
Use cell referencing
Name B29 NPV and designate as output (Results > Output > In Cell)
Look at Model > Simulation on right hand box
Verify in Options > All Options sampling method is Monte Carlo and 1000 trials
Simulate
Add marker for mean, name marker (Statistics > Markers > Type > Mean)
Do F9 a couple times
Copy and paste distribution into Word or Powerpoint
Show options on drop down menu (statistics, chart options, etc)
Paste the Sensitivity into Word or Powerpoint
Results
Lesson 28 Objectives
28.1 Understand the sources of uncertainty in a MODA model

28.2 Understand how we can use multiple objective decision analysis to model the impact of independent and dependent uncertainties

28.3 Be able to compare uncertain alternative and select the best decision that balances value and risk

28.4 Be able to interpret a probabilistic tornado diagram
Lesson 29, Risk Attitude
C & R (Ch. 13 pg. 527-539)
Problems 13.1, 13.3, 13.4, 13.6

Lesson 31
Submit draft single and multiple objective deterministic decision models (SODA and MODA) in Excel. One file, labeled yourname.xlxs by start of class.
Sources of
Uncertainty??
Desk Problem
Use the same Monte Carlo simulation approach as SODA

Put uncertain distribution on the independent variables
Scores if independent variables
Probability of detection since all UAVs use different sensors
Non-score independent variable if 1 or more scores depend on an independent variable
Suppose weight, range, and endurance dependent on material strength and Micro Hawk and Micro Eagle use the same new composite material

We are going to use @Risk for the Monte Carlo Simulation
Color Cells > Color @Risk Function Cells
Define distributions for independent uncertainties cells F15, F16, J15, J16, and L87
Add outputs and name for cells R32, R33, R34, R35, R37 (Alternative Values)
Application Settings > Reports > Place Reports In > Active Workbook > OK
Iterations > 1,000
Start Simulation
Browse Results > Add Overlay, select cells R32, R33, R34, R35, R37
Select Type of Distribution > Cumulative Ascending
Name the chart title > Comparison of alternatives
What can we conclude about the alternatives?
Explain VFT versus risk management
Put cursor on output cell, e.g, R33
Browse Results > Select tornado graph symbol > Tornado – Change in Output Mean
Interpret the results
How can you manage the risk of the material strength?
What does the tornado diagram look like for the Raven? Why?
Interpret the results. The material strength and P[D] are the major uncertainties. All weather is a minor uncertainty for both.
How can you manage the risk of the material strength? Use proven materials. If you have to do a new development, add a development test to assess strength early in the program.
What does the tornado diagram look like for the Raven? Why? There is no tornado diagram since there are no uncertainties.
Monte Carlo simulation is the same concept for SODA and MODA models
MODA models need to include the probabilistic dependence so distributions are only placed on independent variables
New systems usually have higher EV but more risk than existing systems
Systems engineers and engineering managers must focus on increasing value and managing risk
Summary
SODA vs MODA
SODA Model
Converting MODA Models to SODA Models
MODA Model
Converting Scores to \$\$
PhD or No PhD??
Functional
Hierarchy
Results
Decision
Trees
New Timeline!
DROP
Lesson 29 Objectives
29.1 Review the development of a MODA deterministic model

29.2 Understand how to convert MODA scores to dollars using your value preferences

29.3 Be able to develop a SODA deterministic model from the MODA deterministic model scores

29.4 Be able to model uncertainty for both models
33.1 Understand the three basic risk attitudes: risk seeking, risk neutral, and risk averse

33.2 Given an exponential utility function and an assumed R, determine the EV, EU, and CE.

33.3 Use the certain equivalent (CE) for a given situation to calculate R for the exponential utility function

33.4 Be able to calculate the risk premium for an exponential utility function.

34.1 Understand the assumptions required to use the exponential utility function

34.2 Be able to assess the exponential utility function risk tolerance parameter.

34.3 Use an exponential utility function to evaluate alternatives

34.4 Use sensitivity analysis to assess the decision sensitivity to the assessed risk tolerance parameter.
Lesson Objectives
Alternative 2:
a) win \$4000 with probability 0.3;
b) win \$000 with probability 0.3, or
c) win \$500 with probability 0.4,
Solution
"Desk" Problem
Risk Tolerance - R
Certain Equivalence

Topics:

Decision Framing

MODA
Math and insights

Influence Diagrams, Decision Trees, and Risk Profiles
Decision tree calculations with missing info that you fill in

Value of Information
Includes Bayes Law calculations

Risk Attitudes
Exponential utility function calculations
TEE Topics