V3 Risk-driven design: A review of Pew & Mavor (2007)

Starts with Boehm's Spiral Model

• Why you start with KLM to look at built systems: used to be given system then
• Can look for missing methods
• Many HCI projects are not in a small box, but cut across boxes/phases,
• Many are constraints on things in the boxes, or ways to view the boxes, or ways to share results across boxes
• Can choose method based on risk if you care about risk, or project if you care about method!
• Some work ignores this flow, and ends up being narrow, undefined wrt to process, and further from designers

Risk-Driven Design

frank.ritter@psu.edu

HCIC Workshop, 15 June 2011

Ritter's Conclusions and Final Insights

Further Insights

* A way to describe the ways of knowing in HCI

-- and where and why

* It provides insights into how to apply HCI, missing aspects of HCI, and insights into ways of knowing

Insight: Impact on next project

--Size of users tasks, complexity of tasks, their interrelation, scope

--May be true for all methods

--So shared to next design, and understanding of designer

Insight: Designers think they are already risk driven

--Good, buy-in to part

--Bad, already know how

--Insight: need to give designers counter examples

Insight: Education and sharable representations are more important than one might think

Small Example1

Shared Representations as Part of Design Process - Uses

Small Example 2

• Examined critically
• Reduce working memory load
• Make explicit what is explicit and implicit
• Produce new connections
• Collaboratively produce new knowledge
• Transfer knowledge

(e.g., see Baumer & Silberman, 2011)

Goals of this tutorial

What does this mean for HCI and HCIC?

The Risk Management Process: Handling Options

• Provide an organizing framework for HSI/HCI/HF:

-- Ways of knowing

-- Teaching materials now on web for free

• Show/Learn how to leverage the results of the report

-- Teach and be taught about system design

-- Provide you with tools to argue for better design

-- by reducing risk

• Discuss the application of user models that it represents

• In offering it I thought I would not be co-chair!
• Play with Prezzi.com

*

**

Problems with (Future) Systems of Systems Development

All HCI techniques can be seen as a way to reduce risk

For each stage, HCI techniques to reduce risk:

• Define opportunities and context of use:scenarios, personas, task analysis

• Define requirements and design solutions:TA, models

• Evaluate:VPA, behavior loggers (e.g., RUI)

• Insert your favorite method

Shared representations are passed between these stages

• Lack of commitment by funders, managers to avoid HSI risks
• Lack of communication between system engineers and human-system experts
• Difficulties providing data about humans into the design process
• Thus, the study/literature survey at beginning of book (also see Booher & Minniger, 2003)

Risk-Driven Design

a review of Pew & Mavor (2007)

Example risks

Pew, R. W., & Mavor, A. S. (Eds.). (2007). Human-system integration in the system development process: A new look. Washington, DC: National Academy Press. http://books.nap.edu/catalog.php?record_id=11893.

• Offers a New look at HCI/HSI - It is risk-driven process
• Offers insights into HCI/HSI organising principles

- - A way of knowing

- - How to argue HCI

- - When to shut up

• 17% of my sabbatical
• Useful for teaching PSU (Stark & Kokini, 2010)

• Can't manufacture
• Can't deliver
• Performance does not match other stake holder requirements
• Wrong types of developers and HIS professionals

• Performance does not satisfy user requirements
• Mismatch of system to context (sand in tools)
• See Booher and Minniger (2003), Casey (1988), etc.

Implications for System Design

Essentials of the Spiral Model

Boehm & Hansen (2001)

Pew and Mavor (2007) Charged to:

• Concurrent development of key artifacts
• Each cycle does Objectives, Constraints, Alternatives, Risks, Review, and Commitment to Proceed
• Level of effort driven by risk
• Degree of detail driven by risk
• Use anchor point milestones
• Emphasis on system and life cycle activities and artifacts

Work with a panel to

• Comprehensively review issues
• Evaluate state of the art in HSI (and engineering)
• Develop a vision
• Recommend a research plan

Life cycle phases

Phase steps

• Evaluate alternatives with risk analysis & prototype
• Develop/verify
• Plan/architect
• Review [with stakeholders]
• Cost

• Exploration
• Valuation
• Architecting
• Development
• Operation

A way to view system design

Comparable to waterfall (seehttp://www.waterfall2006.com/)

• other approaches in book

Risks related to humans (users) are often ignored by system engineers

• People naturally work on risks, so theory is not just normative but descriptive
• Risks related to hardware are ignored by HF professionals
• Can/could/should bring in experts to advise

If no HCI risks, then nothing needed from HCI

• See recommendations in book
• Other recommendations?

Can be used to classify HCI ways of knowing:

Methods for Reducing HSI Risks

Three major periods of use

• Define context of use
• Define requirements and design solutions
• Evaluate

All fit back into spiral

All used to reduce risks using previous approaches

We have bags of these methods!

Classification to period is somewhat arbitrary

Not exhaustive, illustrative lists to follow

References

Acknowledgements

• Barry Boehm
• Jeremy Lothian
• Dick Pew
• Anne Mavor
• Erika Poole
• ACS Lab
• guy on plane
• ONR N091-086/P10008, N00014-11-1-0275, N00014-06-1-0164
• The committee

Baumer, E. P. S., & Silberman, M. S. (2011). When the implication is not to design (technology). In Proceedings of CHI, 2011, 2271-2274. ACM: New York, NY.

Boehm, B., & Hansen, W. (2001). The Spiral Model as a tool for evolutionary acquisition. Crosstalk: The Journal of Defense Software Engineering, 14(5), 4-11.

Booher, H. R., & Minninger, J. (2003). Human systems integration in Army systems acquisition. In H. R. Booher (Ed.), Handbook of human systems integration (pp. 663-698). Hoboken, NJ: John Wiley.

Casey, S. M. (1998). Set phasers on stun: And other true tales of design, technology, and human error. Santa Barbara, CA: Aegean.

Pew, R. W., & Mavor, A. S. (Eds.). (2007). Human-system integration in the system development process: A new look. Washington, DC: National Academy Press. http://books.nap.edu/catalog.php?record_id=11893.

Stark, R. F., & Kokini, C. (2010). Reducing risk in system design through human-systems integration. Ergonomics in Design, 18(2), 18-22.