DoE

Outline

Introduction

Some Basic Principles & Terminology

Basic Concepts in DoE

Activities for the Application DoE

Skills Required to Apply DoE

Gap between DoE and industries

Some Real Life Problems and DOE solutions

Introduction

Purpose of Experimentation

Comparing Alternatives

A common use is planning an experiment to gather data to make a decision between two or more alternatives

Identifying the Signifcant Inputs

(Factors) Afecting an Output (Response), selecting the few that matter from the many possible factors.

Achieving an Optimal Process Output

What are the necessary factors, and what are the levels of those factors, to achieve the optimum setting of the factors?

Reducing Variability

A multi-part strategy to reduce factor variation and make a factor more robust or fit to use, e.g., meet its performance requirements regardless of variation..

Introduction

Approaches of Experimentation

One Factor at a Time (OFAT)

One factor at a time method also known as one-variable-at-a-time (OVAT) or monothetic analysis is a method of designing experiments involving the testing of factors, or causes, one at a time instead of multiple factors simultaneously.

“Best-guess” experiments or Trial and Error (T&E)

A best-guess experiment is a fundamental method of problem-solving and used a lot. It is characterized by repeated, varied attempts which are continued until success, or until the practicer stops trying.

Design of Experimens

Design of experiments (DOE) is defined as a branch of applied statistics that deals with planning, conducting, analyzing, and interpreting controlled tests to evaluate the factors that control the value of a parameter or group of parameters.

Introduction

What's DoE?

Design of Experiments (DOE) is also referred to as Designed Experiments or Experimental Design, which is a powerful statistical technique for improving product/process designs and solving process production problems

A series of tests conducted in a systematic manner to further the understanding of an existing process or to explore a new process or product.

Design of Experiment (DOE) is a powerful statistical technique for improving product/process designs and solving process production problems

A tool to develop an experimentation strategy that maximizes learning using a minimum of resources.

A methodology for systematically applying statistics to experimentation. Different from OFAT widely practiced

Some Basic Principles & Terminology

Factors, Levels, Responses

Factors

Factors are inputs to the process Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are Flour, Eggs, Sugar and Oven. Potential factors can be categorized using the Cause & Effect Diagram

Levels

Levels represent settings of each factor in the study Examples include the oven temperature setting, no. of spoons of sugar, no. of cups of flour, and no. of eggs

Response

Response is output of the experiment In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels.

DoE Terminology

• Sometimes factors do not behave the same when they are looked at together as when they are alone; this is called an interaction
• Interaction plots are most often used to visualize interactions during DOE through ANOVA
• Interaction plot can be used to visualize possible interactions between two ormore factors
• Parallel lines in an interaction plot indicate no interaction
• The greater the difference in slope between the lines, the higher the degree of interaction
• However, the interaction plot doesn't alert you if the interaction is statistically significant

Interaction

• Running the trials in an experiment in random order
• Notion of balancing out effects of “lurking” variables
• Randomization is a statistical tool used to minimize potential uncontrollable biases in the experiment by randomly assigning material, people, order that experimental trials are conducted, or any other factor not under the control of the experimenter
• Randomization of the run order is needed to minimize the impact of those variables outside of the experiment that we are not studying.

Randomization

• Replication is making multiple experimental runs for each experiment combination. This is one approach to determining the common cause variation in the process so that we can test effects for statistical significance.
• Replication of a basic experiment without changing any factor settings, allows the experimenter to estimate the experimental error (noise) in the system used to determine whether observed differences in the data are “real” or “just noise”, allows the experimenter to obtain more statistical power

Replication

Basic Concepts in DoE

Factors

Factors are inputs to the process Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are Flour, Eggs, Sugar and Oven. Potential factors can be categorized using the Cause & Effect Diagram

Levels

Levels represent settings of each factor in the study Examples include the oven temperature setting, no. of spoons of sugar, no. of cups of flour, and no. of eggs

Response

Response is output of the experiment In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels.

DoE Benefits

DoE requires less resources (experiments, time, material, etc.) for the amount of information obtained.

The estimates of the effect of each factors (variable) on the response are more precise

The interactions between factors can be estimated systematically (Interactions are not estimable with OFAT experiments)

There is experimental information in a larger region of the factor space.

Common Uses

Comparison

This is one factor among multiple comparisons to select the best option that uses t‒-test, Z-‒test, or F-‒test.

Variable screening

There are usually two-level factorial designs intended to select important factors (variables) among many that affect performances of a system, process, or product.

Transfer function identification

If important input variables are identified, the relationship between the input variables and output variable can be used for further performance exploration of the system, process or product via transfer function.

System Optimization

The transfer function can be used for optimization by moving the experiment to optimum setting of the variables. On this way performances of the system, process or product can be improved.

Robust design

deals with reduction of variation in the system, process or product without elimination of its causes.

Factorial Design

One Factorial Design

Experiments with only one factor are often called simple comparative experiments. In these cases, t‒-test or ANOVA were used for analysis.

k

Full Factorial designs at two level (2 )

2

3

A full factorial design is convenient for a low number of factors if the resources are available. Conceptual approach for DOE is explained for two 2 and three 2 factors as well as general 2 factorial design, in which k represents number of factors while number 2 represents number of levels.

k

Basic Concepts in DoE

Response Surface Methodology (RSM)

Factors

Factors are inputs to the process Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are Flour, Eggs, Sugar and Oven. Potential factors can be categorized using the Cause & Effect Diagram

Levels

Levels represent settings of each factor in the study Examples include the oven temperature setting, no. of spoons of sugar, no. of cups of flour, and no. of eggs

Response

Response is output of the experiment In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels.

Subject Matter Knowledge

With RSM, a series of full factorial experiments and map the response are conducted to generate mathematical equations that describe how factors affect the response.

Factors

Central Composite Design (CCD)

DoE

• Region of Operability
• Region of Interest

Experimentation

A CCD consisits of 3 parts:

Factorial points

Center points

Axial (star) points

Responses

A CCD is often executed by adding points to factorial points (highly efficient, but beware of blocking!).

Emprical Model

(Correlation)

Properties:

• efficient
• rotatable
• corner points

ANOVA

Box-Behnken Design (BBD)

A BB consisits of 3 parts:

Statistical Assessment

3D surface, 2D Contour, & Interaction plots

Factorial points (at the midpoints of the edges)

Center points

Properties:

• efficient (few runs)
• (almost) rotatable
• No corner points

Optimization

Activities for the DoE application

Factors

Factors are inputs to the process Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are Flour, Eggs, Sugar and Oven. Potential factors can be categorized using the Cause & Effect Diagram

Levels

Levels represent settings of each factor in the study Examples include the oven temperature setting, no. of spoons of sugar, no. of cups of flour, and no. of eggs

Response

Response is output of the experiment In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels.

Measure

Define

Pre-Analyze

Activities

Brief Explanation

Tools

Every variable that can affect the response must be listed in this preliminary stage

Identify factors

Fishbone

Classify factors

The identified factors must be classified in primary ones that will be considered for experimentation, those which will be kept constant and those nuisance factors which hinder experimentation

Affinity Diagram

The ranges for the quantitative factors must be defined as well as the number of possible levels for each qualitative factor

Choose ranges and levels

Brainstorming

Based on previous knowledge, they must predict the effect of factors and its interaction on response, before selecting the experimental design

State actual process knowledge

Tools

Activities

Brief Explanation

Important characteristics of the factors must be listed in order to make a correct selection of the design

Characterize the factors

Software aided DoE

Define characteristics needed for the design

The characteristics desired for the design must be defined for choosing the design

A useful design must be chosen, suitable with the previous listed characteristics

Choose experimental design

The design selected establishes the number of levels for each factor. So values, belonging to the range pre-established, must be defined for each coded levels

Select levels

Activities

Breif Explanation

Tools

Experimentation is a team process, so strategies and tools must be used for the best selection of the working team

Formulate problem

Fishbone

State relevant background

Information from literature review and previous experimentations must be documented as any information that may be useful for the project

Process Mapping

Choose response

The variable(s) that will measure the result of the Experimentation is called the response.

Flow Chart

Once response is chosen, a measurable objective must be set with a deadline to achieve it

State objective

Experiment

Analyze

Activities

Tools

Activities

All possible factors effects must be calculated. These effects include interactions and second order effects if necessary

Determine factors effects

RSM

Activities

Brief Explanation

Tools

ANOVA

ANOVA analysis or probability plots must determine which effects are statistically significant

Determine significant effects

Regression

Optimization

Computational Experimentation

Expeirmental Experimentation

Outline experiment

Special care must be taken for planning and arranging the experimentation (materials sources, machine scheduling, lab request suppliers, ....etc)

If it is required, once the model is obtained, response can be optimized throughout the studied region

2D Contour Plots

3D Surface Plots

Evaluate new experiments

The possibility to carry out more experiments must be taken into account

Evaluate trial runs

Before experimentation is carried out, it is recommended to make some trial runs. They permit one to check experimental error and assumptions made in previous steps

Perform the experiment and recollect data

The experiment must be performed carefully as planned and data must be collected for further analysis

Control

Improve

Activities

Tools

Brief Explanation

Procedures & Standardization

A control plan must be set, in order to obtain the benefit proposed from the last activity

Implement controls

Draw conclusions and do recommendations

A specific procedure must be established in order to achieve whether the expected benefits of the experimentations were obtained or not

Control Plan Development

Implement new conditions

Experimentation is an iterative inductive deduction learning process, so once the whole cycle is finished, new experimentation must be evaluated

Activities

Tools

Computational Experimentation

Expeirmental Experimentation

Once a new condition is presented as the solution, it is convenient to make some confirmatory test to validate the result obtained from experimentation in those values

Confirming testing

Draw conclusions and do recommendations

Once confirmation of the experimentation is obtained, conclusions and recommendation must be elaborated. Graphics are stimulated because they are the best way to present results

Once response is chosen, a measurable objective must be set with a deadline to achieve it

Implement new conditions

Activities

Tools

Experimentation is a team process, so strategies and tools must be used for the best selection of the working team

Formulate problem

Fishbone

State relevant background

Information from experience, previous projects and control charts must be document as any information that may be useful for the project

Process Mapping

Choose response

The variable(s) that will measure the result of the process is called the response. This should be continuous, precise and related to the client’s perception of quality

Flow Chart

State objective

Once response is chosen, a measurable objective must be set with a deadline to achieve it

Activities

Tools

Every variable that can affect the response must be listed in this preliminary stage

Identify factors

Fishbone

Classify factors

The identified factors must be classified in primary ones that will be considered for experimentation, those which will be kept constant and those nuisance factors which hinder experimentation

Affinity Diagram

The ranges for the quantitative factors must be defined as well as the number of possible levels for each qualitative factor

Choose ranges and levels

Brainstorming

Based on previous knowledge, they must predict the effect of factors and its interaction on response, before selecting the experimental design

State actual process knowledge

Activities

Tools

Important characteristics of the factors must be listed in order to make a correct selection of the design

Characterize the factors

Software aided DoE

Define characteristics needed for the design

The characteristics desired for the design must be defined for choosing the design

A useful design must be chosen, suitable with the previous listed characteristics

Choose experimental design

The design selected establishes the number of levels for each factor. So values, belonging to the range pre-established, must be defined for each coded levels

Select levels

Activities

Tools

Expeirmental Experimentation

Computational Experimentation

Outline experiment

It’s easy to underestimate the logistical and planning aspects of the experiment. Consequently, special care must be taken for arranging the experimentation

State relevant background

Information from experience, previous projects and control charts must be document as any information that may be useful for the project

Choose response

The variable(s) that will measure the result of the process is called the response. This should be continuous, precise and related to the client’s perception of quality

Analyze

Activities

Tools

All possible factors effects must be calculated. These effects include interactions and second order effects if necessary

Determine factors effects

RSM

ANOVA

ANOVA analysis or probability plots must determine which effects are statistically significant

Determine significant effects

Regression

Optimization

If it is required, once the model is obtained, response can be optimized throughout the studied region

2D Contour Plots

3D Surface Plots

Evaluate new experiments

The possibility to carry out more experiments must be taken into account

Activities

Tools

Expeirmental Experimentation

Computational Experimentation

Once new condition are presented as the solution, it is convenient to make some confirmatory test to validate the result obtained from experimentation in those values

Confirming testing

Draw conclusions and do recommendations

Once confirmation of the experimentation is obtained, conclusions and recommendation must be elaborated. Graphics are stimulated because they are the best way to present results

Once response is chosen, a measurable objective must be set with a deadline to achieve it

Implement new conditions

Activities

Brief Explanation

Tools

Procedures & Standardization

A control plan must be set, in order to obtain the benefit proposed from the last activity

Implement controls

A specific procedure must be established in order to ascertain whether the expected benefits of the experimentations were obtained or not

Draw conclusions and do recommendations

Control Plan Development

Implement new conditions

Experimentation is an iterative inductive deduction learning process, so once the whole cycle is finished, new experimentation must be evaluated

Portfolio Gallery

Display a gallery of your best works

Image #3

Image #1

Image #2

Image #4

Add a short description of the

image or project here.

Image #7

Image #5

Image #6

Image #8

Add a short description of the

image or project here.

Single Portfolio Item

Show more details of a single project

Project Details

"This is example dummy text. Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed dodes eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in repre henderitas in voluptate."

Skills Required to Apply DoE

Factors

Factors are inputs to the process Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are Flour, Eggs, Sugar and Oven. Potential factors can be categorized using the Cause & Effect Diagram

Levels

Levels represent settings of each factor in the study Examples include the oven temperature setting, no. of spoons of sugar, no. of cups of flour, and no. of eggs

Response

Response is output of the experiment In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels.

For the successful application of DoE in the industry, we generally require the following skills:

Understanding appropriate fundamental technical skills of statistics and experimental design.

Statistical skills

Learning to use a specific statistical computer package.

Understanding how to use statistical thinking effectively

Planning Skills

1

Understanding an experimentation significance

Identifying all sources of variation

2

Defining the objectives of the experiment

3

Understanding a particular problem

Teamwork Skills

2

4

Understanding time and experimental budget

5

Determining how many people are involved with the experimentation and establishing who is doing what

Engineering skills

3

Engineering skills

1

Teamwork Skills

Determination of the number of levels of each factor and the range at which each factor can be varied,

1

Sharing understanding of the experimental goals

2

Determination of what to measure within the experiment

Statistical skills

4

2

Better communication among people with different skills

3

Determination of the capability of the measurement system in place

3

Better communication among people with different skills

4

Determination of what factors can be controlled and what cannot be

Gap between DoE and industries

Most of the papers were published by researchers in the USA (26.7%), followed India (10%), the UK (10%), Taiwan (6.7%) and Canada (6.7%). Each of the remaining countries represented 3.3% of the total.

39% of industries follow Best Guest strategies, while 41% of industries conduct experimentation using the OFAT strategy. Furthermore, only 20% of industries carry out experimentation with a pre-established statistical methodology.

Statistical

Statistical Designed

Experimentations

(DoE)

Skills

Best Guess & OFAT

Experimentations

Most used DoE strategy was the classical (51.6%), followed by the Taguchi method (35.5%), screening method (6.5%), RSM (3.2%), and a new method (3.2%)

17 articles were published in the last 10 years, three papers were published in the 90’s, and four studies were published between 2000 and 2010.

https://www.emerald.com/insight/1754-2731.htm

Some Real Life Problems and DOE solutions

Factors

Factors are inputs to the process Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are Flour, Eggs, Sugar and Oven. Potential factors can be categorized using the Cause & Effect Diagram

Levels

Levels represent settings of each factor in the study Examples include the oven temperature setting, no. of spoons of sugar, no. of cups of flour, and no. of eggs

Response

Response is output of the experiment In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels.

IBM (US)

Motorola (US)

Nabisco (US)

Recommendations

Semiconductor fails customer specs

Excess scrap, cookie dough, and out-of-roundness conditions

High residue levels in a surface-mount process

28 test responses were optimized revealing a ‘sweet spot’ where all specs are met with minimal cost.

Process factors settings were optimized and established to reduce flux and thin residue

Brainstormed likely problem factors and found vital few factors from many insignificant ones using factorial design. The result saves 16 pounds of dough and eliminates out-of-roundness conditions

Separated Design Elements:

Customize

the Layout

Recommendations

Factors

Factors are inputs to the process Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are Flour, Eggs, Sugar and Oven. Potential factors can be categorized using the Cause & Effect Diagram

Levels

Levels represent settings of each factor in the study Examples include the oven temperature setting, no. of spoons of sugar, no. of cups of flour, and no. of eggs

Response

Response is output of the experiment In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels.

DOE is an effective and powerful tool used by industries and researchers improve, discover, solve problems and save time and cost of experimentation.

The literature search confirmed that DoE is hardly used; however, this technique is necessary for the experimentation frequently carried out within industries.

The trend of use of DOE is rapidly growing and it is expected to expand over new scientific and industrial areas and have rapid growth there.