How To Establish Sample Sizes For Process

Validation Using Statistical Tolerance Intervals

By Mark Durivage, ASQ Fellow

The first article in this series, Risk-

Based Approaches To Establishing

Sample Sizes For Process Validation

(June 2016), established the

relationship between risk and sample

tolerance limits, which use the confidence level (how sure we are) and reliability value

(population value) to determine appropriate statistically valid sample sizes for process

validation.

Start With FMEA

Before we begin, we must establish our definitions of risk and their associated confidence

level and reliability value. These definitions can and should vary based upon the

organizational needs. A good place to determine the risk level is failure mode and effects

analysis (FMEA), a systematic group of activities designed to recognize, document, and

evaluate the potential failure of a product or process and its effects. FMEA uses a risk

priority number (RPN), which is based on the frequency, detection, and severity of a

be selected using Table 3. Figure 1 depicts the linkage between FMEA, risk, and confidence

level and reliability.

Table 1: Example FMEA

Figure 1: Risk process for determining the appropriate confidence level and reliability

Table 2 shows an example of risk level definitions with accompanying defect

classifications. These definitions can and will vary based upon the product(s) produced

and its intended and unintended uses.

Table 2: Example of Risk Level Definitions

Table 3 depicts example confidence levels and reliability values based upon risk. Of course,

different confidence and reliability levels can and should be utilized based upon an

data is normally distributed. There are many ways to determine if data is normally

distributed, including computer programs and spreadsheets. However, for small samples

(15 or fewer) normal probability plots can be used to assess normality.

Normal probability plots can be constructed to look for linearity when using one variable.

Normal probability plots provide a visual way to determine if a distribution is

approximately normal. If the distribution is close to normal, the plotted points will lie close

to a line. Normal probability plots are constructed by doing the following:

1. Arrange the data from smallest to largest.

1. Plot each z-score against its corresponding data value.

1. Perform a graphical review of the normal probability plot for the data points to verify

However, it is generally best practice to use one-sided tolerance intervals — this method

will provide a more conservative approach, because the risk is placed on one side rather

than being split. If the specification is bilateral, use the specification that is closest to the

sample mean, which is calculated from the initial sample.

The first step in using statistical tolerance intervals to determine sample sizes for process

validation is to calculate the mean and standard deviation from a small sample, which

should capture the expected range of variation that can reasonably be expected from the

process (different batches of materials, operators, etc.). To determine this minimum initial

sample for a one-sided test:

And for a two-sided test:

number of samples to establish the sample mean and standard deviation.

Single-Sided Specification Example

A pouch-sealing operation is considered high risk according the FMEA in Table 1. High

risk requires 95% confidence (0.05α) and 99% reliability (0.01β), as shown in Table 3. The

validation team decided to use a δ of 1.0σ. Table 4 requires an initial sample of 16. The 16

samples were found to be normally distributed, with a mean of 3.2 lbs and sigma of 0.23.

The pouch-sealing process specification requires a minimum pull strength of 2.5 lbs.

For a single-sided specification:

The calculated k value is 3.043. (The k value selected should be equal to or less than the

calculated value.) High risk requires 95% confidence (0.05α) and 99% reliability (0.01β),

For a one-sided tolerance:

Since the calculated tolerance interval (3.037 lbs) is greater than the minimum pull

strength of 2.5 lbs, the validation passed. We can state that we are 95% confident that the

process is 99% reliable. In other words, we can claim that 95% of the time, 99% of the

products produced will exceed the minimum pull strength of 2.5 lbs.

Sample size does matter! Let Mark Durivage show you which method

for determining sample size is best for your process validation

efforts:

Risk-Based Approaches To Establishing Sample Sizes For

Process Validation

of 4.3. Filling process specification is 1000 g ±5 g.

For a double-sided specification:

The calculated k value is 2.326. (The k value selected should be equal to or less than the

calculated value.) Medium risk requires 95% confidence (0.05α) and 95% reliability

(0.05β), which Table 6 indicates a minimum of 80 samples. A minimum of 80 samples

must be used for the validation. Because the company typically uses three batches for

process validation, 27 samples will be randomly drawn from each of three batches for a

total of 81 samples.

Three validation lots were manufactured. The 81 samples were found to be normally with a

sample mean of 994 g and sample standard deviation of 4.2.

the process.

Choosing The Right Method For Your Process

Although the examples provided use single-sided and double-sided specifications, in

practice I prefer to use the single-sided method, to account for an off-center process (same

principle as C ). Remember, the criteria used should be based upon risk and should be

documented and proceduralized. When a validation fails, there are three distinct options

available:

intervals to establish sample sizes for process validation activities is a widely used and

accepted practice in FDA-regulated industries.

The methods presented here have been used and successfully defended during audits and

inspections. I cannot emphasize enough the importance of proceduralizing (documenting)

well as fully worked examples, like those demonstrated above, to provide clarity and

guidance for those individuals writing, performing, executing, and approving process

validation activities.

Subsequent articles in this series will provide additional how-to examples for applying

risk-based sample size techniques to process validations in your organization. Previous

articles include:

Theorem

in Lambertville, Michigan. Please feel free to email him at

[email protected] with any questions or comments, or

connect with him on LinkedIn.