Failure amplification method: an information maximization approach to categorical response...

1. GENERAL

The key idea in this article is the use of amplification factor(s) to manipulate the failure probabilities to improve the efficiency of designed experiments with binary data. The notion of amplifying or accelerating failure to improve efficiency is well known in the reliability

literature. However, there has been relatively little discussion of using it with experimental design where the goal is to study the effects of design factors (Nair, Escobar, and Hamada 2003). Joseph and Wu are to be congratulated for developing this idea in the context of binary data and for using real applications to motivate the problem. Wang (2002) investigated a similar use of acceleration factors for designed experiments with time-to-failure data (reliability improvement studies). Section 2 of this discussion elaborates on the connections between traditional accelerated life testing in reliability and the problem studied here.

As the authors note at the beginning of Section 4, the two most important questions faced by practitioners in using the methodology are: How much amplification should be applied?, and how to achieve this level in practice? These questions seem to be addressed only perfunctorily in the article, and they remain largely unresolved.

The recommendation in Section 4.1 for amplifying to the LD50 point of the distribution and the corresponding designs in Sections 4.2 deal with the case of a location parameter and known scale parameter. But the focus of the article is on designed experiments with location and dispersion effects, so practitioners need information on what to do in these situations. The optimality of the LD50 level seems to be limited to only a few cases of interest. For instance, even with location models, estimation of the unknown scale parameter requires conducting the experiment at two levels of the amplification factor. As discussed in Section 3, results on optimum amplification levels for general situations are complicated and can be difficult to implement in practice. In any case, a systematic treatment of the problem needs a precise specification of the parameters being estimated and the optimality criteria for various situations of interest. We consider some of these issues in Section 3, but a comprehensive development will require additional research.

As is usual in discussions, our comments focus on areas where more work is needed. This should not, however, detract from the importance of the contributions in the article.

2. CONNECTIONS TO ACCELERATED TESTING IN RELIABILITY

Accelerated stress testing has been used extensively in reliability applications to induce early failures and thus improve the efficiency of reliability studies. Although this has been done primarily in the context of time-to-failure data, there are some quite close connections to the problem being studied here, on which we elaborate.

We begin with the commonly used accelerated failure time (AFT) model. Let Y(s) be the log of the time to failure at acceleration level s. The AFT model assumes that the distribution of Y(s) follows a location-scale model with cdf F([y - [mu](s)]/[sigma]); that is, the location parameter [mu](s) depends on the acceleration factor s, whereas the scale parameter is constant. It is also often assumed that after an appropriate transformation of the acceleration factor, [mu](s) varies linearly with s. This is the case with, for example, the well-known Arrhenius model for temperature acceleration and the power law model for voltage (see Meeker and Escobar 1998). Suppose that s is already in this transformed scale [e.g., s = log(M) in the present paper]. Then we have the model

Y(s) = [[alpha].sub.0] + [[alpha].sub.1]s + [sigma][epsilon], (1)

and the distribution of Y(s) is

F([y - [[alpha].sub.0] - [[alpha].sub.1]s]/[sigma]). (2)

Common choices for the baseline distribution F are standard normal or smallest extreme value (or lognormal and Weibull in the original scale).

Figure 1(a) shows how the lifetime distribution changes as the acceleration level, s, varies for an underlying symmetric location model. In accelerated life tests (ALTs), the study is usually censored at some fixed time C (type I censoring). One observes the lifetime T if the device fails before C; if not, then one knows only that the unit survived past C.

The goals of ALTs include estimation of the unknown parameters [[alpha].sub.0], [[alpha].sub.1], and [sigma] or estimation of quantities (i.e., percentiles, quantiles, hazard rates) associated with the distribution at the operating condition [s.sub.0], that is, F([y - [mu]([s.sub.0])]/[sigma]). Although the optimal plans depend on the estimation problem of interest, they call for testing at two points, one point at the extreme of the high acceleration level (subject to caveats about the underlying model holding). This minimizes the expected number of censored observations and thereby maximizes efficiency. There is an extensive literature on optimal and more robust compromise test plans for ALTs (see Meeker and Escobar 1998 and references therein).

Suppose now that we are still dealing with time to failure but observe only I|T [less than or equal to] C| if T [less than or equal to] C instead of the actual time to failure. This is the case with, for example, inspection data where we know only whether the unit has failed or not but not the actual time of failure. Then we are exactly in the binary data situation considered by Joseph and Wu but without the design factors. The failure probability of the binary data is given by

p(s) = F([C - [[alpha].sub.0] - [[alpha].sub.1]s]/[sigma]). (3)

When there is no connection to lifetime data, the distributions in Figure 1(a) can still be viewed as those of the underlying latent variable discussed by Joseph and Wu and C as the threshold parameter. In this case the threshold value C is unknown, and one can estimate only C - [[alpha].sub.0] and not both of them separately. In the rest of this discussion, with an obvious abuse of notation, we let [[alpha].sub.0] denote C - [[alpha].sub.0]. Note that the model in (3) is more general than that of Joseph and Wu, because it involves an unknown slope parameter [[alpha].sub.1]. This parameter arises naturally in the Arrhenius and power law models.

The new contribution in the article is the use of acceleration or amplification in conjunction with designed experiments to efficiently estimate the effects of design factors. This is depicted in Figures 1(b) and 1(c), which show how the distributions vary as a function of a single design factor, say A, at two levels and the acceleration factor. Figure 1(b) corresponds to a location model in which the underlying probabilities of interest are given by

[FIGURE 1 OMITTED]

p(s, i) = F([[[alpha].sub.0] - [[alpha].sub.1]s - [[alpha].sub.A][x.sub.i]]/[sigma]), i = 1, 2. (4)

Specifically, we have assumed that the design factor has only a location effect [[alpha].sub.A] and no dispersion effect; [x.sub.i] takes on values [+ or -]1 corresponding to the low and high levels of A. The treatment in Joseph and Wu assumes that [[alpha].sub.1] is known (= 1).

For this case and with [sigma] known, it is clear that one can estimate both [[alpha].sub.0] and [[alpha].sub.A] by testing at just one level of the acceleration factor s. The discussion in Section 4.1 of Joseph and Wu considers estimation of only [[alpha].sub.0], whereas the primary interest is presumably in [[alpha].sub.A] or at least in both parameters. Our discussion in Section 3 shows that the LD50 point (s = [[alpha].sub.0]) remains optimal for estimating either [[alpha].sub.A] alone or both [[alpha].sub.0] and [[alpha].sub.A]. However, the LD50 point now refers to the baseline distribution without design effects, that is, F([[[alpha].sub.0] - s]/[sigma]). This conclusion is also intuitively clear from of Figure 1(b), because this choice of s provides maximal separation of the failure probabilities at the low and high levels of A. We note, however, that this result seems to be true for two-level experiments only. Preliminary calculations suggest that the result does not extend to situations with more than two levels. The optimal acceleration still appears to be the LD50 of some baseline distribution, but it does not correspond to the center of the design space.

The problem formulation of Joseph and Wu is rather general, and the failure probabilities depend on both location and dispersion effects of the control and noise factors,

F([[[alpha].sub.0] - s - [mu](X,N)]/[sigma](X,N)). (5)

In this case the optimum amplification levels are complicated functions of the unknown parameters and the underlying distribution. To see this, consider a very simple version with a single factor A that has only a dispersion effect, that is,

p(s,i) = F([[[alpha].sub.0] - s]/[exp([[phi].sub.1] + [[phi].sub.A][x.sub.i])]), i = 1, 2, (6)

where again [x.sub.i] = [+ or -]1 at the low and high levels of A. Figure 1(c) shows how the distributions vary as a function of the acceleration factor. The LD50 point of s = [[alpha].sub.0] is now the worst possible amplification level. It leads to a 50% failure probability for both levels of A, so there is no way to separate the two distributions. As discussed in Section 3, there are two different optimal levels of amplification, and these values depend on the underlying distribution and the unknown values of the [phi]'s. Clearly, the picture will get much more complicated if we also consider location effects and several factors.

Finally, an implicit but important assumption in the article is that the design factors do not interact with the amplification factor. Figure 2 shows two different situations where there is a single design factor that has only a location effect but interacts with the acceleration level. For Figure 2(a), the optimal strategy is to decrease the acceleration factor as low as possible (subject to other constraints) while for whereas (b), the acceleration factor should be made as large as possible. The situation will be even more complicated if the two lines intersect. The optimum acceleration (or deceleration as the case may be) depends on the unknown interactions, the allowable range of the acceleration factor, and the validity of the model within this range.

[FIGURE 2 OMITTED]

3. OPTIMALITY CRITERIA AND ACCELERATION LEVELS FOR MAXIMIZING INFORMATION

As previously noted, a systematic investigation of the optimal amplification levels requires specification of the estimation problem of interest and the optimality criteria (Fig. 3). Here we consider (briefly) several situations and criteria of interest.

Consider first the simple model in (3) with no design factors, [[alpha].sub.1] = 1, and known scale parameter [sigma]. Note that C - [[alpha].sub.0] now corresponds to [[alpha].sub.0]; also, our [[alpha].sub.0] here is the same as log(u) in equation (35) in the article. Suppose that the goal is to estimate [[alpha].sub.0] with minimum variance. The variance of [^.[alpha].sub.0] is [f.sup.2]([eta])/[F([eta])(1 - F([eta]))]. This is maximized at [eta] = 0 (or taking s = [[alpha].sub.0]) for the normal, logistic, and several other symmetric distributions. As noted in the article, [eta] = 0 corresponds to the choice of s = [[alpha].sub.0], the LD50 point, which is unknown and must be estimated. But because this is the original estimation problem that we started with, not much is to be gained from using acceleration in this case.

The primary problem of interest is estimation of the design effects, not estimation of [[alpha].sub.0]. Consider the simple model in (4) with [[alpha].sub.1] = 1 and with a single design factor A with only a location effect [Figs. 1(b) and 1(c)]. Assume, as is done in the article, that [sigma] is known. It is sufficient to conduct the experiment at a single acceleration level of s to estimate both parameters. Further, it is reasonable to allocate the same sample size to settings at the low and high levels of A. Then the information matrix for estimating [[alpha].sub.0] and [[alpha].sub.A] is

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII].

where [[eta].sub.1] = ([[alpha].sub.0] - s - [[alpha].sub.A])/[sigma], [[eta].sub.2] = ([[alpha].sub.0] - s + [[alpha].sub.A])/[sigma], and [phi] is density of standard normal variable. We may be interested in estimating just the design effect [[alpha].sub.A] or both [[alpha].sub.0] and [[alpha].sub.A]. To optimize the latter, consider the D-optimality criterion, which requires maximizing the determinant of the information matrix. It can be shown that both of these are optimized by taking s = [[alpha].sub.0], the same conclusion as before. However, this LD50 point now corresponds to the distribution at the center of the design space [at x = 0 in (4) above]. It appears that this conclusion remains true even with several design factors as long as they are all at two levels and have only location effects. Some preliminary calculations suggest that this is not true if there are factors with more than two levels. Note also that selecting s to ensure that the average failure probability is about 50% (which seems to be the recommendation in the article) can be different from setting s equal to the LD50 point at the center of the design space when the factors have more than two levels.

[FIGURE 3 OMITTED]

So far, the (constant) scale parameter [sigma] is assumed to be known. What happens if this is not the case? With an experiment based at a single acceleration level, one can estimate only the ratios [[alpha].sub.0]/[sigma] and [[alpha].sub.A]/[sigma]. This may be fine if one is interested in identifying only whether factor A has an important effect (with several factors, which ones are active) instead of estimating the actual magnitude [[alpha].sub.A]. For this case, it appears (although we have not checked the details carefully) that the choice of s = LD50 point of the distribution at the center of the design space remains optimal. If we are interested in estimating [sigma] also, then, as the authors note, the experiment must be conducted at least at two levels of the acceleration factor s. We suspect that the recommendations in the article and by Sitter and Wu (1993) remain optimal, although now the results depend on the underlying distribution.

As noted in Section 2, the experiment can have several different goals. Suppose that one wants to determine the appropriate design factor setting and then estimate the failure probability at some appropriate acceleration level [s.sub.0] (design stress condition). For example, consider only one design factor, A, with the high level being the best. Then the goal may be to estimate

p([s.sub.0],[x.sub.0]) = F([[[alpha].sub.0] - [s.sub.0] - [[alpha].sub.A]]/[sigma]) (7)

efficiently. In this case, we need an estimate of [sigma]. Moreover, the optimal setting of s for estimating the foregoing parameter would not be the same as that (those) for estimating the parameters [[alpha].sub.0], [[alpha].sub.A], and [sigma], so a compromise is needed. An appropriate choice of optimality criteria would ensure this trade-off.

Let us now turn to estimation of dispersion effects. Consider the situation with a single design factor A with a dispersion effect, that is,

p(s,i) = F([[[alpha].sub.0] - s]/[exp([[phi].sub.0] + [[phi].sub.A][x.sub.i])]), i = 1, 2. (8)

For simplicity of exposition, assume that [[alpha].sub.0] is known. Again, a single acceleration level is sufficient for estimating the two parameters [[phi].sub.0] and [[phi].sub.A]. The information matrix is given by

[MATHEMATICAL EXPRESSION NOT REPRODUCIBLE IN ASCII],

where [[eta].sub.1] = ([[alpha].sub.0] - s)/exp([[phi].sub.0] - [[phi].sub.A]) and [[eta].sub.2] = ([[alpha].sub.0] - s)/exp([[phi].sub.0] + [[phi].sub.A]). We can consider either minimizing the variance of [^.[phi].sub.A] or maximizing the determinant of the information matrix. We will consider the latter. As shown in Figure 3, there are two values of s that yield the global optimum, and these values depend on [[phi].sub.A], [[phi].sub.0], and the underlying distribution. Note that the choice of LD50 is the worst possible choice here, because the failure probabilities equal 1/2 at both the high and low levels of A.

As we begin to see, the optimum amplification levels for the general location and dispersion model considered in the article will depend on the unknown parameters and underlying distributions in a complicated way. This is true even for the restricted model with [[alpha].sub.1] known. Many of the commonly used acceleration models involve the additional parameter [[alpha].sub.1]. For most estimation problems, we also have to estimate the scale parameter(s). This will require at least two levels of acceleration. Also, the results can be sensitive to the choice of optimality criteria, such as A or D optimality.

Similar problems arise with optimal test plans for ALTs in reliability and in our investigation of ALTs with designed experiments (Wang 2002). One needs preliminary estimates of the unknown parameters to select the optimal designs. An alternative in these situations is to use a Bayesian approach that incorporates prior information about the unknown parameters. The sequential designs discussed in the article appear to be aimed at identifying the LD50 point. As we see, this is appropriate only in restricted situations. A different formulation is needed for general situations. One compromise is a two-stage design in which the first-stage experiment is used to obtain initial estimates, and these estimates are then used to determine (approximately) optimal design in the second stage. Appropriate choice of sample sizes between the two stages then becomes an interesting question.

Of course, one must be very cautious about using optimality results that are based on unverifiable model assumptions, especially with binary data. Within the reliability literature, the generally accepted approach has been to strive for compromise test plans that achieve a reasonable trade-off between efficiency and robustness/sensitivity to assumptions (see Meeker and Escobar 1998 and references therein). Investigation of similar plans for the present situation is a worthwhile research project.

4. MISCELLANEOUS

A considerable part of the article focuses on loss functions and associated two-stage optimization. Although these results are interesting from a technical viewpoint, they are applicable only to the specific loss functions considered here. In our experience, it is rare that engineers have precise information on loss functions, especially in situations in which there are multiple objectives and decision makers.

As the authors note, there are inherent difficulties in trying to estimate location and dispersion effects with binary data. There can be serious estimability problems in dealing with the general location and dispersion models considered in the article, especially if there are several control and noise factors.

In the response modeling approach to robust design, the variance is generally assumed to be constant or to at most depend on the control factors only, that is, [sigma](X). The rationale for this is that the effect of the observed noise factors N are captured through the control-by-noise interactions in the location model. This article postulates a much more general setup with [sigma](X, N), where the scale parameter depends on both the control factors and noise factors. It is not clear how to interpret this model and how to characterize the overall dispersion effects of the control factors.

It is worth reiterating the authors' point that in experiments with a control factor as the amplification factor and only location effects for the design factors, there should be at least two failure modes, and their effects should be in opposite directions. With a single failure mode, one can simply set the design factors at convenient settings (to minimize cost, for example) and then select the level of the control factor to optimize the failure probability. There is no need for experimentation in this case. However, in our experience there are many applications in which the amplification factor is an environmental variable, such as temperature, voltage, or relative humidity. (These are probably "noise" factors in the authors' taxonomy.) The notion of using amplification with designed experiments will be useful in these cases even for a single failure mode.

Finally, the term "amplification" seems to suggest that increasing (amplifying) the failure probability is always desirable. Unlike time-to-failure data, where this is generally true (i.e., leads to less censoring), this is not the case with binary data. A more appropriate term would be "failure optimization," although this would mean abandoning the clever acronym of "FAMe"!

ACKNOWLDGMENTS

This research was supported in part by National Science Foundation DMS grant 02-04247.

ADDITIONAL REFERENCES

Nair, V., Escobar, L., and Hamada, M. (2003), "Design and Analysis of Experiments for Reliability Assessment and Improvement," in Mathematical Reliability: An Expository Perspective, eds. R. Soyer, T. A. Mazzuchi, and N. Singpurwala, International Series in Operations Research and Management Science, Vol. 67, Dordrecht: Kluwer Academic Publishers.

Wang, X. (2002), "Some Topics in Reliability," doctoral thesis proposal, University of Michigan, Dept. of Statistics.

Vijayan N. NAIR and Xiao WANG

Department of Statistics

University of Michigan

Ann Arbor, MI 48109