What is D-error?
D-error is a measure of how good or bad a design is at extracting information from respondents in a choice experiment. A design with a low D-error is better than a design with a high D-error, provided that both designs are for the same experiment. Comparing D-error between designs for different experiments is meaningless.
When generating designs using D-optimal methods in
cbc_design()
, several of the methods
("stochastic"
, "modfed"
, "cea"
)
use an algorithm that minimizes D-error to find efficient experimental
designs. The specific type of D-error computed depends on the prior
assumptions you provide.
Note: A “D-optimal” design is not necessarily the best design for your experiment. These designs optimize information about the “main effects” of interest, at the expence of information about interactions. If you feel interactions may be important, consider using a different design method or consider including interactions in your priors.
Types of D-error
Prior Parameter Assumptions
When computing D-error, a prior assumption about the respondent parameters needs to be made:
- -error assumes that all parameters are zero — i.e., respondents have no preference for any of the attribute levels
-
-error
assumes that all respondent parameters are equal to a fixed parameter
vector
- -error assumes that respondent parameters are distributed according to a probability distribution (typically multivariate normal)
How cbcTools Chooses D-error Type
In cbc_design()
with D-optimal methods
("stochastic"
, "modfed"
, "cea"
),
the type of D-error minimized depends on your prior specifications:
-
No priors provided (
priors = NULL
) → Uses -error -
Fixed parameters (using
cbc_priors()
with fixed values) → Uses -error -
Random parameters (using
cbc_priors()
withrand_spec()
) → Uses -error
Working Example
Let’s work through the mathematical steps of D-error computation using the same example from the literature.
Computing -error (Fixed Parameters)
For -error, we assume specific parameter values: .
Step 2: Compute Choice Probabilities
Using the multinomial logit formula:
For Question 1: - Utility Alt1: - Utility Alt2: - -
Similar calculations for Questions 2 and 3 give us the choice probabilities for each alternative in each question.
Computing -error (No Priors)
-error is a special case where , making all alternatives equally likely.
When all parameters are zero: - All utilities = 0 - All choice probabilities = where is the number of alternatives
For our 2-alternative case: for all alternatives.
The information matrix calculation follows the same formula, but with equal probabilities. This simplifies the calculation considerably since:
Computing -error (Random Parameters)
-error assumes parameters follow a probability distribution. For example, if we assume:
The computation involves:
- Draw parameter samples from the distribution (e.g., draws)
- Compute -error for each parameter draw
- Average the results to get -error
Draw | Parameter 1 | Parameter 2 | Parameter 3 | -error |
---|---|---|---|---|
1 | 0.25 | -0.73 | 0.67 | 1.45 |
2 | 1.14 | -0.67 | 0.67 | 1.90 |
3 | 0.69 | -0.50 | 1.23 | 1.92 |
… | … | … | … | … |
1000 | 1.15 | -1.67 | 0.57 | 2.82 |
-error = mean of all -errors = 1.90
Note: The example used in this article is inspired by this article on displayr.com.