Stated Choice design comparison in a developing country: recall and attribute nonattendance
- Richard A Iles^{1}Email author and
- John M Rose^{2}
DOI: 10.1186/s13561-014-0025-3
© Iles and Rose; licensee Springer. 2014
Received: 18 April 2014
Accepted: 18 September 2014
Published: 24 October 2014
Abstract
Background
Experimental designs constitute a vital component of all Stated Choice (aka discrete choice experiment) studies. However, there exists limited empirical evaluation of the statistical benefits of Stated Choice (SC) experimental designs that employ non-zero prior estimates in constructing non-orthogonal constrained designs. This paper statistically compares the performance of contrasting SC experimental designs. In so doing, the effect of respondent literacy on patterns of Attribute non-Attendance (ANA) across fractional factorial orthogonal and efficient designs is also evaluated. The study uses a `real' SC design to model consumer choice of primary health care providers in rural north India. A total of 623 respondents were sampled across four villages in Uttar Pradesh, India.
Methods
Comparison of orthogonal and efficient SC experimental designs is based on several measures. Appropriate comparison of each design's respective efficiency measure is made using D-error results. Standardised Akaike Information Criteria are compared between designs and across recall periods. Comparisons control for stated and inferred ANA. Coefficient and standard error estimates are also compared.
Results
The added complexity of the efficient SC design, theorised elsewhere, is reflected in higher estimated amounts of ANA among illiterate respondents. However, controlling for ANA using stated and inferred methods consistently shows that the efficient design performs statistically better. Modelling SC data from the orthogonal and efficient design shows that model-fit of the efficient design outperform the orthogonal design when using a 14-day recall period. The performance of the orthogonal design, with respect to standardised AIC model-fit, is better when longer recall periods of 30-days, 6-months and 12-months are used.
Conclusions
The effect of the efficient design's cognitive demand is apparent among literate and illiterate respondents, although, more pronounced among illiterate respondents.
This study empirically confirms that relaxing the orthogonality constraint of SC experimental designs increases the information collected in choice tasks, subject to the accuracy of the non-zero priors in the design and the correct specification of a `real' SC recall period.
Keywords
Discrete choice experiment Experimental design India Unqualified Efficient Orthogonal Attribute nonattendanceBackground
The use of Stated Choice (SC) methods within health economics is widespread. The research focuses of SC studies in this literature are diverse covering a range of perspectives. These include: patient preferences for non-market medical interventions, health professional preferences towards prescribing medicines and treatments, health care priority setting and consumer preferences towards health insurance schemes [1]–[3]. A limited number of SC studies have focused on evaluating the performance of experimental designs, which underpin the use of SC in all studies [4]–[6].
Experimental designs determine the purposeful mixing of choice alternatives' attributes and their levels. Different statistical properties and constraints governing the mixing of attribute levels are used in many SC studies across the broad field of applied economics [7],[8]. Orthogonal designs often ensure each attribute pair combination appears equal number of times, which (often) results in zero correlation structure between attributes. An alternative group of designs are referred to as efficient. These designs, assuming non-zero prior parameter estimates, mix attribute levels so as to reduce elements of the Asymptotic Variance-Covariance (AVC) matrix.
Despite the established use of efficient designs in several applied economic fields, the health economic literature is firmly centred on applying orthogonal designs [3]. Recent exceptions to this trend exist. Araña et al. [9] in their evaluation of decision rules used a non-orthogonal design with zero-priors following the design study by Carlsson and Martinsson [4]. Hole [10] used the same D-optimality design procedure in SAS with zero set for prior estimates.
In all SC studies, the analyst must select the experimental design prior to going into the field. Whilst in theory efficient designs should perform better, there is little (and mixed) supporting empirical evidence. The work of Louviere et al. [11] indicates that the efficient design results in greater error variance. Bliemer and Rose [7] find that as per theory, efficient designs produce lower standard errors, but not necessarily larger t-ratios as they tend to produce lower scale (higher error variance). Higher scale of a design may be due to the presence of more choice task dominance, which may cause estimation problems.
Despite the above being known, what has not been studied to date are other effects of using different designs. There exists SC literature on design features, which states that designs may influence the outcome or process used in answering the questions shown [12],[13]. Attribute non-Attendance (ANA) is a well-documented survey respondent process aimed at simplifying choice task by reducing the number of attribute level trade-offs [14]–[18]. However, empirical studies to date use a fixed design. This paper seeks to add to the literature by exploring the effect of different experimental design among respondents with low levels of literacy.
This paper statistically compares the effect of respondent literacy on patterns of ANA across fractional factorial orthogonal and efficient designs. Literate and illiterate sub-samples of respondents provide a natural context to compare the cognitive burden of experimental designs and the resulting modelled output [19],[20]. The results of both stated and inferred ANA are presented. The additional consideration of recall bias is included in the analysis, due to the study's use of a `real' SC scenario.
The current study is based on consumers' choice of a `real' market good – choice of health care provider to treat fever symptoms in rural north India. Although many SC studies measure preferences for non-market goods, another set of SC studies - termed `real' SC surveys – model choice for market goods [21]–[23]. Respondents' processing of `real' SC choice tasks is assumed to be aided in part by the association they make with their most recent market transaction across the same alternatives. While the ability to make connections with recent market based decision-making situations is likely to reduce the cognitive load of SC choice tasks, well-established recall problems are expected to exist as well [24],[25].
For the purposes of this research, the terms orthogonal and efficient are used to refer to fractional factorial orthogonal and efficient designs respectively. Efficient designs relax the orthogonality constraint and allow for some correlation across levels and the use of non-zero priors. The term `efficient' follows the same use by Scarpa et al. [26] and Bliemer and Rose [7]. The alternate design, `orthogonal', uses implicit zero prior estimates and maintains orthogonality across attribute levels and also follows the same by Bliemer and Rose [7].
Methods
Attribute nonattendance
Not controlling for ANA in SC choice tasks is accepted as potentially leading to biased willingness-to-pay and welfare estimates [18],[27]. Two methods exist to account for ANA. These are stated and inferred. Stated ANA has respondents state at the time of completing the survey which, if any, attribute level they ignored in making their choice. Inferred uses a latent class model to classify respondents depending on the estimated probability that respondents ignore selected attributes. The literature is inconclusive as to which ANA method is superior [15],[28]. As a result, this study employs both methods.
Study context
This study estimates demand for rural allopathic^{a} health care providers in the north Indian state of Uttar Pradesh (UP). This demand is based on the counter-fractural assumption that government rural Bachelor of Medicine and Bachelor of Surgery (MBBS) doctors are always available at their assigned rural Primary or Community Health Centre [29],[30].
Survey respondents for the purposes of the design comparison come from two UP districts covering two of the four economic regions of the state. One district is from the Bundelkhand (southern region) and the other is from the Eastern region. These districts approximately represent the interquartile range of mean per capita income across UP [31]. Three districts were sampled. Respondents from two of the three districts are initially used due to the inclusion of a stated ANA question in these surveys. The sample of all three districts is then used for inferred ANA estimation for comparative purposes. Ethics approval was obtained from Griffith University's Human Ethics Committee.
Design
Stated Choice experimental design alternatives, attributes and levels
Doctor type | Price (INR)^{#} | Medicine | Distance | Recommendation |
---|---|---|---|---|
Jhola Chhap | (30, 60, 90) | Pill, | At Home, | Positive, No Recommendation, Negative |
50, 100, 150 | Pill & Injection | In village, | ||
Government MBBS | (1, 10, 20) | Free, | In village, | Positive, No Recommendation, Negative |
1, 25, 50 | Extra Charge | 5–15 km | ||
Private MBBS | (70, 140, 210) | Uncertain | In village, | Positive, No Recommendation, Negative |
100, 200, 300 | Treatment | 5–15 km | ||
None of the above | 0 | 0 | 0 | 0 |
Efficient designs, with the use of non-zero priors to maximise the determinant of the Fisher Information matrix, relax the orthogonality constraint to maximise the information gathered from a given set of choice tasks. The parameter estimates from the pilot orthogonal survey are used in the construction of the alternate efficient design. Ngene was again used to construct the efficient design.
D-error comparisons between orthogonal and efficient designs
Experimental design specific D-error | Priors = 0 | Priors ≠ 0 |
---|---|---|
D_{z}-error (orthogonal) | 0.199 | 0.222 |
D_{p}-error (efficient) | 0.190 | 0.208 |
Efficiency measures in the SC experimental design literature often use several `error' measurements. Scarpa et al. [26] argue that efficiency comparison across the orthogonal and efficient designs is not meaningful unless the D-error measurement accounts for the different experimental design assumptions. Table 2 shows that the constraint of orthogonality reduces the `efficiency' of an experimental design, irrespective of whether priors are used. Using the appropriate D-error measurement alone does not provide a balanced comparison. In such a case, the orthogonal design scores 0.199, compared to the efficient score of 0.208. This comparison suggests that the orthogonal design is more efficient. Moreover, within either design the use of zero priors provides D-error scores that are lower. Maintaining the orthogonality constraint the design assuming zero priors is 0.199, compared to the non-zero priors design score of 0.222. The same is also true for the efficient design.
However, holding the priors constant across the two experimental designs shows that the efficient design provides a lower D-error score. When zero-priors are used the efficient design registers a 0.190 D-error, compared to the 0.199. Likewise, when priors are non-zero the efficient design has a D-error of 0.208, compared to 0.222.
Alternative dominance by design
Strict | Moderate | Weak | |
---|---|---|---|
Orthogonal | 0 | 4 | 9 |
Efficient | 1 | 2 | 9 |
Definition | price_{a} < price_{b,c} | price_{a} <= price_{b,c} | price_{a} <= price_{b,c} |
dist_{a} <= dist_{b,c} | dist_{a} = lowest | dist_{a} = lowest | |
recomm_{a} < recomm_{b,c} | recomm_{a} <= recomm_{b,c} | recomm_{a} <= recomm_{b,c} | |
med_{a} = best | med_{a} = best | ||
med_{b} = worst | med_{b} = worst |
Samples
Rural blocks and their corresponding villages were selected in a stratified quasi-random sampling frame. District administrative blocks were randomly selected and from selected blocks, gram panchayats (local level of administration with elected council governing a collection of four to seven villages) were then randomly selected. Relationships with elected village leaders in each of the sample villages were developed during several preliminary visits associated with collecting qualitative data. Village households were randomly selected by enumerators with either the personal or delegated assistance of village leaders. As per ethics approval, no incentives were given to respondents and all respondents were verbally informed as to their right to end their involvement in the survey without penalty. Respondents provided verbal consent before commencing any surveys.
A total of 623 respondents were sampled across four villages in September 2012. A total of 5607 choice tasks were completed. The Bundelkhand sample (district 1) answered 3285 choice tasks from each design and from the Eastern region (district 2) the sample was 2322. Each respondent answered choice tasks from one randomly assigned design. On average the survey was completed in 25 minutes. The majority of this time was spent introducing the survey in reduced form (i.e. 2 alternatives and 2 attributes). Respondents were then progressively stepped through larger choice tasks until the enumerator was satisfied the respondent understood the concept of the survey.
Descriptive statistics of district 1 and 2 samples
Efficient | Orthogonal | |||||
---|---|---|---|---|---|---|
Mean/Per cent | St. dev | Min/Max | Mean/Per cent | St. dev | Min/Max | |
Age | 40.5 | 15.2 | 18/80 | 39.7 | 15.6 | 18/88 |
Age <30 (%) | 34.4 | 35.3 | ||||
Age >55 (%) | 22.2 | 21.2 | ||||
Female (%) | 47.6 | 48.1 | ||||
Household size | 7.27 | 3.9 | 1/29 | 7.37 | 3.4 | 2/23 |
Religion (%) | ||||||
Hindu | 81.0 | 76.0 | ||||
Brahmin | 11.3 | 10.6 | ||||
Kshatriya | 3.9 | 4.2 | ||||
Vaishya | 36.7 | 32.7 | ||||
Sudhra | 24.4 | 23.4 | ||||
Tribe | 4.5 | 5.4 | ||||
Muslim | 19.0 | 23.4 | ||||
Income | ||||||
Household income p.a* | 54485 | 46901 | 1000/350000 | 51603 | 36653 | 14000/300000 |
Personal income p.a* | 17580 | 24439 | -/200000 | 17887 | 22967 | -/154000 |
Education (%) | ||||||
Illiterate | 46.9 | 45.2 | ||||
< Primary | 10.6 | 12.8 | ||||
Primary | 12.9 | 13.8 | ||||
< High | 14.5 | 16.0 | ||||
High | 8.0 | 7.7 | ||||
Intermediate | 4.2 | 3.2 | ||||
University | 2.6 | 0.9 | ||||
Sample size | 311 | 312 |
Percentage of district 1 and 2 samples, by recall
Pilot | Efficient | Orthogonal | |
---|---|---|---|
Recall | |||
≤ 14 days | 18.8 | 27.0 | 22.1 |
15-30 days | 6.3 | 23.8 | 23.4 |
2-6 months | 18.8 | 21.2 | 22.4 |
7-12 months | 18.8 | 14.5 | 16.7 |
1-5 years | 37.5 | 13.2 | 14.7 |
Sample size | 16 | 311 | 312 |
Results
Full trade-off modelling
where j is the number of alternatives, x_{i} is a vector of consumer (i.e. decision-maker) characteristics and i represents an individual consumer. This model assumes that each choice made is independent from all others (IID assumption).
Full trade-off with MNL and full recall period
Efficient | Orthogonal | |||||
---|---|---|---|---|---|---|
Coeff. | St. err^ | t-ratio | Coeff. | St. err^ | t-ratio | |
Unqualified - jhola chhap - doctor | ||||||
Constant | −0.354 | 0.007 | −1.03 | 0.234 | 0.004 | 1.10 |
Price | −0.020 | <0.001 | −13.01 | −0.020 | <0.001 | −14.44 |
Distance (in village) | −0.088 | 0.001 | −1.62 | 0.078 | 0.001 | 1.46 |
base: at home | ||||||
Medicine (pill + Inject) | 0.136 | 0.001 | 2.54 | 0.136 | 0.001 | 2.52 |
base: (pill) | ||||||
Recomm. +ve | −0.021 | 0.001 | −0.28 | −0.032 | 0.001 | −0.44 |
base: no recomm. | ||||||
Recomm. +ve | −0.135 | 0.002 | −1.54 | −0.043 | 0.002 | −0.55 |
base: no recomm. | ||||||
Private MBBS | ||||||
Price | −0.022 | <0.001 | −9.22 | −0.016 | <0.001 | −15.30 |
Distance (5–15 km) | −1.796 | 0.003 | −12.38 | −1.206 | 0.002 | −12.65 |
base: in village | ||||||
Recomm. +ve | 0.324 | 0.002 | 2.61 | −0.322 | 0.002 | −2.62 |
base: no recomm. | ||||||
Recomm. +ve | −0.298 | 0.003 | −2.24 | 0.283 | 0.002 | 2.68 |
base: no recomm. | ||||||
Government MBBS | ||||||
Constant | −1.285 | 0.007 | −3.65 | −0.802 | 0.003 | −4.48 |
Price | −0.012 | <0.001 | −4.03 | −0.011 | <0.001 | −4.02 |
Distance (5–15 km) | −1.512 | 0.001 | −24.81 | −1.617 | 0.001 | −29.41 |
base: in village | ||||||
Medicine (extra INR) | −0.321 | 0.001 | −4.30 | −0.462 | 0.001 | −8.70 |
base: free | ||||||
Recomm. +ve | −0.291 | 0.002 | −3.18 | 0.097 | 0.001 | 1.34 |
base: no recomm. | ||||||
Recomm. +ve | −0.095 | 0.002 | −1.16 | −0.113 | 0.002 | −1.47 |
base: no recomm. | ||||||
None | ||||||
Constant | −5.536 | 0.007 | −14.53 | −5.575 | 0.005 | −20.12 |
LL | −1951.2 | −1849.6 | ||||
AIC | 3936.4 | 3733.1 | ||||
AIC/n | 1.4 | 1.3 | ||||
ρ ^{ 2 } | 0.318 | 0.360 | ||||
n | 2799 | 2808 |
The model fit statistics indicate that the orthogonal design fits the data better than the corresponding efficient design and data. The Akaike Information Criteria (AIC) for the orthogonal design and data are also lower. This is also true for the standardised AIC. A ρ^{2} of 0.36 translates into a standard linear regression-based coefficient of determination measure of approximately 0.7 [34]. The efficient design ρ^{2} is 0.318. The ρ^{2} measure continued to show that the orthogonal design/data is a better fit, however, this measure is dataset specific and so should not be the basis for comparison.
The parameter t-ratios from the orthogonal design are also consistently higher than those from the efficient design. Twelve t-ratios from the orthogonal design are larger than their corresponding efficient design t-ratios. Of these, nine are statistically significant at the five per cent level.
Full trade-off with MNL and ≥ 14 day recall
Efficient | Orthogonal | |||||
---|---|---|---|---|---|---|
Coeff. | St. err^ | t-ratio | Coeff. | St. err^ | t-ratio | |
Unqualified - jhola chhap - doctor | ||||||
Constant | 0.314 | 0.010 | 1.11 | 0.692 | 0.012 | 2.36 |
Price | −0.021 | <0.001 | −7.27 | −0.014 | <0.001 | −5.56 |
Distance (in village) | −0.126 | 0.003 | −1.30 | 0.127 | 0.004 | 1.26 |
base: at home | ||||||
Medicine (pill + Inject) | 0.369 | 0.004 | 3.63 | 0.140 | 0.004 | 1.38 |
base: (pill) | ||||||
Recomm. +ve | 0.260 | 0.005 | 1.96 | −0.008 | 0.006 | −0.05 |
base: no recomm. | ||||||
Recomm. +ve | −0.065 | 0.005 | −0.45 | −0.168 | 0.006 | −1.15 |
base: no recomm. | ||||||
Government MBBS | ||||||
Price | −0.031 | <0.001 | −6.24 | 0.000 | <0.001 | 0.01 |
Distance (5–15 km) | −1.323 | 0.004 | −11.83 | −1.417 | 0.004 | −13.30 |
base: in village | ||||||
Medicine (extra INR) | −0.682 | 0.004 | −5.68 | −0.430 | 0.004 | −4.14 |
base: free | ||||||
Recomm. +ve | 0.346 | 0.005 | 2.47 | −0.119 | 0.006 | −0.83 |
base: no recomm. | ||||||
Recomm. +ve | −0.764 | 0.005 | −5.77 | −0.188 | 0.006 | −1.27 |
base: no recomm. | ||||||
Private MBBS | ||||||
Constant | −2.348 | 0.007 | −12.44 | −1.181 | 0.007 | −6.47 |
LL | −561.6 | −482.8 | ||||
AIC | 1147.7 | 989.5 | ||||
AIC/n | 1.5 | 1.6 | ||||
ρ ^{ 2 } | 0.239 | 0.228 | ||||
n | 765 | 621 |
The standardised AIC scores show that the efficient design and data provide a better fit to the MNL model. The efficient design has a standardised AIC of 1.50 compared to the orthogonal result of 1.60.
Stated ANA modelling
Stated ANA with MNL and ≤ 14-day recall
Efficient | Orthogonal | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
Weak dominance | No dominance | Weak dominance | No dominance | |||||||||
Coeff. | St. err^ | Coeff. | St. err^ | Coeff. | St. err^ | Coeff. | St. err^ | |||||
Unqualified - jhola chhap - doctor | ||||||||||||
Constant | −0.130 | 0.023 | −0.265 | 0.009 | 0.748 | ** | 0.018 | 0.301 | 0.011 | |||
Price | −0.007 | * | <0.001 | −0.015 | *** | <0.001 | −0.017 | *** | <0.001 | −0.011 | *** | <0.001 |
Distance (in village) | −0.300 | * | 0.009 | −0.117 | 0.004 | 0.037 | 0.008 | 0.163 | 0.004 | |||
base: at home | ||||||||||||
Medicine (pill + Inject) | 0.153 | 0.009 | 0.381 | *** | 0.004 | −0.072 | 0.007 | 0.169 | 0.004 | |||
base: (pill) | ||||||||||||
Recomm. +ve | 1.350 | *** | 0.021 | 0.398 | *** | 0.005 | 0.259 | 0.011 | 0.199 | 0.006 | ||
base: no recomm. | ||||||||||||
Recomm. +ve | −0.831 | *** | 0.015 | −0.201 | 0.006 | −0.737 | *** | 0.012 | −0.440 | ** | 0.007 | |
base: no recomm. | ||||||||||||
Government MBBS | ||||||||||||
Price | −0.001 | 0.001 | −0.034 | *** | <0.001 | −0.017 | ** | <0.001 | −0.005 | <0.001 | ||
Distance (5–15 km) | −1.660 | *** | 0.011 | −1.463 | *** | 0.004 | −1.744 | *** | 0.008 | −1.586 | *** | 0.005 |
base: in village | ||||||||||||
Medicine (extra INR) | −1.295 | *** | 0.014 | −0.860 | *** | 0.005 | −0.704 | *** | 0.008 | −0.540 | *** | 0.005 |
base: free | ||||||||||||
Recomm. +ve | 1.448 | *** | 0.019 | 0.631 | *** | 0.006 | −0.287 | 0.011 | 0.057 | 0.007 | ||
base: no recomm. | ||||||||||||
Recomm. +ve | −1.131 | *** | 0.016 | −0.943 | *** | 0.005 | −0.391 | * | 0.011 | −0.442 | ** | 0.007 |
base: no recomm. | ||||||||||||
Private MBBS | ||||||||||||
Constant | −1.796 | *** | 0.014 | −2.325 | *** | 0.007 | −1.316 | *** | 0.011 | −1.213 | *** | 0.007 |
LL | −199.7 | −545.3 | −284.1 | −458.8 | ||||||||
AIC | 423.4 | 1115 | 592.3 | 941.5 | ||||||||
AIC/n | 1.2 | 1.5 | 1.5 | 1.6 | ||||||||
ρ ^{ 2 } | 0.328 | 0.251 | 0.283 | 0.253 | ||||||||
n | 359 | 748 | 413 | 608 |
Controlling for stated ANA helps provide more consistent coefficients. Only one set of coefficients have differing signs in the results summarised in Table 8. The positive Distance (unqualified) coefficient continues in the orthogonal design `weakly dominant' and `no dominance' groups.
The use of a LC model, being more data intensive, requires a comparison of the designs using a larger sample. Data from a third district in UP is included. This additional data was not included in the previous MNL modelling due to the lack of information on respondents' stated ANA. The descriptive statistics of the new sample are consistent with those in Tables 4 and 5.
Inferred ANA class probabilities
Latent class MNL class probabilities for illiterate respondents from districts 1, 2 and 3 by recall period
Efficient | Orthogonal | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Recall # | Recall period | No fixed | Med. fixed | Recom fixed | Dist. fixed | No fixed | Med. fixed | Recom fixed | Dist. fixed | |
1 | ≤ 14 days | Prob. | 0.69 | 0.31 | <0.01 | <0.01 | 0.93 | 0.06 | 0.01 | n/a |
t-ratio | 9.60 | 4.40 | <0.01 | <0.01 | 39.45 | 2.80 | 0.84 | n/a | ||
2 | ≤ 30 days | Prob. | 0.65 | 0.34 | <0.01 | 0.01 | 0.96 | 0.04 | <0.01 | <0.01 |
t-ratio | 9.77 | 6.10 | 0.89 | 0.30 | 23.32 | 2.34 | <0.01 | <0.01 | ||
3 | ≤ 6 months | Prob. | 0.45 | 0.47 | 0.03 | 0.04 | 0.99 | 0.01 | <0.01 | <0.01 |
t-ratio | 16.61 | 15.96 | 4.77 | 2.69 | 48.00 | 0.76 | <0.01 | <0.01 | ||
4 | ≤ 12 months | Prob. | 0.67 | 0.31 | 0.01 | 0.01 | 0.98 | 0.02 | <0.01 | n/a |
t-ratio | 15.30 | 7.50 | 1.56 | 1.16 | 57.79 | 1.96 | <0.01 | n/a | ||
5 | Total | Prob. | 0.60 | 0.37 | 0.01 | 0.03 | n/a | n/a | n/a | n/a |
t-ratio | 13.46 | 8.92 | 2.58 | 3.64 | n/a | n/a | n/a | n/a |
Latent class MNL class probabilities for literate respondents from districts 1, 2 and 3 by recall period
Efficient | Orthogonal | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Recall # | Recall period | No fixed | Med. fixed | Recom fixed | Dist. fixed | No fixed | Med. fixed | Recom fixed | Dist. fixed | |
1 | ≤ 14 days | Prob. | 0.86 | 0.14 | <0.01 | <0.01 | 0.96 | 0.04 | <0.01 | n/a |
t-ratio | 16.88 | 2.75 | <0.01 | <0.01 | 36.95 | 1.62 | <0.01 | n/a | ||
2 | ≤ 30 days | Prob. | 0.84 | 0.16 | <0.01 | <0.01 | 0.95 | 0.06 | <0.01 | n/a |
t-ratio | 19.03 | 4.31 | <0.01 | <0.01 | 51.89 | 3.02 | <0.01 | n/a | ||
3 | ≤ 6 months | Prob. | 0.87 | 0.12 | 0.01 | <0.01 | 0.81 | 0.19 | 0.001 | <0.01 |
t-ratio | 24.00 | 3.23 | 2.81 | <0.01 | 13.74 | 3.40 | 0.09 | <0.01 | ||
4 | ≤ 12 months | Prob. | 0.89 | 0.09 | 0.01 | 0.01 | 0.95 | 0.05 | 0.002 | n/a |
t-ratio | 21.93 | 2.27 | 2.85 | 0.86 | 53.27 | 2.98 | 0.62 | n/a | ||
5 | Total | Prob. | 0.88 | 0.10 | 0.01 | 0.01 | 0.17 | 0.82 | <0.01 | 0.02 |
t-ratio | 28.75 | 3.53 | 2.95 | 1.61 | 6.35 | 26.43 | 0 | 0.94 |
The experimental design attribute `medicine' is consistently the second most likely attribute to be ignored by respondents. This is the case for both designs, although at different probability levels. The probability that the `recommendation' and `distance' attributes are ignored by respondents, in both designs, generally have low t-ratios and probabilities under five per cent. For efficient design illiterate respondents, the medicine ANA ranged between 0.47 and 0.31. This range is lower for efficient design among literate respondents – 0.16 to 0.09. No noticeable difference is evident across literacy groups in answering orthogonal design choice tasks where the ranges are: 0.06 to 0.01 for illiterate respondents and 0.18 to 0.04 for literate. The one exception is with the full recall period with the orthogonal design, using literate respondents. In this scenario the ANA for medicine in the orthogonal design has a probability of 0.82.
Discussion
SC experimental design theory indicates that efficient designs should produce parameter output with lower standard errors and higher coefficients. The AVC provides the link between the relative efficiency of an experimental design and the data driven standard error of a given parameter. A lower appropriate D-error measure of a design is expected to translate into a lower standard error of a given coefficient. The results of this study indicate that the recall period does impact on experimental design model fit. In the base case of full recall and assuming full trade-off, only one efficient design parameter conforms to theory with a lower standard error and higher coefficient. Modelling using a 14-day or less recall sub-set of data provides seven efficient design parameters conforming to design theory.
The selection of recall period has a strong effect on standardised AIC measures of experimental design fit to the respective datasets. The datasets defined by respondents with a 14-day or less recall period consistently show that the efficient design had a better model fit. This effect is evident when: i) full trade-offs are assumed, ii) stated ANA is incorporated into MNL models and iii) inferred ANA is modelled using a LC model (results not shown in paper). The evidence of this pattern under differing attribute trade-off assumptions suggests that the benefits of greater information collected through efficient designs outweighs the potential noise created by inconsistent respondent choices due to cognitively more complex efficient designs.
Controlling for design induced alternative dominance indicates that the estimated results from choice tasks that contain fewer dominant alternatives provide more consistently paired coefficients and standard errors. Among the `no dominance' group of choice tasks the efficient design results continue to provide higher coefficients and lower standard errors. The opposite is true of choice tasks in the `weakly dominant' group. Of the nine choice task from each design defined as at least `weakly dominant', three orthogonal designs induced estimated probabilities for the dominant alternative at ≥ 0.95, while the corresponding number of efficient design choice tasks is five. The lower number of dominant choice task, defined by design and the data, is thought to help produce higher coefficients and lower standard errors.
However, the repeated opposing coefficient signs between the two designs, across several parameters and when controlling for ANA and recall bias, is unexpected. Controlling for non-trading across choice task attributes through stated ANA reduced the number of opposing coefficient signs, however, the Distance parameter for the unqualified – jhola chhaap - provider remained negative for the efficient design and positive for the orthogonal. The increase in distance travelled to consult a health care provider is expected to reduce the likelihood of consulting that provider, all other things being equal. The positive Distance coefficient in the orthogonal design maybe a result of uncontrolled heuristics.
The higher estimated probabilities in the LC model among the orthogonal design, associated with respondents trading across all attributes, indicates that respondents are less likely to employ choice task heuristics. These higher probabilities, relative to efficient design choice tasks in Tables 9 and 10, are evident among literate and illiterate respondents. The interpretation of higher estimated probabilities for `no fixed' attributes at zero, corresponding to reduced likelihood of employing heuristics, is supported by a relatively higher proportion of literate respondents grouped in the `no fixed' category. The lower probability of respondents simplifying orthogonal design choice tasks indicates that these choice tasks are on average less cognitively demanding and less likely to induce respondent choice inconsistencies.
The use of a MNL model as the basis for model fit design comparisons is a limiting factor of this analysis. While MNL output is generally robust against significant biases, the IID assumption is strong and may have an unaccounted affect on model fit. Moreover, the effects of choice task blocking is also not controlled in the study.
Conclusions
The cognitive demands of efficient designs have a real impact on the likelihood of respondents' employing choice task heuristics. The effect of the perceived cognitive demand of the efficient design is apparent among literate and illiterate respondents, however, it is more pronounced among illiterate respondents. Accounting for ANA when modelling choice data further enhanced the comparative statistical performance of the efficient design.
The use of an appropriate D-error measure provides an important insight into the performance of different experimental designs. However, as speculated in the literature, the role of alternative dominant choice tasks, either by design and/or ANA, also affects commonly used goodness-of-fit measures. As such care should be taken in the comparative evaluation of alternative experimental designs.
This study empirically confirms that relaxing the orthogonality constraint of SC experimental designs increases the reliable information collected in choice tasks, subject to the correct specification of a `real' SC recall period. As model parameter estimates diverge from the non-zero priors used in the efficient design construction, the relative statistical performance of the orthogonal design improves and outperforms the efficient design. Although not presented in this study, the importance of convergence between non-zero priors and model parameter estimates suggests that SC Bayesian experimental designs would better manage potential non-zero prior uncertainty.
Endnotes
^{a}Allopathic health care in India is based primarily on the use of pharmaceutical goods to treat symptoms. This is in contrast to traditional Indian systems of medicine.
^{b}A range of terms are used to define unqualified allopathic health care providers in India, however, the term jhola chhap is widely used among rural north Indian settings and is also acknowledged in legal settings (see: [32]).
Authors' contributions
RI conceived of the study, participated in the experimental design, managed the collection of data, carried out data manipulation and statistical analysis and drafted the manuscript. JR participated in the experimental design and assisted in framing the experimental design comparison. Both authors read and approved the final manuscript.
Additional file
Declarations
Authors’ Affiliations
References
- Ryan M, Gerard K: Using discrete choice experiments to value health care programmes: current practice and future research reflections. Appl Health Econ Health Policy 2003, 2: 55–64.PubMedGoogle Scholar
- Guttmann R, Castle R, Fiebig D: Use of Discrete Choice Experiments in Health Economics: An Update of the Literature. University of Technology, Working Papers. Sydney; 2009.Google Scholar
- de Bekker-Grob EW, Ryan M, Gerard K: Discrete choice experiments in health economics: a review of the literature. Health Econ 2012, 21: 145–172. 10.1002/hec.1697View ArticlePubMedGoogle Scholar
- Carlsson F, Martinsson P: Design techniques for stated preference methods in health economics. Health Econ 2003, 12: 281–294. 10.1002/hec.729View ArticlePubMedGoogle Scholar
- Viney R, Savage E, Louviere J: Empirical investigation of experimental design properties of discrete choice experiments in health care. Health Econ 2005, 14: 349–362. 10.1002/hec.981View ArticlePubMedGoogle Scholar
- Reed Johnson F, Lancsar E, Marshall D, Kilambi V, Mühlbacher A, Regier DA, Bresnahan BW, Kanninen B, Bridges JFP: Constructing experimental designs for discrete-choice experiments: report of the ISPOR conjoint analysis experimental design good research practices task force. Value Health 2013, 16: 3–13. 10.1016/j.jval.2012.08.2223View ArticlePubMedGoogle Scholar
- Bliemer MCJ, Rose JM: Experimental design influences on Stated Choice outputs: an empirical study in air travel choice. Transp Res A Policy Pract 2011, 45: 63–79. 10.1016/j.tra.2010.09.003View ArticleGoogle Scholar
- Ferrini S, Scarpa R: Designs with a priori information for nonmarket valuation with choice experiments: a Monte Carlo study. J Environ Econ Manag 2007, 53: 342–363. 10.1016/j.jeem.2006.10.007View ArticleGoogle Scholar
- Araña JE, León CJ, Hanemann MW: Emotions and decision rules in discrete choice experiments for valuing health care programmes for the elderly. J Health Econ 2008, 27: 753–769. 10.1016/j.jhealeco.2007.10.003View ArticlePubMedGoogle Scholar
- Hole AR: Modelling heterogeneity in patients' preferences for the attributes of a general practitioner appointment. J Health Econ 2008, 27: 1078–1094. 10.1016/j.jhealeco.2007.11.006View ArticlePubMedGoogle Scholar
- Louviere Jordan J, Islam T, Wasi N, Street D, Burgess L: Designing discrete choice experiments: do optimal designs come at a price? J Consum Res 2008, 35: 360–375. 10.1086/586913View ArticleGoogle Scholar
- Collins AT: Attribute Nonattendance in Discrete Choice Models: Measurement of Bias, and A Model for the Inference of Both Nonattandance and Taste Heterogeneity. PhD thesis. The University of Sydney, Institute of Transport and Logistics Studies; 2012.
- Hess S, Rose JM, Polak J: Non-trading, lexicographic and inconsistent behaviour in Stated Choice data. Transp Res Part D: Transp Environ 2010, 15: 405–417. 10.1016/j.trd.2010.04.008View ArticleGoogle Scholar
- Hensher DA: How do respondents process Stated Choice experiments? Attribute consideration under varying information load. J Appl Econ 2006, 21: 861–878. 10.1002/jae.877View ArticleGoogle Scholar
- Hensher D, Rose J, Greene W: The implications on willingness to pay of respondents ignoring specific attributes. Transportation 2005, 32: 203–222. 10.1007/s11116-004-7613-8View ArticleGoogle Scholar
- Lagarde M: Investigating Attribute Non-Attendance and its consequences in choice experiments with latent class models. Health Econ 2013, 22: 554–567. 10.1002/hec.2824View ArticlePubMedGoogle Scholar
- Hole AR, Kolstad JR, Gyrd-Hansen D: Inferred vs stated attribute non-attendance in choice experiments: a study of doctors' prescription behaviour. J Econ Behav Organ 2012, 96: 21–31. 10.1016/j.jebo.2013.09.009View ArticleGoogle Scholar
- Scarpa R, Zanoli R, Bruschi V, Naspetti S: Inferred and stated attribute non-attendance in food choice experiments. Am J Agric Econ 2013, 95: 165–180. 10.1093/ajae/aas073View ArticleGoogle Scholar
- Byrd DA, Sanchez D, Manly JJ: Neuropsychological test performance among Caribbean-born and U.S.-born African American elderly: the role of age, education and reading level. J Clin Exp Neuropsychol 2005, 27: 1056–1069. 10.1080/13803390490919353View ArticlePubMedGoogle Scholar
- Schneider BC, Lichtenberg PA: Influence of reading ability on neuropsychological performance in African American elders. Arch Clin Neuropsychol 2011, 26: 624–631. 10.1093/arclin/acr062PubMed CentralView ArticlePubMedGoogle Scholar
- Alfnes F: Consumers' Willingness to Pay for the color of salmon: A choice experiment with real economic incentives. In American Agricultural Economics Association 2005 Annual meeting; 24–27 July. Providence, RI. 2005:31.
- Lusk JL, Schroeder TC: Auction bids and shopping choices. B E J Econ Anal Pol 2006, 6: 1–37.Google Scholar
- Meenakshi JV, Banerji A, Manyong V, Tomlins K, Mittal N, Hamukwala P: Using a discrete choice experiment to elicit the demand for a nutritious food: willingness-to-pay for orange maize in rural Zambia. J Health Econ 2012, 31: 62–71. 10.1016/j.jhealeco.2012.01.002View ArticlePubMedGoogle Scholar
- Clarke PM, Fiebig DG, Gerdtham U-G: Optimal recall length in survey design. J Health Econ 2008, 27: 1275–1284. 10.1016/j.jhealeco.2008.05.012View ArticlePubMedGoogle Scholar
- Das J, Hammer J, Sanchez-Paramo C: The impact of recall periods on reported morbidity and health seeking behavior. J Dev Econ 2012, 98: 76–88. 10.1016/j.jdeveco.2011.07.001View ArticleGoogle Scholar
- Scarpa R, Rose JM: Design efficiency for non-market valuation with choice modelling: how to measure it, what to report and why. Aust J Agric Resour Econ 2008, 52: 253–282. 10.1111/j.1467-8489.2007.00436.xView ArticleGoogle Scholar
- Hensher D, Greene W: Non-attendance and dual processing of common-metric attributes in choice analysis: a latent class specification. Empir Econ 2010, 39: 413–426. 10.1007/s00181-009-0310-xView ArticleGoogle Scholar
- Scarpa R, Gilbride TJ, Campbell D, Hensher DA: Modelling attribute non-attendance in choice experiments for rural landscape valuation. Eur Rev Agric Econ 2009, 36: 151–174. 10.1093/erae/jbp012View ArticleGoogle Scholar
- Banerjee A, Deaton A, Duflo E: Wealth, Health, and Health Services in Rural Rajasthan. Am Econ Rev 2004, 94: 326–330. 10.1257/0002828041301902PubMed CentralView ArticlePubMedGoogle Scholar
- Chaudhury N, Hammer J, Kremer M, Muralidharan K, Rogers FH: Missing in action: Teacher and health worker absence in developing countries. J Econ Perspect 2006, 20: 91–116. 10.1257/089533006776526058View ArticlePubMedGoogle Scholar
- Government of Uttar Pradesh: Human Development Report 2006, Uttar Pradesh. Lucknow: Planning Department ed; 2006.
- Shukla VK, Kumar S: Praveen Kumar vs The State of Uttar Pradesh. 2013. In 64481 (High Court of India (Allahabad) ed.
- Coast J, Al-Janabi H, Sutton EJ, Horrocks SA, Vosper AJ, Swancutt DR, Flynn TN: Using qualitative methods for attribute development for discrete choice experiments: issues and recommendations. Health Econ 2012, 21: 730–741. 10.1002/hec.1739View ArticlePubMedGoogle Scholar
- Choice Metrics: Ngene. 1.1.1 edition. 2012.
- Domencich TA, McFadden D: Urban Travel Demand-A Behavioral Analysis. 1975.Google Scholar
Copyright
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.