November 11, 2021

Creating an effective stated choice experiment for tourism research

by Lisa Isle

Abstract

Studying consumer preferences and behavior in tourism and leisure often involves the use of stated choice experiments. These experiments aim to understand how specific attributes of goods and services impact consumer decision-making. Traditionally, these experiments have used orthogonal experimental designs, following the practices of general linear models. However, researchers in various fields have started adopting efficient designs, which are better suited for multinomial models and offer advantages over orthogonal designs. This paper examines the existing literature on both design approaches and suggests that efficient designs may be more appropriate for stated choice experiments in the tourism field. Efficient designs have the potential to require smaller sample sizes and produce more reliable parameter estimates. By reviewing and analyzing the literature, this paper provides researchers in the tourism and leisure field with a better understanding

INTRODUCTION

Stated choice (SC) experiments are widely used to assess consumer willingness to pay for specific attributes of goods and services. In these experiments, respondents are presented with several choice sets, each consisting of alternatives with different attribute levels. They then select their preferred alternative for each set. For instance, in a SC experiment about a downhill ski area, respondents might choose between different levels of attributes such as lift wait time, terrain park elements, variety of trail difficulty, and lift ticket prices.

Researchers then use these choice observations to gain insight into consumer preferences and their trade-offs between alternatives. Various studies have demonstrated the effectiveness of SC experiments for this purpose (Araña et al., 2016; Bliemer & Rose, 2011; Bliemer et al., 2009; Hagmann et al., 2015; Lyu & Lee, 2015).

Overall, SC experiments have proven to be a valuable tool in the data collection process for understanding consumer behavior and preferences.

A primary issue for SC experiments involves the construction of choice situations and the allocation of attribute levels to the design matrix. Traditionally, researchers have relied on the principle of orthogonality to minimize correlations between attribute levels (Rose & Bliemer, 2009) and orthogonal designs are still the most common type of SC experimental design (Barros & Assaf, 2012; Lyu & Lee, 2015; Scarpa & Rose, 2008). Louviere, Hensher and Swait (2000) provide a thorough review of orthogonal designs. In recent years, however, some researchers have criticized the appropriateness of orthogonal designs for SC experiments (e.g., Rose & Bliemer, 2009) and an increasing number of SC experiments have used new methods, such as efficient design, to allocate the attribute levels to the design matrix (Carlsson & Martinsson, 2003; Ferrini & Scarpa, 2007; Kanninen, 2002; Kessels et al., 2006; Rose & Bliemer, 2009). In the tourism and leisure fields, most of the SC modeling studies have used orthogonal designs (see e.g., Albaladejo-Pina & Díaz-Delfa, 2009; Azari et al., 2012; Grigolon et al., 2012; Huybers, 2003; Kelly et al., 2007; Lyu & Lee, 2015; Oh & Ditton, 2006; Oh, Ditton, Gentner, & Riechers, 2005) and relatively few have adopted efficient experimental designs, despite the fact that a number of studies in other fields have recently shown the usefulness of efficient designs for SC experiments (Beck, Fifer, & Rose, 2016; Bliemer & Rose, 2011; Bliemer et al., 2009; Devarasetty, Burris, & Douglass Shaw, 2012). Efficient designs generate data on which model parameters can be estimated with lower expected standard errors. Efficient designs, however, require parameter estimates, which are sometimes unknown unless previous studies or pilot studies have been conducted. Efficient designs also require more computing power, which has, until recent years, been a barrier to using efficient designs to develop choice sets. Today, however, commercial software (e.g., Ngene) is now available and makes the generation of efficient choice sets significantly easier. This paper reviews the literature involving efficient designs and discusses that an SC experiment that generated choice sets with efficient designs would be more appropriate for tourism research using SC modeling.

ORTHOGONAL DESIGNS

When an experiment that generates choice sets is orthogonal, the attributes of the experiment are statistically independent. This allows the researcher to determine each attribute’s influence on the observed choices that comprise the SC experiment (Bliemer & Rose, 2011; Choicemetrics, 2014; Regier, Ryan, Phimister, & Marra, 2007; Rose & Bliemer, 2009). Rose and Bliemer (2009) suggest there are three reasons orthogonal designs are used most often. First, as mentioned above, the choice sets generated with orthogonal designs generally have no correlation between attributes and, therefore, the influence of each attribute can be assessed independently. A second reason orthogonal designs are commonly used is that they are relatively easy to construct. Finally, Rose and Bliemer (2009) contend that history and inertia are responsible for the widespread use of orthogonal designs in SC experiments. Historically, the theory for discrete choice experiments was borrowed primarily from linear models, mostly ANOVA and linear regression models (Addelman, 1962; Green, 1974; Louviere & Woodworth, 1983). Because most SC studies have been done for practical purposes, few studies have compared design methodologies to identify best practices for experiment design. Moreover, orthogonal designs seem to have performed adequately thus reinforcing their use (Bliemer & Rose, 2011).

Despite their popularity, the literature has shown that orthogonal designs often do not maintain orthogonality within the data sets used to estimate discrete choice models (Bliemer & Rose, 2011; Hensher, Milthorpe, Smith, & Barnard, 1990; Rose & Bliemer, 2009). There are several reasons for this loss of orthogonality within the data sets (Rose & Bliemer, 2009). First, when subsets (or blocks) of choice sets are used, as is the case for almost all SC studies because presenting respondents with every possible choice set (a full factorial design) is not feasible in most studies, some correlation will likely occur. This is because it is rare that each block is evenly represented in the final data, in part because each block will realize different response rates. Secondly, when demographic variables such as age, gender and income are added to analysis, as they usually are, correlations often exist between these variables and attributes of the SC experiment. Finally, researchers who have produced choice sets orthogonally often review the final sets to look for situations that are either unrealistic or clearly favorable. In these cases, the choice sets are typically removed from the survey, thus interfering with the orthogonality of the resulting data. For these reasons, although the full set of choice sets produced by orthogonal designs are typically free of correlation between attribute levels, the data generated in these studies usually do have some level of correlation, though it is minimized.

REFERENCES

Addelman, S. (1962). Orthogonal main-effect plans for asymmetrical factorial experiments. Technometrics, 4(1), 21–46. http://doi.org/10.1080/00401706.1962.10489985

Albaladejo-Pina, I. P., & Díaz-Delfa, M. T. (2009). Tourist preferences for rural house stays: Evidence from discrete choice modelling in Spain. Tourism Management, 30(6), 805– 811. http://doi.org/10.1016/j.tourman.2009.01.001

Araña, J. E., León, C. J., Carballo, M. M., & Gil, S. M. (2016). Designing tourist information offices the role of the human factor. Journal of Travel Research, 55(6), 764–773. http://doi.org/10.1177/0047287515587113

Azari, K. A., Arintono, S., & Hamid, H. (2012). Effects of parking and cordon charge policies on tourist’s modal shift in central business district of Mashhad city, Iran. Current Issues in Tourism, 15(5), 489–496. http://doi.org/10.1080/13683500.2011.634495

Barros, C. P., & Assaf, A. G. (2012). Analyzing tourism return intention to an urban destination. Journal of Hospitality & Tourism Research, 36(2), 216–231. http://doi.org/10.1177/1096348010388658

Beck, M. J., Fifer, S., & Rose, J. M. (2016). Can you ever be certain? Reducing hypothetical bias in stated choice experiments via respondent reported choice certainty. Transportation Research Part B: Methodological, 89, 149–167. http://doi.org/10.1016/j.trb.2016.04.004

Bliemer, M. C. J., & Rose, J. M. (2011). Experimental design influences on stated choice outputs: An empirical study in air travel choice. Transportation Research Part A: Policy and Practice, 45(1), 63–79. http://doi.org/10.1016/j.tra.2010.09.003

Bliemer, M. C. J., Rose, J. M., & Hensher, D. A. (2009). Efficient stated choice experiments for estimating nested logit models. Transportation Research Part B: Methodological, 43(1), 19–35. http://doi.org/10.1016/j.trb.2008.05.008

Carmichael, C. & McCole, D. T. (2015). Understanding motivations of potential partners to develop an outdoor recreation program in an urban area. Journal of Outdoor Recreation and Tourism, 7-8, 55-64.  DOI: 10.1016/j.jort.2014.09.001

Carlsson, F., & Martinsson, P. (2003). Design techniques for stated preference methods in health economics. Health Economics, 12(4), 281–294. http://doi.org/10.1002/hec.729

Choicemetrics. (2014). Ngene User Manual and Reference Guide. Choicemetrics Ltd. Retrieved from http://choice-metrics.com/documentation.html

Culbertson, M. J., McCole, D. T. & McNamara, P. E. (2014). Practical Challenges and Strategies for Randomized Control Trials in Agricultural Extension and Other Development Programs.  Journal of Development Effectiveness, 6(3), 284-299. DOI: 10.1080/19439342.2014.919339

Devarasetty, P. C., Burris, M., & Douglass Shaw, W. (2012). The value of travel time and reliability-evidence from a stated preference survey and actual usage. Transportation Research Part A: Policy and Practice, 46(8), 1227–1240. http://doi.org/10.1016/j.tra.2012.05.002

Fowkes, T., & Wardman, M. (1988). The design of stated preference travel choice experiments: With special reference to interpersonal taste variations. Journal of Transport Economics and Policy, 22(1), 27–44.

Green, P. E. (1974). On the design of choice experiments involving multifactor alternatives. Journal of Consumer Research, 1(2), 61–68. http://doi.org/10.1086/208592

Grigolon, A. B., Kemperman, A. D. A. M., & Timmermans, H. J. P. (2012). The influence of low-fare airlines on vacation choices of students: Results of a stated portfolio choice experiment. Tourism Management, 33(5), 1174–1184. http://doi.org/10.1016/j.tourman.2011.11.013

Malete, L., McCole, D., Tshepang, T., Ocansey, R., Mphela, T., Maro, C., Adamba, C., and Kazi, J. (2019). Effects of a multiport-sport PYD intervention program on life skills and entrepreneurship in youth athletes. Journal of Sport & Exercise Psychology, 41(1), 77-88.

Hagmann, C., Semeijn, J., & Vellenga, D. B. (2015). Exploring the green image of airlines: Passenger perceptions and airline choice. Journal of Air Transport Management, 43, 37– 45. http://doi.org/10.1016/j.jairtraman.2015.01.003

Hensher, D. A., Milthorpe, F. W., Smith, N. C., & Barnard, P. O. (1990). Urban tolled roads and the value of travel time savings. Economic Record, 66(2), 146–156. http://doi.org/10.1111/j.1475-4932.1990.tb01714.x

Holecek, D., McCole, D. T., & Lee, J. (2016). Tasting Room Visitor Surveys: Experience with and Enjoyment of Cold-Hardy Wines. The Northern Grapes News, 5(2), pp. 8-9.

Holecek, D., & McCole, D. T. (2015). Who are the consumers of cold-climate wines. Cornell University, The Northern Grapes News. Available at https://ecommons.cornell.edu/bitstream/handle/1813/110104/2013-14-Y3-NB-Obj4-Consumers-Of-Cold-Climate-Wines.pdf?sequence=1

Huybers, T. (2003). Domestic tourism destination choices: A choice modelling analysis. The International Journal of Tourism Research, 5(6), 445–459.

Kanninen, B. J. (2002). Optimal design for multinomial choice experiments. Journal of Marketing Research, 39(2), 214–227. http://doi.org/10.1509/jmkr.39.2.214.19080

Kelly, J., Haider, W., Williams, P. W., & Englund, K. (2007). Stated preferences of tourists for eco-efficient destination planning options. Tourism Management, 28(2), 377–390. http://doi.org/10.1016/j.tourman.2006.04.015

McCole, D. T., Bobilya, A., Holman, T., Lindley, B. (2019). Benefits of summer camp: What do parents value? Journal of Outdoor Recreation, Education and Leadership, 11, 239-247. DOI: 10.18666/JOREL-2019-V11-I3-9672

McCole, D. T. & Holecek, D., & Popp, A. (2014). Understanding the travel behaviors of wine tourists in Michigan’s Leelanau Peninsula. The Northern Grapes News, 3(3), pp. 3-6.

Kessels, R., Goos, P., & Vandebroek, M. (2006). A comparison of criteria to design efficient choice experiments. Journal of Marketing Research, 43(3), 409–419. http://doi.org/10.1509/jmkr.43.3.409

Kessels, R., Jones, B., & Goos, P. (2011). Bayesian optimal designs for discrete choice experiments with partial profiles. Journal of Choice Modelling, 4(3), 52–74. http://doi.org/10.1016/S1755-5345(13)70042-3

Lee, J. .-H., McCole, D., Holecek, D. (2020). Exploring winery visitors in the emerging wine regions of the north central United States. Sustainability, 12(4), 1642. DOI: 10.3390/su12041642

McCole, D. T., Holecek, D., Eustice, C., & Lee, J., (2018). Understanding wine tourists in emerging wine regions: An examination of tasting room visitors in the Great Lakes region of the U.S. Tourism Review International, 22(2), 153-168. DOI: 10.3727/154427218X15319286372306

McFadden, D. (1978). Modeling the choice of residential location. Transportation Research Record, (673). Retrieved from https://trid.trb.org/view.aspx?id=87722

Popp, L. (2013). Understanding the push and pull motivations and itinerary patterns of wine tourists. Master’s Thesis. Michigan State University

McCole, D. T. & Joppe, M. (2014). The search for meaningful tourism indicators: The case of the International Upper Great Lakes Study. Journal of Policy Research in Tourism, Leisure and Events, 6(3), 248-263. DOI: 10.1080/19407963.2013.877471

Oh, C.O., & Ditton, R. B. (2006). Using recreation specialization to understand multi-attribute management preferences. Leisure Sciences, 28(4), 369–384. http://doi.org/10.1080/01490400600745886

Louviere, J. J., Hensher, D. A., & Swait, J. D. (2000). Stated Choice Methods: Analysis and Applications. Cambridge University Press.

Louviere, J. J., Street, D., Burgess, L., Wasi, N., Islam, T., & Marley, A. A. J. (2008). Modeling the choices of individual decision-makers by combining efficient choice experiment designs with extra preference information. Journal of Choice Modelling, 1(1), 128–164. http://doi.org/10.1016/S1755-5345(13)70025-3

Lyu, S. O., & Lee, Y. (2015). How do golf tourists manage golfing constraints?: A choice modeling approach. Journal of Hospitality & Tourism Research, 1096348015597036. http://doi.org/10.1177/1096348015597036

Oh, C.O., Ditton, R. B., Gentner, B., & Riechers, R. (2005). A stated preference choice approach to understanding angler preferences for management options. Human Dimensions of Wildlife, 10(3), 173–186. http://doi.org/10.1080/10871200591003427

McCole, D. T. & Vogt, C., (2011). Informing sustainability decisions: The role of parks, recreation, and tourism scholars in addressing unsustainability. Journal of Park and Recreation Administration. 29(3), 38-54.

Sándor, Z., & Wedel, M. (2005). Heterogeneous conjoint choice designs. Journal of Marketing Research, 42(2), 210–218. http://doi.org/10.1509/jmkr.42.2.210.62285

Street, D. J., & Burgess, L. (2004). Optimal and near-optimal pairs for the estimation of effects in 2-level choice experiments. Journal of Statistical Planning and Inference, 118(1–2), 185–199. http://doi.org/10.1016/S0378-3758(02)00399-3

Street, D. J., & Burgess, L. (2007). The Construction of Optimal Stated Choice Experiments: Theory and Methods (1 edition). Hoboken, N.J: Wiley-Interscience.

Rose, J. M., & Bliemer, M. C. J. (2009). Constructing efficient stated choice experimental designs. Transport Reviews, 29(5), 587–617. http://doi.org/10.1080/01441640902827623

Zhang, L., & Hanks, L. (2018). Online reviews: The effect of cosmopolitanism, incidental similarity, and dispersion on consumer attitudes toward ethnic restaurants. International Journal of Hospitality Management, 68(January), 115-123. https://doi. org/10.1016/j.ijhm.2017.10.008

Zhang, Z., Liang, S., Li, H., & Zhang, Z. (2019). Booking now or later: Do online peer reviews matter? International Journal of Hospitality Management, 77(January), 147- 158. https://doi.org/10.1016/j.ijhm.2018.06.024

Zhang, Z., Ye, Q., Law, R., & Li, Y. (2010). The impact of e-word-of-mouth on the online popularity of restaurants: A comparison of consumer reviews and editor reviews. International Journal of Hospitality Management, 29(4), 694-700. https:// doi.org/10.1016/j.ijhm.2010.02.002

Zou, P., Yu, B., & Hao, Y. (2011). Does the valence of online consumer reviews matter for consumer decision making? The moderating role of consumer expertise. Journal of Computers, 6(3), 484-488. http://www.jcomputers.us/vol6/jcp0603-14.pdf