An often occuring problem in market(ing) research is satisficing. In general, people have limited cognitive resources and attempt to minimize cognitive effort. As such, rather than attempting to find an optimal solution to a problem, people might go with the first minimally acceptable alternative that comes to mind. Responding to a survey often requires some (or a great deal) of cognitive effort. In a multiple choice task, an individual must understand the question, keep multiple alternatives in mind in order to compare options, and evaluate the quality of each option in relation to the question. Participants might satisfice in questionnaires by choosing the first alternative that fits the question (as opposed to the best alternative) or, in extreme cases, by answering completely randomly. Consequently, there is a lot of -often- unobserved noise in your data.
Read the rest of this entry »
Oppenheimer, Meyvis and Davidenko (2007) proposed a method to detect satisficing in surveys. They proposed that participants who are satisficing will often not bother to read the questions or instructions in a survey. Assuming that these questions and instructions are necessary to enable participants to complete the survey such that the data are useful, the identification and elimination of these participant’s data should substantially increase the power of a study. They created an instructional manipulation check (IMC), which is a measure of whether respondents read (1) the instructions, (2) the questions.
Simply put, in the instructional texts a sentence such as e.g. “If you read this sentence, write ‘I read the instructions’ on top of the first page of this booklet” are inserted in the instructions section. In an internet survey, respondents are requested to click on the page’s title instead of the ‘continue’-button.
Another instructional manipulation check might be inserted into a matrix containing multiple Likert-items, by writing e.g. “If you read this question, click on the little blue button in right corner”.
Do we care? Yes. Research by Oppenheimer and colleagues demonstrated that 14% up to 46% of participants did not read the instructions in the surveys they conducted. 7% of respondents did not read the questions when using the IMC as part of a Likert-item matrix. When comparing a model with the full dataset (all participants) this model was quite different from an alternative model (which only included respondents who followed all instructions).
This entry was posted
on Monday, May 4th, 2009 at 3:17 PM and is filed under Literature, Market Research.
You can follow any responses to this entry through the RSS 2.0 feed.
You can skip to the end and leave a response. Pinging is currently not allowed.
Monday, May 4th, 2009 by
May 22nd, 2009 at 10:39 AM
Very nice ánd very easy method to detect satisficing. I used their method last year in an internet survey and had 25% satisficers! If I would not have incorporated that simple item, my model would have been worthless… But now, without those satisficers, I have a nice result