The quality of the data we analyze in consumer or B2B marketing research is a direct function of the quality of the questions we ask. Nowhere is this truer than with open-ended questions. The side of the marketing research fence that is qualitative draws deeply from the well of consumer thought. The use of unstructured questions on surveys is an attempt to elicit that depth of feeling and emotion and harness it for consumer insight.
In a recent article in Quirk’s, By the Numbers: For Better Insights from Text Analysis, Elicit Better Comments, author Carolyn Lindquist of Service Management Group (SMG) offers up three best practice points for improving the quality of open-ended comments. The recent ascent of text analytics programs is driving the need for ever-richer comments, and more of them.
#1 – Target your open-ended questions so they best meet the needs and interest of respondent segments. If you are using complex logic in conjunction with a satisfaction question then different follow-up questions can be tied to the various satisfaction responses.
#2 – Shift from ‘what’ questions to ‘why’ questions. Research conducted by SMG shows that questions asking ‘why’ elicited responses that were approximately one-third longer than questions asking ‘what’. For example;
Please tell us why you were less than satisfied with the service at Skippy’s restaurant?
Please tell us why you were not highly satisfied with the quality of service at John’s Auto?
Please tell us why you were highly satisfied with your most recent experience on Sky Way Airlines?
#3 – Place open-ended questions late in the survey and not early. People think placing open-ended early in the survey allows respondents the opportunity to ‘spill their beans.’ It would be easy to hypothesize that open-ended questions placed early in the survey flow would be advantageous due to lower levels of respondent fatigue. However the opposite is the case. Research on research shows that respondents believe a survey will take a greater time commitment when it begins with open-ended questions. This has a dampening effect upon completion rates.
The same research indicated higher levels of disconnect between sentiment in open-ended responses and subsequent closed questions when the open-ended questions were placed early in the survey flow.
In summary, target your open-ended questions to specific responses on your closed-ended questions. Using advanced logic will facilitate this. Second, shifting your focus to asking why instead of what. This will drive deeper level comments. Lastly, location is king. Place your open-ended questions later in the survey. It will positively effect survey response rate and enhance the connection between closed and open-ended questions.
Learn more about Cvent's Text Analytics features (new enhancements and out puts coming in May! Stay tuned for a sneak peak...)