{"id":5616,"date":"2018-10-05T09:44:06","date_gmt":"2018-10-05T16:44:06","guid":{"rendered":"https:\/\/athis-consulting.com\/news\/?p=5616"},"modified":"2018-10-05T09:45:23","modified_gmt":"2018-10-05T16:45:23","slug":"can-big-data-support-smart-market-research","status":"publish","type":"post","link":"https:\/\/athis-technologies.com\/news\/report\/2018\/can-big-data-support-smart-market-research\/","title":{"rendered":"Why Big Data can support Market Research Experts?"},"content":{"rendered":"
<\/div>\n

Today\u2019s businesses see market data as a commodity. Readily accessible information about consumer activity and preferences allows market researchers to develop large data sets to mine for consumer insights. And indeed, a look through recent market research industry publications shows that discussions in the field have been dominated by a focus on data analysis.<\/p>\n

But more often than not, insight into what customers really care about is hampered by the quality of the data being collected. Some market researchers conflate the idea of data quality with sample size, with the belief that reliability, validity, and other characteristics of \u201cgood measurement\u201d derive solely from the amount of data collected. This is certainly not the case.<\/p>\n

A heavy emphasis on data collection and analysis is irrelevant if it omits the first and most important step of market research \u2014 the design of the metrics. In the psychometric tradition, survey development and the construction of specific survey questions has been emphasized as the most important step in the research process. Unfortunately, this step is getting short shrift by most market researchers today.<\/p>\n

Failing to assess the measures that are the foundation of business decisions poses a colossal risk. Making data-driven decisions based on poor measures can be infinitely worse than making decisions without data at all.<\/p>\n

To help organizations think more critically about the measures they use to collect information about consumers, we\u2019ve outlined four common misconceptions held by many market researchers and provide suggestions for how to break away from these mistaken beliefs.<\/p>\n

Belief in Measurement (Without Thinking About What We\u2019re Measuring)<\/h3>\n

Historically, when market researchers wanted to measure a construct, such as how consumers feel about a particular brand (for example, \u201cbrand love\u201d), they would ask respondents to rate questions that directly describe the construct, such as \u201cHow much do you love this brand?\u201d<\/p>\n

This kind of \u201cmeasurement by describing\u201d has its share of problems. For instance, many constructs are too abstract for regular consumers to report on in concrete terms. Think about how you\u2019d reply if you were asked how much brand love you have for Tide laundry detergent. Most people couldn\u2019t get more specific than reporting general approximations such as \u201ca lot\u201d or \u201ca little.\u201d<\/p>\n

Researchers have begun to move toward methods that use self-reported data in better ways. Instead of asking, \u201cHow much do you love this brand?\u201d today\u2019s best practice is to pose statements that a consumer might endorse\u00a0if<\/em>\u00a0they loved the brand. For example, \u201cdisagree\/agree\u201d statements like \u201cI would drive 20 miles to purchase [Brand]\u201d would be fully endorsed only if the respondent really loved the brand. We derive the level of the construct from behaviors respondents say they would engage in.<\/p>\n

The \u201cmeasure by deriving\u201d approach requires a deep conceptual understanding of what is being measured. But many market researchers still ask questions the old way, descriptive of what\u2019s being measured (\u201cI like the ad I just saw\u201d) rather than descriptive of derivative behaviors (\u201cI showed the ad to friends\u201d).<\/p>\n

If market researchers continue to write surveys that measure constructs overtly instead of by their derivative behaviors, the data will likely be subject to uncertainty and error that could easily be avoided. They need to put more thought into what exactly they want to know, carefully consider what behaviors should be the consequence of that construct, and develop the measure from there.<\/p>\n

Belief in the One-Question Measure<\/h3>\n

Market research is not exempt from the financial pressures of business. Cost-consciousness trickles into our work when clients ask for short surveys that cover as many topics as possible \u2014 which often results in a single question for each topic. For example, \u201cHow satisfied are you with your experience?\u201d might be the only question in a survey that assesses \u201ccustomer experience.\u201d<\/p>\n

There are a number of challenges with this kind of thinking. First, the single question might not be a unique measure of \u201ccustomer experience\u201d but instead a measure of some other construct such as \u201cagreeableness.\u201d It is not uncommon in the world of psychometrics for items to \u201ccross load,\u201d meaning they can be a measure of more than one thing. Second, assuming that a question does measure what we want it to measure, it rarely ever measures all aspects of a construct. In the example of customer experience, many aspects \u2014 including price, promotions, interaction with employees, and perceptions of the brand \u2014 join to influence\u00a0an individual\u2019s global impression of his or her experience<\/a>. Asking just one question fails to assess these multiple aspects and neglects the variations in people\u2019s impressions.<\/p>\n

As well, limiting a measure to only one question doesn\u2019t always allow an organization to measure the change in a respondent\u2019s impressions, because it forces a measurement ceiling on respondents. Imagine a scenario where an organization wants to test new ads by measuring the effect the ad has on brand impression. Respondents who already love the brand might have \u201cmaxed out\u201d on their measure of brand impression if they gave the brand a 7 on a seven-point scale prior to ad exposure. Even if they\u2019re shown really good ads, they can\u2019t increase their score in the post test \u2014 they can only stay at a 7. This limitation to the measure does not correspond with the way reality works: More often than not, brand loyalists\u2019 passion for a brand can be increased with ads and brand actions that resonate.<\/p>\n

Having multiple questions that measure different aspects of the same construct is a fail-safe way to make sure data-driven decisions are actually based on all of the aspects of the construct of interest. It\u2019s a way to capture all of the information that the measure can supply.<\/p>\n

Belief That All Questions Are Created Equally<\/h3>\n

Even if market researchers heed the advice to use multiple questions for each construct they are measuring, they must remember that different questions provide different information.<\/p>\n

Consider the standardized tests that are used widely in the U.S. for college admissions. The SAT and ACT are designed to include easy items that most test-takers will answer correctly and difficult items that fewer can answer correctly. Market research questions work in a similar manner. There are some behaviors that almost anyone would do whether they love a brand or not, such as read some of the brand\u2019s posts on social media. There are other behaviors that only those who really love the brand would do, such as spend a large portion of their disposable income on it or travel a considerable distance to purchase it.<\/p>\n

This point can be illustrated visually. (See \u201cEasy- and Harder-to-Endorse Statements Provide the Whole Picture.\u201d) For example, assume that the construct (also called\u00a0latent trait<\/em>) \u201cbrand love\u201d ranges from -4 to 4. People who dislike the brand are on the lower end and people who love it are on the higher end. Respondents in the middle, at 0, have a 51% probability of endorsing the \u201ceasy\u201d statement of \u201cI would read [Brand\u2019s] posts on social media\u201d and only a 6.8% probability of endorsing the \u201cdifficult\u201d statement of \u201cI would drive 20 miles to purchase [Brand].\u201d A higher level of brand love is required to endorse the more difficult statement. In this example, a respondent with a higher latent score of 1 has a 75% probability of endorsing the easy item and jumps to a 90% probability of endorsing the hard item.<\/p>\n

Marketers need to select questions and statements with a range of difficulties to ensure that useful information can be captured from all respondents.<\/p>\n

Belief in Using Scales Without Scaling<\/h3>\n

One of the most important jobs that market researchers have when doing research is to provide an interpretation of what they are measuring.<\/p>\n

One way to do this is by using norms, which are created when a researcher has collected the same measurement on a number of individuals or groups. By knowing how others score, researchers can see where an individual falls relative to everyone else. Another approach is criterion-reference scaling. For example, tests for a driver\u2019s license are criterion-referenced, where \u201cproficiency\u201d is established using a cut-off score (for instance, getting 85% of answers correct is required to pass). Rather than considering how an applicant scores relative to others, the focus is on whether the driver is proficient based on scores that are predetermined by state agencies.<\/p>\n

But while both norms and criterion-reference approaches help aid with the interpretation of a score relative to other people (via norms) or relative to some other external criterion (via criterion-reference), they do not aid in the interpretation of how an individual scores relative to the latent trait being measured \u2014 in the example \u201cEasy- and Harder-to-Endorse Statements Provide the Whole Picture,\u201d that would be how much an individual dislikes or loves a brand to begin with on the scale of -4 to 4. Pairing how individuals score on a measure with their level of the latent trait is a step that market researchers often skip.<\/p>\n

\n