5 Shopper Insight Fundamentals that still apply today

Shopper Insights as a discipline has gone through a few definitions, and practices in the 20 or so years since it was recognized. Much has changed in retail. New retail technologies and ways of selling crop up at every industry meeting and capture our attention every day.

The challenges presented by new channels and technology demand knowledge to manage success, and many people have argued that they dictate completely new approaches to research. In the rush for the new, there are concerns that proven fundamentals may fall by the wayside. Even in complex and innovative times, we should not be so quick to reject them. They are still productive.

Fundamental #1: It Is Not Just About “Stores”

A Shopper insight is learning at the point of purchase that can shape, influence and refine Marketing and Sales Strategies. Shopper research produces those insights. There are plenty of other definitions, however, it probably is not worth spending much time dissecting or specifying your definition to perfection. Although having one does help an organization know what it is talking about, and why it requires focus. Shopper Insights remains a singular discipline within the practice of Market Research.

The point of purchase itself refers to any area where a transaction happens. Note that the definition is not specifically about a “store”. The Point of Purchase is any place that is designed to capture the attention of potential buyers and convert that attention to a sale. It should manifest the marketing and sales strategies of sellers and “store” owners. It may be a store, an e-commerce website, a community market exchange, a restaurant, a bank, or even a doctor’s office. The principles of shopper insight apply to any area where a transaction takes place. I’ll refer to them all as stores for ease.

Fundamental #2: Shopper v Consumer is an Important Distinction

This definition also implies a clear delineation between shoppers and consumers and between shopping and consumption. Although obviously related, the targets and their psychology are different. The focus of shopper insights on purchasing per se dictates different questions, business challenges, and, sometimes, different focus. At the most fundamental distinction, shoppers and consumers can be entirely different people and not just have different mindsets and goals. Pets and infants, at the extreme, never shop for their own food and toys. Shopping and consumption are different behaviors and require different approaches.

There are two common questions about this distinction:

1. Do I need separate people to manage shopper insights versus consumer?
2. If I do need separate people, where should they report in the organization?

The answer to the first question is “Not necessarily”. There are some variations in skillset, but generally, insights skills travel well between the two disciplines. So the answer really depends on the needs and culture of the organization.

The answer to #2 is “It depends”. Research shows that reporting structure is not related to shopper marketing productivity. The Conference Board and several of the large management consulting firms have indicated that while lots of organization structures exist, none has an effect on productivity. Results show the critical variables for any insights discipline, to be the focus, and, most importantly, curiosity. If the organization or individuals are curious enough, any reporting structure can be productive.

Zundamental #3: Questions Are Really Important

The questions generated by that curiosity are the key drivers of valuable insight.

These are my BIG strategic shopper insights questions:

1. What drives category growth?
2. What is the size of the prize?
3. Who are the key category targets?
4. What are those target’s wants and needs?
5. What is the sequence of shopper experiences that lead to purchase?
6. How and where can we influence that path to purchase?
7. How to optimize the reach and content of shopper communication to be more influential?
8. How does the purchase environment affect expectations and outcomes?
9. How to ensure the maximum impact of Point of Sale programs on key indicators?
10. How can retail and brand equity interaction be leveraged to mutual benefit?

These are enduring strategic questions, as relevant today as twenty years ago, and as relevant to new channels as old. They are the most frequent follow-up requests from my lectures, resonating with my audiences. It seems obvious that, as a discipline, we have not finished answering them.

Why Questions are Important

These questions cascade into everything on the shopper insight agenda and dictate the methods used to find answers. They bring life and order to curiosity and refine the ability to understand and solve your business challenge. The methodologies should be chosen after comparing their ability to meet the goals with the resources available.

Getting the question right leads to deeper insights and more innovative solutions. I designed and co-teach a course on how marketing and salespeople can become more insightful. We do not teach research techniques, but focus on asking the right kinds of questions to inspire; find potential root causes; stimulate and provoke; with researchable answers. If you want to improve or start a shopper insights group, start with the quality of your questions, not with techniques.

Here is an inspiring list of shopper questions from Kate Newlin’s book, Shopportunity!: How to Be a Retail Revolutionary (2006):

“I want to understand why we’re willing to spend $3.50 for a latte at Starbuck’s, but bristle at a 10-cent increase in the price of toothpaste. I want to learn why we’ll drive miles out of our way to buy a bag of 100 razor blades for 50-cents less than at our local store, and then spend $3.99 on a tub of pretzels we didn’t know we needed and that we’ll resent having eaten once it’s gone.”

I’d like to know those answers too. They are inspiring shopper insights questions. They are precise, researchable, energizing, and generalizable to other situations.

Fundamental #4: Existing “Tools” for Knowing Shoppers Are Good If You Know How to Use Them

Shopper Insight tools are similar to those of other disciplines but often applied in a different way and emphasis. The shopping environment plays an important role in usage & results.

I prefer this higher-level grouping of shopper insights tools:

A. Observation
B. Conversation
C. Measurement
D. Experimentation
E. Modeling

All are important and all have demonstrably led to increases in our understanding of shopper motivations and behavior. Not every problem requires every tool, but the use of every tool adds to the general knowledge of shopping.

Observation

This is the fundamental technique of Shopper Research. Relied on more often than in any other insights discipline. Despite the oceans of scanner data or click patterns, there is no substitute for observing your shoppers. Sometimes observing is enough by itself.

The medium is irrelevant (live or filmed). Seeing the action is essential. In our course on becoming more insightful, we show videos, and simply ask, “What did you see?” Invariably, a video of a child and his father shopping for breakfast cereal will quickly bring forth product placement or POS material critiques versus a description of the activities seen. Only after multiple “tell me what you see” requests does the class realize their error. Only then do they note who the real shopper is (the child), and the limitations posed by the child’s height (can’t see top shelves). Quieting the inner expert and ego is key to really seeing and then discovering profound shopper insight.

Conversation

There are many ways of talking to shoppers (1:1 IDI’s, focus groups, projective techniques, surveys, “shop-a-longs”). All have limitations, but all are also valuable for certain shopper insight problems.

In Shopportunity, we see the author’s search for deeper truths about shopping behavior. She conducted Shopper interviews to ask hypnotized respondents to describe in detail their most “memorable & powerful shopping experience”. The majority of women recounted the purchase of their wedding dress. Most men shared the purchase of their first car.

Kate found the ideal, super-powered shopping experiences against which all other shopping is compared. These experiences share psychological components: Anticipation, Pursuit, Prominence, and Appreciation. High levels of anticipation built for years; the perfect item is hunted and located with some difficulty. Once attained, the purchase gains status, is treasured, and gives back in memories for years. It is a powerful story whose significance is buried in the subconscious but revealed through conversation with the shopper.

The availability of massive amounts of data leads some researchers and marketers to deride these data, calling the stories lies, exaggerations, or the result of faulty memories. “Shoppers don’t know or remember why they choose, so the data are unreliable at best and mendacious at worst.” They advocate passionately for the abandonment of these tools in favor of “more objective” methods.

Tools are valuable if you know how to use them properly. A well-designed survey, with appropriate benchmarks and analytics, is still a source of insight. Used properly it will reward, otherwise, it is like using a screwdriver to drive a nail. It isn’t the screwdriver’s fault.

Measurement

There is no shortage of shopper things to measure, from stores, sales material, and respondents. As usual, deciding which measures count depends on the question. As well as who is asking (buyers or sellers, retailers or vendors, creators or users). And how you will be collecting the data (direct behavior measurements or the result of shopping). It is hard to see the value in “Why” questions if the “What” and “How” are not adequately and accurately measured.

Some measures may prove to be extremely useful. However, the current state of measurement in shopper marketing gives truth to the Albert Einstein quote “not everything that counts can be counted, and not everything that can be counted counts.”

• Evolving technology offers opportunities to measure in new ways. E-commerce practically defined the phrase “Big Data” and shed light on the digital path to purchase.
• Similarly, tracking sensors and videos measure shoppers’ interaction with products and aisles so the physical path to purchase can be defined with precision (previously impossible).
• Nielsen or IRI reads a product selling in 80% of the stores in the country. Direct measurement in the store shows 20% of shoppers actually see the item. The correct national distribution then is ~16%.
• Designing stores for excellent shopping experiences with accurate knowledge of behavior improves annual revenue by as much as 12%.

Understanding and measuring behavior leads to improvements that help generate increased sales. Proven.

Using your question and decisions to guide measurement is a good way to sort the measurement potential. Is the measure valid? Is it accurate? Does it help to understand your question?

Experimentation

The above assertion about the potential of physical store redesign based on actual shopper behavior is real data. A large-format retailer in the NE United States rearranged an entire store based on measures and analysis of shopper preference and traffic. The principled redesign immediately resulted in a 17% increase in sales versus a nearby control store and a sustained 12% increase over the entire year. There was no other remodeling.

There are lots of ways of knowing the marketing world, but it is hard to beat a good experiment with a great control group.

Large e-commerce businesses conduct thousands of experiments every day making design decisions driven by data and confirmed by experiments. In the digital world, almost everything we experience has been chosen by experiment; by myriad shoppers choosing one option (A) over another (B). Results and adjustments are realized almost in real-time.

That approach is more expensive and tedious in the physical world, but it is still important. Evolving virtual reality technology is allowing testing of almost any physical option in a virtual setting where we can represent any store environment in a virtual manifestation of the shopping experience. In the words of Dr. Steven Needel, a pioneer in this technology, “we can conduct package tests without a package, price tests without paying, and store tests without the store.”

Experimentation should be a vital component of any shopper insight practice, and virtual testing is an essential shopper insight tool. For example, there is a long-standing theory in CPG space management, that the shelf arrangement should reflect, “how shoppers make decisions.” For our category the metric for the hierarchy of shopper decisions clearly indicated that shoppers were more loyal to a key product attribute than to brand. The theory would then dictate that the shelf should be arranged almost exactly the opposite of the current state which assumes the brand is the most important. Rearranging those shelves even in a limited test would be both logistically and financially very expensive, so we tested the idea in a virtual environment. The results of all of the virtual testing showed no difference between the two conditions. The virtual experiment both cast doubt of the practice based on the theory, and prevented us from any expensive upheaval at the shelves.

One defends virtual shopping by using the same argument for surveys. Yes, VR is not real. Yes, respondents don’t spend real money. But despite those seeming limitations, shopping decisions in a virtual store mostly correlate with shopping in a real environment. Item shares match. Trial and repeat patterns validate. Even purchase dynamics largely agree. These validations are regularly published.

Properly trained and prepared, virtual store shoppers provide us with valuable and valid data to guide decisions. What limitations we know are pretty easily avoided or managed. Purchase decisions involving heavy items, and shopping where any sensory experience besides visual is important can be a limitation. But for the most part virtual reality shopping is a valid, indispensable tool, for experimentation.

Modeling

At their best, models are ways of thinking as much as they are an approximation of reality. Whether physical or mathematical, models are a way to make sense of a complex world, and shopping is a complex world. There is lots of evidence that the use of models improves decisions. In fact that is evidence that using lots of models helps even more.

Models help us organize information, make better and faster decisions, and develop and adopt more effective strategies. Given the amount of data in the shopper discipline, it should be no surprise that models are useful. However, there is no question that among all of the described ways of knowing about shoppers, model building is the most complicated and requires much technical skill.

The model’s ability to rapidly simulate a market or environment and reveal the consequences of a change or disruption is invaluable. Pepsi built a model of workplace beverage consumption that illustrates this. The workplace shopping environment is data-poor. Beverage sales can only be captured through survey or diary methods. We modeled a 35-employee business whose office was in an office park to understand potential influences of consumption and purchase. The model development allowed for continuous comparison between our hypotheses from survey analytic work, real-world observations, and the model. Model assumptions were adjusted to gain harmony among the sources. We then simulated the effect of different marketing and sales tactics on outcomes.

Prior to modeling, most research identifying characteristics of workers, demographics, and consumption patterns implied that the only way to grow was to make access to beverage consumption as ubiquitous and easy as possible.

Modeling simulations revealed a surprising tactic. The optimum vending strategy was to place machines in the building’s lobby, not close to the offices. As few as 2 machines in the lobby accounted for as much consumption and sales as 4 times that number on the office floor. The maximum exposure to building traffic three times a day drove more consumption and sales than the convenience of having the machine on the floor. Subsequent field-testing confirmed the strategy. One model overcame years of assumptions about the best way to deploy vending machines.

Frankly, it does not always work that way, but models remain an essential tool for shopper work.

Fundamental #5: Long Term Value

In some cases, we might prioritize accurate prediction over understanding. This is the situation for many digital A|B tests. An algorithm or a test uses the data to predict whether one alternative is better than another. The prediction may produce no insight as to why the result is true, and in the world of just in time digital ad delivery, the results of an A|B test run by a computer, evaluated by a computer, and automatically selected by a computer, the prediction is often judged enough. While this process may be efficient, it is a big change from the past where management understanding was a key to success. The need to understand is a foundational reason for market research itself.

Prediction and understanding are related but independent goals of Market Research, and it is possible to separate them. It is a business decision. The complexity of Big Data and the algorithms that read it have increased the frequency with which predictive models take precedence as more marketing occurs in digital environments where automation is possible and desirable.

Striking the right balance between prediction and understanding is still required for shopper insight. While accurate and timely prediction is often essential to leveraging opportunity, understanding is as often the foundation for innovation. Ancient Babylonian mathematicians were extremely good at predicting eclipses and other astronomical phenomena, but it was not until 800 years later that Ptolemy began to understand how the planets and sun functioned in the solar system. Ptolemy’s insight led to far more innovation than just knowing when an eclipse would occur. Understanding why a result was achieved is important.

These overall processes and questions of shopper insight endure, but smaller, tighter, seemingly important, and urgent questions appear, driven by perceived crises, fire drills, and novelty. Despite the trend, eCommerce is still in its infancy for many categories, and its knowledge base is fairly shallow. At an advisory meeting for the fast-growing e-commerce arm of a large retailer, the head engineer reported finding they were continuing to deliver ads to customers who purchased expensive products post-purchase. He announced they were discontinuing that “inefficient” process immediately. A few veterans informed him that continuing to view advertising after big-ticket purchases is common across many categories and cutting them off would hamper the store’s ability to cement loyalty and convince the shopper they had made the right decision. The engineers had no exposure to wider learning about shopper behavior.

The current situation is forcing the accelerated adoption of online shopping tools and generating much anxiety about the collective uncertainty of the future. Although we will never know everything, we can use these fundamentals to ask good questions and to know enough to make good decisions.