Listening to the recent Freakonomics podcast “Policy-Making is not a Science (Yet)” encouraged me to stop and think more carefully about some of the research conducted within the Food Focus team. Broadly, we are focused on small-scale or regional issues looking at how people change their behaviour when some kind of intervention is introduced. My own research looks at how individuals react to different information presented on food products. While some of the consumer research we conduct has some obvious restrictions, for example, many of our research participants are students, we do our best to recruit a representative sample. This means recruiting beyond campus boundaries and targeting a diverse group of people. Guelph has been described as a good test population as its population is fairly representative of Canada’s overall demographics. However, we inevitably deal with different participant biases and the potential that the individual characteristics we tease out of the data are not representative when rolling out programs on a large-scale.
The reason some programs fail in the ‘real world’ is not necessarily because they are ineffective or ill-intentioned. The reality is that real world circumstances and participants tend to be different than the context created within a research study. This causes differences in the way the program is run and therefore impacts the overall results. It doesn’t necessarily mean that the experiments or results from the pilot projects are flawed. However, it does mean that more thought must be put into how and why they were successful in the first place and how these results might translate to large-scale implementation (Dubner, 2020).
Our experiments are conducted in a lab that is set-up like a grocery store. Grocery store aisles have been installed and the shelves and fridges are stocked with a variety of food products. We use eye-tracking glasses to examine how people interact with product information as they shop within a realistic grocery environment. We have conducted research looking at various information and labelling schemes in the following areas: genetically modified and engineered food labels, local labels, price changes, suboptimal fruits and vegetables (food waste) and carbon footprint labelling. This is typically done in terms of adding labels to product packaging, signage in the lab or price changes.
In these consumer experiments I hypothesize that we may have muted results as we don’t simulate education or prior knowledge of any new labelling or information we are attaching to the products. We rely on the observational skills of our participants to notice this new information. Outside of a testing environment, there would be a marketing or educational campaign launched to accompany any new information added to product labels. This changes the dynamic between the participant and the information. Ultimately, this presents some issues when attempting to scale up the research or intervention, and what unintended consequences may result.
These issues make it difficult to not only predict what these unintended consequences might be, but also how the intended impacts might be affected. For example, consider a sustainability label designed to slowly push people towards making more environmentally friendly choices. This label is going to potentially create a premium for products deemed to be environmentally friendly. This may lead to fewer people being willing to switch to those products or, worst-case scenario, existing consumers leave. These are maybe extreme cases but all possible, nonetheless.
Most of the research we conduct at the University of Guelph is more so intended to bring an element of realism to an otherwise survey dominated field. This isn’t to disparage surveys; they are extremely useful in their own right, but using a realistic environment provides our research with an attempt to avoid or try to tackle the hypothetical bias issue.
It is a case of doing the best with the tools available and pushing the field forward into a more realistic environment when possible. This will hopefully make the policy and real-world implications more relevant.
References
Dubner, S. “Policy Making is not a Science (Yet)” Freakonomics (405), Freakonomics, Feb 12th, 2020 https://freakonomics.com/archive/
Recommended Citation Format: Baynham, A. "The Problems with Scale". Food Focus Guelph (77), Department of Food, Agricultural and Resource Economics, University of Guelph, March 30th, 2020.