Contact Us
COVID-19, Evidence-Based Decision Making, and Rapid Change: Adapting programs and policies in uncertain times
The world around us has changed rapidly in the wake of COVID-19. All levels of government have mobilized responses to a complex problem, signifying the incredible impact we can have when we are able to implement change swiftly. Rapid change that flattens the curve and saves lives is nothing short of inspiring.
This change should also be encouraging to stakeholders working in the gambling harm reduction field. There is a significant opportunity for harm reduction programs and policies to be created or modified in a fast and responsive manner to meet the unprecedented risks of, and complexity of, harms during COVID-19. Inspiration alone, however, might not curtail the trepidation of stakeholders who find themselves in unfamiliar waters, trying to navigate the chasm between acting quickly and upholding the importance of evidence-based decision making.
Under normal circumstances, where time is a luxury afforded to us, evidence-based decision making often consists of in-depth research reviews, stakeholder consultations, and jurisdictional scans, among other efforts. Additionally, approval processes for policy or program change may be lengthy, with significant lag time between change inception and implementation. There is no doubt that, just as we are seeing in other sectors around the world, the gambling sector also needs to change quickly to be responsive to the impact COVID-19 has on its harm reduction programs, policies, and initiatives. The question is, can evidence-based decision making and rapid change co-exist? The answer is yes.
Think about a light bulb as a metaphor for how we can blend evidence, with the need for rapid change, to create impactful change that can be mobilized to benefit the greater good. Evidence provides the base, like the electrical contact that connects you to the harnessed collective power of the world around you. It creates a point of connection, allowing you to draw from science and knowledge, and funnel it in a way that will provide insight into your problem. A change mentality, like the wires and filaments, allows you to take that evidence and adopt it to your unique context to create possibility. These possibilities, driven by innovation, dedication, and courage, make it possible to build capacity. Building capacity and creates the kind of light that shines not only for you, but also lights the way of others.
Evidence
Evidence can–and should–be the foundation for innovation, even when time prevents a thorough review of the literature. The “evidence pie” illustration below reminds us that expertise, experience, good ideas, and pragmatics, among others, are valid types of evidence we can use to inform our decision making. Remember, evidence is a complex concept, nuanced by a combination of context, personal, and scientific knowledge, value systems, and political pressure. Although we commonly talk about evidence-informed decision making in gambling harm reduction in reference to scientific research, there are many other forms of evidence we should leverage.
Adapted from: Bennett, G., & Jessani, N. (Eds.). (2011). The knowledge translation toolkit:
Bridging the know-do gap: A resource for researchers. SAGE Publications India.
In addition to looking closely at the variety of types of evidence available to us, we also need adjust our threshold of what is “enough” evidence to act on, in comparison to what our threshold for action might normally be. When do you have “enough evidence” to inform a decision, especially when time is limited, and you need to act quickly? In some ways, we never have enough evidence, because real life situations are dynamic, often putting us in circumstances we did not anticipate. Instead of asking “Do we have enough evidence?” try asking “Does the evidence we have make us feel relatively confident about the decision we’re making?”. If the answer is no, chances are you do not have the quantity (or quality) of evidence you need, in your unique context, to make your decision.
Implementing the Plan Do Study Act cycle covered below helps offset that burden of deciding when evidence is enough, by reminding us that some of the best evidence comes from doing, measuring change, and then adjusting our intervention accordingly.
Change mentality
The practice of evidence-based decision making and knowledge translation can take inspiration from the quality improvement model of Plan Do Study Act (PDSA). The PDSA cycle recognizes that moving from planning to doing, without too much paralysis in between, is the key to instigating change and continuous improvement. As Dr. Mike Evans said in his quality improvement in healthcare video, “The main point is start. Find something you can improve and get going (…) I would advise not to let what you can’t do prevent you from what you can do. It’s time to entertain complexity but focus on simplicity.”
Adopting a change mentality means embracing that the potential value of change can only be realized through action, experimentation, and reflective learning. It means advocating for change, even the though the process of change can be unpredictable, messy, disruptive, and challenging.
The Model for Improvement, developed by the Associates in Process Improvement, includes the PDSA cycle in addition to three guiding questions. The PDSA cycle features four actions that help breakdown this complex process: plan, do, study, act.
1. Plan:
Pulling from the evidence at hand, answer three questions:
- What are we trying to accomplish? Be clear on what you believe the gap and opportunity to be. What impact do you want to have and why is it important?
- How will we know if a change is an improvement? Choose specific, measurable indicators of change. Remember that “soon is a not a time and some is not a number” (Dr. Mike Evans).
- What can we change that will result in an improvement? Marrying evidence with context, using the FAME scale from the Joanna Briggs Institute is a useful tool for helping us hone in on defining change. FAME interventions are feasible, appropriate, meaningful, and effective. Remember that you will need to make this determination based on the evidence available to you. Thinking about interventions in terms of FAME can also prompt us to consider what barriers and facilitators to change exist, and how the proposed intervention considers them.
Finally, no change comes without risk, just like no accomplishment comes without sacrifice. Risk identification and monitoring can be simplified by taking these steps (adapted from Dr. David Shore’s steps for effective risk management):
- Brainstorm possible risks, including those that are influenced by both intrinsic and extrinsic factors
- Calculate the risks, assessing the likelihood of the risk occurring and the impact it would have if it did occur
- Know your risk tolerance. Identify what level of risk you are willing to accept and what signs you are approaching that threshold will look like
- Create a contingency plan. For each risk, think about how you reduce the risk or at least the impact it might have if it occurs. Planning now helps cope with risks as they emerge.
- Monitor risks over time so you can respond in real time
2. Do:
Implement the change.
3. Study:
Measure the impact of your change, looking at both process and outcome indicators, as well as unintended consequences. Don’t be afraid to adapt your course of action based on real time feedback.
4. Act:
“Adopt, adapt, or discard” the change (Dr. Mike Evans), based on the results.
Building capacity
Building system capacity, internally and externally, in the gambling harm reduction field to adapt quickly in uncertain times is only possible if we actively work to share our PDSA cycle experiences with others. Just like truncated timelines require us to move through our review of the evidence process, we also need to compress our knowledge translation cycle to disseminate learnings and diffuse innovation quickly.
Here are seven key questions you can ask to help make sense of how you can maximize capacity building under tight timelines:
- Why is it important that we share these findings quickly? What do we hope to achieve by sharing?
- How quickly do you need to need to mobilize the learnings of your PDSA cycle, internally and externally, to maximize the impact?
- Within these timelines, what resources (financial and human capital) do you have available to you to help you disseminate your learnings?
- Based on your timelines and resources, which stakeholders, internally and externally, should you prioritize in engaging?
- Based on your timelines and resources, which strategies can you use to share your learnings?
- What are the key messages, findings, and lessons learned you want to share with these stakeholders?
- How will you use the feedback and insights from stakeholders to continually improve your strategy?
Just like with any model, there is a risk of oversimplifying how difficult or complicated a process can be in real life. Moving from paper to practice requires a will to forge new paths in the absence of trail markers. Having a map of the landscape can help you keep moving in the right direction, but twists, turns, and unsure footing are still inevitable. What is important to remember is that with adversity, comes opportunity. There are light bulb moments if we look for them.
Jess Voll is GREO’s Evaluation Specialist. Contact her at jess@greo.ca.
See our Resources for Safer Gambling during COVID-19 page to browse evidence related to key topics we anticipate will be relevant to gambling stakeholders during and following the COVID-19 crisis.