RE-AIMing e-Mental Health: A Rapid Review of Current Research

Purpose

Executive Summary

Decision makers and service providers in the mental health sector need up-to-date, evidence-based information to guide them in shaping how e-mental health care can be optimized in Canada. Guided by the strategic priorities of the Mental Health Commission of Canada (MHCC), the purpose of this rapid review was to map recent research on e-mental health interventions and describe trends in the factors that either catalyze or deter implementation. Using the RE-AIM Framework, 261 relevant articles were located and systematically reviewed around the themes of reach, effectiveness, adoption, implementation and maintenance. The results point to opportunities and gaps in the current research literature and provide five key recommendations on where capacity-building work and new lines of research inquiry are most needed, namely: scaling up existing services; reflecting priority populations; co-defining and measuring costs and benefits; finding balanced solutions; and advancing system integration.

Overview

High rates of mental health problems and low levels of access to traditional mental health services (i.e. delivered one-on-one in clinic settings), continue to be cited as reasons to more quickly integrate innovative approaches to providing services. This, combined with exponential growth in communication technologies and development of effective digital interventions to address mental health, has made improving the design and application of e-health technologies a mental health priority of national concern. In response, in Canada’s first mental health strategy, Changing Directions, Changing Lives, the MHCC recommended increasing “the use of tele-mental health and e-mental health by building better infrastructure, providing on-going training and support, and greater flexibility in how services are funded.” Today, technology allows people to work through coping strategies for depression with the help of a chatbot, face a phobia using a virtual reality headset, or learn how to manage anxiety through a self-guided online program. Booking appointments online, video calls, or text messaging with therapists whom you do not ever have to meet in person, are all now possible pathways to mental health care. Not unlike e-health services generally, the uptake of e-mental health care in Canada still faces certain limitations.

These include the fact that such interventions:

• Are complex, i.e. consist of several interacting components, target different groups, have multiple and variable outcomes, and/or permit personalization or tailoring.
• Are context-dependent i.e. are influenced by the rapidly changing technologies that are available and used by the public and providers.
• Perform differently depending on the way they are implemented.
• Have different effects on different individuals.

Furthermore, e-mental health research typically assesses and appraises interventions in head-to-head comparisons, while decision making needs an integrated perspective on the value of a technology within the larger system. Given these levels of complexity, it is important to document and synthesize available information as fully as possible and to assess if and how this may affect the uptake of e-mental health services.

Goals of the Scoping Review

Despite the clear potential of digital technology to connect people and health services in new ways, evidence suggests that this potential is not being fully realized in Canada.The purpose of this rapid scoping review was to provide a comprehensive overview of current understandings about e-mental health interventions, so that decision makers and service providers have up-to-date information to guide them. This review is an important tool for synthesizing a vast amount of information about the factors that could either catalyze or deter implementation of e-mental health interventions and services in Canada. The high-level questions guiding the review were:

• What is the evidence on effectiveness and cost-effectiveness of e-mental health in treating mental illness or substance use problems?
• What factors may influence the effect of e-mental health on health outcomes?
• What is the evidence of effectiveness of e-mental health for specific population groups (e.g. children, youth and emerging adults, and minority groups such as First Nations, Inuit, Métis; and/or other population groups)?

Methodology

This review was structured according to the Arksey and O’Malley five-step approach:

1. Clarify the scope of the review
2. Search for evidence
3. Appraise the literature and extract data
4. Chart the data
5. Collate, summarize and report

Below we describe in general terms the procedure that we followed at each step.

Clarify the scope

To clarify the scope of the rapid review, our team met and consulted with staff at the MHCC to review the key research questions and previous work in the area. We were especially interested in understanding how this rapid review could provide usable and actionable information for policy makers and practitioners who often find it challenging to make decisions regarding investing—or disinvesting—in e-mental health interventions. These tend to be complex and comprise multiple components, which may act independently or interdependently, offering no means of truly discerning or isolating the “active ingredient.” The MHCC hosted a national roundtable on e-mental health in January 2017 to provide a forum for stakeholders to ask questions, discuss the issues and challenges they encounter, and describe the ways in which summary information would be most useful to them. We attempted to incorporate these perspectives in conducting our review.

Search for Evidence

As an update to Lal & Adair, (2014)14 this rapid scoping review synthesized national and international research published from 2010 to 2016 inclusive. Since this review was conducted within an abbreviated time frame, we did not attempt an exhaustive search for all evidence; instead, we captured a comprehensive snapshot of research in the field. We included only English-language articles about studies that evaluated a technologybased health intervention. Interventions could be user-led and did not require clinician involvement. Studies had to involve at least 10 participants. We excluded manuscripts that were categorized as a review, commentary, theoretical paper, published abstract, dissertation, book chapter, design paper, research protocol, or editorial—or if it did not report on empirical or evaluative data. An information specialist constructed, pilot-tested and ran our search strategies for each database. Our team defined the search strategies by consensus, informed by important MeSH (medical subject headings) terms and key words from previously published systematic reviews in the field, as well as by relevant terms for mental health conditions and the technology used (search terms are provided in Appendix 1). In November 2016, we searched 15 major databases (e.g. MEDLINE®, CINAHL®, EMBASE, ProQuest Theses and Dissertations, OVID HealthStar, Cochrane Database of Systematic Reviews, Health Technology Assessment Database, PsycINFO®, ACM Digital Library, IEEE Library). The literature search results were uploaded to EndNote X7 and duplicates removed.

Appraisal

To identify which studies to include in our review, we involved two independent reviewers (two research assistants) to correctly apply the inclusion/exclusion criteria using a two-step process. At the title, abstract, or descriptors-level of screening, the reviewers independently screened citations to identify potentially relevant articles for full review. At the full-text stage of screening, 40 potentially eligible articles were randomly assigned to the pair of investigators to screen independently. Pairs of coders met to resolve discrepancies and reach consensus. The research team met for weekly progress updates and to resolve discrepancies.
The methodological quality of the studies we included varied considerably, as evidenced by their differing analytic approaches, sample sizes and study designs. We decided that all studies, even those of poorer quality, might offer important insights into broad trends in e-mental health research that could aid in informed decision making. Therefore, all studies that met inclusion criteria were included in the synthesis.

Chart the Data

We created a data-charting form to collect and sort key pieces of information from the full text of each study, using the RE-AIM Framework. Data extracted from the literature included standard study information (such as author, year of publication, sample size) and additional information to examine mental health populations and outcomes (such as target population, type of technology, type of intervention, treatment outcomes, and treatment effect). The independent coders and one of the senior investigators undertook a trial charting exercise and team consultation to ensure consistency with the questions and purpose—in keeping with the suggestion of Daudt et al.30 For a subset of 20 articles identified for inclusion, we extracted data from evidence sources (research assistant #1), which were checked for accuracy and completeness by a second reviewer (research assistant #2).

Collate, summarize and reportIt is important to note that, unlike a typical systematic review with stringent inclusion/exclusion criteria, we aimed to include and analyze a broad range of research. This limits our ability to provide detailed analysis of individual programs or studies, but allows us to present a few relevant trends. We used a narrative synthesis of the findings to report results derived from this review.