North Bay Mental Health Orientation with PhotoVOICE: Evaluation Report

Purpose

In the spring of 2009, Opening Minds issued a Request for Interest (RFI), seeking existing programs aimed at reducing stigma among its initial target groups of healthcare providers and youth. The North Bay Regional Health Centre responded to this RFI and entered into a partnership with OM. The North Bay Regional Health Centre (NBRHC) is an amalgamation of two hospitals, one of which was a mental health hospital. Stigma reduction was identified as a priority for the new hospital. NBRHC was interested in learning whether the one-hour mental health component embedded within the hospital’s orientation program for new staff was effective at reducing stigma against mental illness.
A three-day hospital orientation is mandatory for all new hires. One component of this larger orientation is a one-hour session on mental illness. The mental health session had three main elements: the screening of a documentary showing participants involved in NBRHC’s PhotoVOICE program, a music video featuring a song written and performed by a former patient, and a personal story of mental illness and recovery, followed by questions and answers with the audience. For most sessions, the personal story was told by the session facilitator, a peer support specialist. If the peer support specialist was not available to speak in person, a digital recording of a personal testimony was shown to participants.

OM conducted an evaluation of the NBRHC Mental Health Orientation with PhotoVOICE Session, which was delivered to approximately 185 new staff at NBRHC throughout the months of August to December, 2012. Further details on the methodology used for this evaluation are provided below.

Methodology

In order to assess attitude change towards mental illness, orientation participants were given a questionnaire package at three different time-points. The first survey was completed before the initial intervention (pre-test survey). The second questionnaire was given to participants immediately following the completion of the one-hour mental health component of the orientation session (post-test survey). The final survey was administered electronically, three months following participants’ attendance at their orientation (follow-up survey).

The pre-test survey contained the 20-item Opening Minds Scale for Health Care Providers (OMS-HC), questions pertaining to experiences with mental illness, and demographic questions (age, gender, training, and professional status). For the post-test and follow up surveys, participants completed the 20-item OMS-HC again so that changes over time could be assessed. They were also asked to indicate which aspect of the mental health component of the orientation session affected them most, and why.

The OMS-HC is a 20-item questionnaire that measures healthcare providers’ attitudes towards people with a mental illness. To complete the scale, participants are asked the extent to which they agree or disagree with each item. Items are rated on a 5-point scale: strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree. A copy of the OMS-HC scale is included as Appendix A.
To create a total scale score for the OMS-HC, all 20 items are summed for each participant. Total scores can range from 20 to 100, with lower scores indicating less stigma. For this particular study, Cronbach’s alphas for the scale were .79 at pre-workshop, .76 at post-workshop, and .73 at follow-up, indicating an acceptable level of internal consistency for the OMS-HC scale.

Paired t-tests were used to analyze total scale scores. Also, by grouping certain questions from the scale together, the OMS-HC can be used to examine three main dimensions of stigma: attitudes towards people with mental illness, healthcare professionals’ attitudes about disclosure of a mental illness, and social distance. A threshold was also created to measure success, defined as the proportion of respondents who obtained 80% or more correct (non-stigmatizing) answers on the post-test.

Key Findings

In all, 182 of the 185 participants completed one or more of the evaluation surveys. Demographic characteristics of survey respondents are highlighted in Section 4.1 below.

Analysis of OMS-HC score change from pre- to post-orientation was performed based on a total of 177 paired pre and post surveys (178 pre-test surveys and 181 post-test surveys were completed). These results are highlighted in Sections 4.2 to 4.4 below. Section 4.5 details participants’ feedback about the orientation session, while Section 4.6 highlights OMS-HC differences by participant type.

The response rate was lower for the three-month follow up survey, with a total of 62 completednsurveys. Given the lower response rate for the follow up survey, results for this component of the study should be interpreted with caution. Follow up survey results are described in section 4.7. These results are not based on a paired analysis.

Individual item scores for the OMS-HC at all three time points are provided in the various data tables in Appendix B.