If you are in distress, you can text 988 at any time. If it is an emergency, call 9-1-1 or go to your local emergency department.

Assessment Framework for Mental Health Apps

4. Clinical Safety Standards

The functionality of any software product, and the ways it is represented or labelled for use, dictates whether it qualifies as a medical device under the Canadian Food and Drugs Act and the Medical Devices Regulations, administered by Health Canada.

Software that has no direct impact on the diagnosis, treatment, or management of an individual’s disease, disorder, abnormal physical state, or symptoms would not be subject to the Medical Devices Regulations. Health Canada’s Software as a Medical Device (SaMD) guidance document can assist manufacturers in determining whether their product is a medical device.

If a software product is identified as a medical device, its risk classification must also be determined to know which regulatory requirements apply. (Medical devices are classified under four classes, with Class I representing the lowest risk and Class IV the highest.) For example, to be sold in Canada, Class I devices do not require a medical device licence, whereas Class II, III, and IV devices do require one (Note: Class I manufacturers, importers, or distributors also have regulatory requirements).

Many mental health apps perform functions that would potentially bring them under the Medical Devices Regulations in Canada. These functions include apps that could help treat, alleviate, diagnose, or prevent mental health-related symptoms. These parameters could impact many of the leading apps in this area, and excluding them from this framework would severely limit its reach.

The approach typically adopted in jurisdictions such as the U.K. and the EU is to ensure that the applicable framework can identify whether or not an app is a medical device under the relevant local regulations, and if it is, to validate whether or not it has been appropriately certified or authorized under those regulations.

This approach is not about bypassing the relevant certification/authorization process. It simply seeks to validate whether that process has already been completed through the establishment of an appropriate mark, certificate, or authorization (i.e., in the Canadian context, a medical device licence).

This is the approach we are proposing for the current assessment framework. Such a strategy would allow apps to be potentially classified as medical devices and would also enable us to identify any without suitable authorization. Any apps deemed to be medical devices without Health Canada authorization would be rejected from the framework process and referred to Health Canada for resolution.

The only other practical route to avoid the risk of inadvertently including unauthorized medical devices in the framework process would be to explicitly exclude apps with relevant functionality from the start. As noted, however, this would severely limit the types of products that could be assessed via the framework. The proposed step-by-step process we would follow if we allow relevant apps into the process (including the criteria) would be as follows:

 

Criteria

Criteria Origin

4a — Q1

Is the application a medical device that is subject to the Medical Devices Regulations according to Health Canada’s Software as a Medical Device guidance document?

If 4a-Q1 is answered no (the application is not intended for one or more medical purposes), then Q2 does not need to be answered.

ORCHA
4a — Q2

If the application fits the criteria for a Class II medical device or higher, is the application authorized for sale in Canada?

  • Health Canada’s Medical Devices Active Licence Listing (MDALL) can be consulted to determine whether a product has an active licence.
  • A medical device establishment licence[1] is issued to Class I manufacturers as well as importers or distributors of all device classes to permit them to import or distribute a medical device in Canada. An MDEL provides Health Canada assurance that procedures are in place to protect the public should a problem with a device be identified.

MHCC

Please note the following:

  • Health Canada does not maintain a list of products (e.g., specific medical devices) under an MDEL.
  • An MDEL does not constitute approval of any specific medical devices imported or distributed by the MDEL holder.

If it appears that the app is subject to the Medical Devices Regulations (i.e., it is a medical device) but has not been authorized by Health Canada, any advertisement, sale, or distribution of the app in Canada would be prohibited. In such cases, the app manufacturer could contact Health Canada’s Medical Devices Directorate.

[1] A medical device establishment licence is issued to Class I manufacturers as well as importers or distributors of all device classes to permit them to import or distribute a medical device in Canada. An MDEL provides Health Canada assurance that procedures are in place to protect the public should a problem with a device be identified.

Developers should consider the risks that come with making a mental health app to ensure that it can be used safely. Because Canada has no formal standards to guide developers in creating safe mental health apps, this framework proposes that they adopt the ORCHA Clinical Safety Assessment (OCSA), whose criteria objectively outline safety considerations app developers could adhere to. Apps that comply with the ISO 14971 (a framework for risk management) and DCB 0129 (a clinical risk management framework) would also pass the OCSA standards. Not only is the OCSA is aligned with the established risk management frameworks for health apps, its criteria have been tailored to the Canadian context (as shown in the criteria below).

 

Criteria

Criteria Origin

4b — Q1

Is the app/solution in scope of the ORCHA Clinical Safety Assessment (OCSA)?

ORCHA

4b — Q2

Why is the application in scope?

ORCHA

4b — Q3

Has the developer provided a thorough summary about why the application is out of scope (e.g., it is complete and matches the functionality put forward)?

ORCHA

4b — Q4

What risks, if any, have been documented concerning harm to a user?

ORCHA

4b — Q5

If the application/solution was deemed to be in scope, has the developer supplied suitable risk management documentation?

ORCHA

4b — Q6

Is there risk management documentation?

ORCHA

4b — Q7

Does the developer outline the need for risk management documentation?

ORCHA

4b — Q8

Does the risk management documentation have a full version history and issue date published?

ORCHA

4b — Q9

Has the developer described their clinical risk management system (e.g., identification of key personnel, their roles and responsibilities, identification of clinical risk management governance structure)?

ORCHA

4b — Q10

Does the safety case mention a test summary (i.e., a summary of any outstanding test issues and their impact on clinical safety)?

ORCHA

4b — Q11

Does the hazard log have a full version history and issue date published?

ORCHA

4b — Q12

Are the hazards listed complete?

Guidance:

From a review of the hazards listed, have all the required details been completed, such as name, clinical impact, and risk ratings?

ORCHA

4b — Q13

Are the potential harms related to the user?

ORCHA

4b — Q14

Do the harms outline what the clinical impact may be for the user?

ORCHA

4b — Q15

Has the developer covered all of the possible causes for each hazard?

ORCHA

4b — Q16

Do the risk ratings look appropriate or do they appear to be “copy and paste” throughout the listed hazards?

ORCHA

4b — Q17

Are hazards split incorrectly into potential and actual harm?

ORCHA

4b — Q18

Has the developer implemented the clinical risk analysis activities defined in the clinical risk management plan?

ORCHA

4b — Q19

Is the clinical risk analysis carried out by a multidisciplinary group?

ORCHA

4b — Q20

Has the developer defined the clinical scope of the health IT system to be delivered?

ORCHA

4b — Q21

Has the developer defined the intended use of the health IT system to be delivered?

ORCHA

4b — Q22

Does the risk management documentation make clear where the application fits into the clinical workflow?

ORCHA

4b — Q23

Has the developer outlined third-party products integrated within the health IT system to be released?

ORCHA

4b — Q24

Has the developer deploying the health IT system considered how it will impact current business processes and ways of working?

ORCHA

4b — Q25

Is there usability- and human-factor-related evidence within the scope?

ORCHA

4b — Q26

Has the developer assessed any infrastructure at the health organization within their scope of influence that is required to support the deployment of the health IT system? (This may be achieved by the manufacturer specifying the minimum system requirements.)

ORCHA

4b — Q27

Is the developer undertaking any data migration? (Where data migration is to be undertaken by the developer, it should be included in the scope of the clinical risk management activities.)

ORCHA

4b — Q28

Is the data migration being undertaken by the developer properly covered in the documentation?

ORCHA

4b — Q29

Has the developer identified any hazards associated with the data migration that has been analyzed and suitably mitigated (working in conjunction with the relevant health organization, as appropriate)?

ORCHA

4b — Q30

Has the developer considered the end-to-end clinical process, including functionality and how that functionality is used?

ORCHA

4b — Q31

Has the developer considered inter- and intra-health IT system messaging?

ORCHA

4b — Q32

Has the developer assessed the health IT system’s architecture and design?

ORCHA

4b — Q33

Is a clear matrix used to define the risk ratings?

ORCHA

4b — Q34

For each identified hazard, has the developer evaluated whether the initial clinical risk is acceptable?

ORCHA

4b — Q35

Has the developer used the risk acceptability criteria that was previously defined?

ORCHA

4b — Q36

Has the developer identified appropriate clinical risk control measures to remove any unacceptable clinical risk?

ORCHA

4b — Q37

Has the developer assessed proposed clinical risk control measures to determine whether new hazards will be introduced as a result of the measures?

ORCHA

4b — Q38

Has the developer assessed proposed clinical risk control measures to determine whether the clinical risks for previously identified hazards will be affected?

ORCHA

4b — Q39

Is the developer managing new hazards or increased clinical risks?

ORCHA

4b — Q40

For each identified hazard, has the developer evaluated whether the residual clinical risk is acceptable?

ORCHA

4b — Q41

Has the developer used the risk acceptability criteria previously defined?

ORCHA

4b — Q42

If the residual clinical risk is unacceptable, has the developer identified additional clinical risk control measures to reduce the clinical risk?

ORCHA

4b — Q43

If the developer has determined that no suitable risk control measures are possible, have they conducted a clinical risk benefit analysis?

ORCHA

4b — Q44

Has the developer’s analysis shown that the clinical benefits of the intended use outweigh the residual clinical risk?

ORCHA

4b — Q45

Has the developer implemented the identified clinical risk control measures (except where these are to be implemented by another organization)?

ORCHA

4b — Q46

Have the clinical risks from all identified hazards been considered and accepted?

ORCHA

4b — Q47

Have any hazard rating reductions been fully justified?

ORCHA

4b — Q48

Is there a professional involved in the clinical safety process?

MHCC

4b — Q49

Is the relevant professional suitably qualified?

ORCHA

4b — Q50

Does the relevant professional have appropriate registration details?

ORCHA

4b — Q51

Have these registration details played an active part in the process?

ORCHA