1. App Overview
Navigation
This assessment framework begins with the app overview. The criteria listed below (grouped into distinct areas) aim to understand an application’s core purpose and functionality. These include the target audience, the type of data the app collects, and the app’s primary functions and features. Because these criteria do not have scoring implications, the application will not be scored based on its functionality. Instead, the criteria will effectively set the scene and inform the standards criteria that need to be posed for that specific application.
Criteria |
Criteria Origin |
|
| 1a — Q1 | Is the application mental health focused? | MHCC |
| Criteria | Criteria Origin |
1b — Q1 | Does the application collect data? | ORCHA |
1b — Q2 | What type of data is collected by the app? | ORCHA |
1b — Q3 | Are users required to sign up/register to use the service? | ORCHA |
1b — Q4 | Is data collected through cookies? | ORCHA |
1b — Q5 | What type of cookies are used? | ORCHA |
1b — Q6 | Is the data (cookie and/or none cookie) collected sensitive, personal, and/or non-personal? | ORCHA |
1b — Q7 | How is none cookie data collected? | ORCHA |
1b — Q8 | What other apps is the application connected to? | ORCHA |
1b — Q9 | What device(s) does the application connect to? | ORCHA |
1b — Q10 | Can the user prevent cookie data from being collected and still use the app? | ORCHA |
1b — Q11 | Does the disabling of cookies impact the use of the application in any way? | ORCHA |
1b — Q12 | Can/is data shared (excluding cookies)? | ORCHA |
1b — Q13 | Can data be shared through a direct, manual action by the user (e.g., by sending data via email or manually choosing to post/share something within the application)? | ORCHA |
1b — Q14 | How is the user able to manually share their data? | ORCHA |
1b — Q15 | Is data only shareable through a direct, manual action by the user (excluding cookies)? | ORCHA |
1b — Q16 | Can the user control any automatic data sharing, through setting individual sharing preferences in the app (excluding cookies)? | ORCHA |
1b — Q17 | Where/with whom can the user share data automatically by manually setting sharing preferences in the app? | ORCHA |
1b — Q18 | Is any data (excluding cookie data) shared automatically as soon as the application is accessed — based only on agreement to relevant terms and conditions (T&Cs) or privacy policy? | ORCHA |
1b — Q19 | Where/with whom is data automatically shared — based only on user agreement to the developer’s privacy policy and/or T&Cs? | ORCHA |
1b — Q20 | What data is automatically shared with the developer? | ORCHA |
1b — Q21 | What data is automatically shared with clinicians/health-care professionals? | ORCHA |
1b — Q22 | What data is automatically shared with other users? | ORCHA |
1b — Q23 | What data is automatically shared with third parties? | ORCHA |
1b — Q24 | What data is automatically shared with other devices? | ORCHA |
1b — Q25 | Is the service/provider a federal government organization listed with Canada’s Privacy Act? | MHCC |
1b — Q26 | Is the service/provider a provincial or territorial government or an agent of a provincial or territorial government? | MHCC |
1b — Q27 | Is the service/provider a charity? | MHCC |
1b — Q28 | Is the organization a not-for-profit group that does not engage in any commercial activities (e.g., sports associations, professional associations, clubs)? | MHCC |
1b — Q29 | Is the product aimed at or likely to be used by Indigenous people in Canada? | MHCC |
1b — Q30 | Will the organization collect and process personal information from Indigenous users? | MHCC |
| Criteria | Criteria Origin |
1c — Q1 | Does the application contain algorithms? | ORCHA |
1c — Q2 | How does the application use the algorithm? | ORCHA |
1c — Q3 | Does the application appear to use artificial intelligence (AI)? | ORCHA |
1c — Q4 | What AI technique is used in the app? | ORCHA |
1c — Q5 | Is the AI monitored/maintained? | ORCHA |
| Criteria | Criteria Origin |
1d — Q1 | Is the application designed to provide information or guidance? | ORCHA |
1d — Q2 | Does the application provide information that is personalized to an end-user’s specific circumstances? | ORCHA |
1d — Q3 | Does the application provide users with information regarding where they are able to find local or suitable support services? | ORCHA |
1d — Q4 | Does the application provide information, resources, or activities to the public or clinicians, either about a specific mental health problem or general health and lifestyle? | ORCHA |
Criteria |
Criteria Origin |
|
| 1e — Q1 | Is the data the application collects automatically assessed for the purposes of evaluating
|
ORCHA |
| 1e — Q2 | Does the application provide an assessment (of the risk) to an individual based on data input or collected by the application of
|
ORCHA |
| 1e — Q3 | Does the application provide an assessment (of the risk) to a health-care professional based on data input or collected by the application of
|
ORCHA |
| 1e — Q4 | Does the application provide the option for further assessment or analysis by a health-care professional? | ORCHA |
| 1e — Q5 | Is the app/does the application include a symptom checker? | ORCHA |
| 1e — Q6 | Does the application discuss suicidal ideation? Guidance: The CDC (Centers for Disease Control and Prevention) defines suicidal ideation as thinking about, considering, or planning suicide. Machine learning techniques that feature engineering or deep learning for automatic detection (based on online social contents) is one means for detecting suicidal ideation. | MHCC |
| 1e — Q7 | Does the application indicate likelihood of a match for the listed mental health problems? | ORCHA |
| 1e — Q8 | Can users filter results to display by highest risk/likelihood/severity? | ORCHA |
| 1e — Q9 | Does the application provide treatment recommendations for the listed mental health problems? | ORCHA |
| 1e — Q10 | Does the application only signpost the user to suitable care or recommend seeking further advice (e.g., Go to A&E, book an appointment with your family doctor, call 911)? | ORCHA |
| 1e — Q11 | Does the application contain a clinical calculator? | ORCHA |
| 1e — Q12 | What type of clinical calculator does the application contain? | ORCHA |
| 1e — Q13 | How does the application prevent disease? | ORCHA |
| 1e — Q14 | What treatment does the application provide? | ORCHA |
| 1e — Q15 | Does the application guide the treatment of a mental health problem? | ORCHA |
| 1e — Q16 | How does the application guide the treatment of the mental health problem? | ORCHA |
| 1e — Q17 | Who does the application provide the treatment guidance to? | ORCHA |
| 1e — Q18 | Is the treatment provided independently of a health-care professional? | ORCHA |
| 1e — Q19 | Does the application support health-care professionals’ decisions about treatments? | ORCHA |
| 1e — Q20 | Does the application follow the path of a procedure/treatment without making any decisions? | ORCHA |
| 1e — Q21 | Does a health-care professional make the final decision regarding treatment based on advice and/or options displayed? | ORCHA |
| 1e — Q22 | Does the application automate the treatment pathway for an individual user? | ORCHA |
| 1e — Q23 | Is the application intended to (or does the developer claim it can be used to) compensate a mental health problem? | ORCHA |
| 1e — Q24 | Does the application predict the fertile window? | ORCHA |
| 1e — Q25 | Does the application claim to be used to prevent pregnancy or to conceive? | ORCHA |
| 1e — Q26 | Does the application use body basal temperature (BBT), recorded through an externally connected thermometer? | ORCHA |
| 1e — Q27 | Does the application use rhythm, BBT and cervical mucus methods to prevent pregnancy or to conceive? | ORCHA |
| 1e — Q28 | Does the developer claim that the application can be used as a natural method of birth control? | ORCHA |
| 1e — Q29 | Is the application intended to be used for the control of conception? | ORCHA |
| 1e — Q30 | Is the application used in combination with drugs or medication (e.g., medication reminders)? | ORCHA |
| 1e — Q31 | Is the application a companion of the device, as opposed to having been designed to connect with a third-party manufacturer’s device? | ORCHA |
| Criteria | Criteria Origin |
1f — Q1 | Does the application allow the monitoring of key health information? | ORCHA |
1f — Q2 | Does the application involve the recording of relevant data over time for the user to access and review (with no “intelligent” manipulation of that data by the app)? | ORCHA |
1f — Q3 | Does the application involve the automated assessment or interpretation of relevant data to deliver alerts, insights, reminders, or adjustments regarding the management of a specific mental health problem? | ORCHA |
1f — Q4 | Is the application simple self-management, enhanced self-management, or advanced self-management? | ORCHA |
1f — Q5 | Is the output of the application’s monitoring intended to affect the treatment of an individual? | ORCHA |
1f — Q6 | Does the application allow others (i.e., not the user) to monitor or view the health data captured? | ORCHA |
1f — Q7 | Does the application automatically measure and/or record data about a user’s specified mental health problem and transmit the data to a professional, carer, or third-party organization without any input from the user? | ORCHA |
1f — Q8 | Does the application generate any alarms or alerts from the data recorded by the application or a connected device? | ORCHA |
1f — Q9 | Are the alarms generated by user-defined filtering rules? | ORCHA |
1f — Q10 | What type of intervention or treatment does the application provide? | ORCHA |
| Criteria | Criteria Origin |
1g — Q1 | Can the application be used by users to have online consultations, conversations, or related health-care services with a health-care professional? | ORCHA |
1g — Q2 | Is this through video consultation? | ORCHA |
1g — Q3 | Does the application allow health-care professionals to provide clinical advice, as opposed to the application providing advice itself? | ORCHA |
1g — Q4 | If the application allows health-care professionals to provide clinical advice through the app, rather than the application providing the advice itself, how does it do this? | ORCHA |
1g — Q5 | Are licensed health-care professionals involved in the delivery of the app? | MHCC |
| Criteria | Criteria Origin |
1h — Q1 | Is this an administrative application which does not directly impact user care? | ORCHA |
1h — Q2 | What administrative functions does the application provide? | ORCHA |
1h — Q3 | Is the application used to facilitate communication between health-care professionals other than for consultation or the delivery of advice? | ORCHA |
1h — Q4 | Does the application allow users to book appointments with a health-care professional? | ORCHA |
| Criteria | Criteria Origin |
1i — Q1 | Does the application allow users to order and request prescriptions? | ORCHA |
1i — Q2 | Does the application constitute a pharmacy service? | ORCHA |
| Criteria | Criteria Origin |
1j — Q1 | Does the application send push notifications? | ORCHA |
1j — Q2 | Does the application send email notifications? | ORCHA |
| Criteria | Criteria Origin |
1k — Q1 | Is the application’s main functionality dependent on the user having one of the devices to connect with the app? | ORCHA |
1k — Q2 | Do any of the features or functions of the application appear to allow it to be used to control a device? | ORCHA |
| Criteria | Criteria Origin |
1l — Q1 | Are there opportunities to link with other users (buddying, forums, or group education)? | ORCHA |
1l — Q2 | Does the application provide an internally hosted forum or online community for their users? | ORCHA |
1l — Q3 | Does the application link to a third-party service to host a forum or online community for their users? | ORCHA |
1l — Q4 | Does the application allow two-way communication between citizens or health-care professionals? | ORCHA |
| Criteria | Criteria Origin |
1m — Q1 | Does the application provide gamification or goal-setting features for the user? | ORCHA |
1m — Q2 | Does the application set goals for the user? | ORCHA |
1m — Q3 | Does the application allow the user to set goals for themselves? | ORCHA |
Criteria | Criteria Origin | |
| 1n — Q1 | Can the application presentation be customized by the user? | ORCHA |
| 1n — Q2 | Does the application respond to preferences in the device? | ORCHA |
| Criteria | Criteria Origin |
1o — Q1 | Is the application totally free? | ORCHA |
1o — Q2 | How is the application funded? | ORCHA |
1o — Q3 | Does the application contain advertisements? | ORCHA |
| Criteria | Criteria Origin |
1p — Q1 | Are the objectives clearly stated by the app/developer? | MHCC |
1p — Q2 | What are the claimed or implied benefits of the app?
| ORCHA |
1p — Q3 | Has the developer clearly stated the application’s target audience? | MHCC |
1p — Q4 | If the developer has clearly stated the application’s target audience, who is the target audience? | MHCC |