Abstract
Objectives: To describe the development of a webpage based on the Intervention Mapping (IM) protocol and usability testing of the Diabetic Retinopathy Health Education Profram (DRHEP).
Methods: The mixed methods pilot feasibility study was carried out between April and September 2021, involving 16 patients with type 2 diabetes mellitus and 5 experts. The usability score was rated according to the System Usability Scale (SUS).
Results: The average SUS score by the experts was 88. The patients gave a higher score of 85 for SUS, with 58 as the lowest. The average SUS score was 72. The findings indicate that the webpage is acceptable, good, and highly usable for users.
Conclusion: The outcomes of this study signify the relationship between effective health applications and how their design might hamper their effectiveness in changing patients’ behavior.
Diabetic retinopathy (DR) often causes disability and becomesa frequent complication of diabetes mellitus (DM), particularly in people of working age.1 Besidesmedicines that result in disease regression, technical advancements in screening and image capturing the fundus features of DR have also cast doubt on the therapeutic significance of the classification of the disease.2
Patients with type 2 diabetes mellitus (T2DM) must make multiple decisions on their progress while attending numerous appointments that require much time and energy. Telemedicine and mobile health can support diabetes self-care by making information and healthcare services easier to understand and more accessible. However, despite the medications that produce disease regression and technical breakthroughs in screening and imaging collection of DR fundus characteristics, therapeutic usesof the Early Treatment of Diabetic Retinopathy Study (ETDRS) classification of DR have raised some doubts.2
Diabetes education videos, patient forums and support groups, and live chats with health coaches make disseminating information and healthcare services more convenient and accessible. Telemedicine and mobile health can also encourage patients to engage in diabetic self-care through a webpage interface. Smartphone notifications can assist users in meeting their health goals, while enhanced connectivity to diabetes care teams allows remote blood glucose monitoring linked with real-time patient progress.3 Thus, technology assistance through a mobile application (app) is a communication tool that monitors disease progression, which benefits patients.
Nevertheless, more interactive and user-friendly educational products that provide interactive measures and use layman’s terms should be produced. Assessment of patients’ needs based on a patient-centred approach, especially among non-proliferative diabetic retinopathy (NPDR) patients, could prevent blindness at the early stage of the disease. Therefore, to address this gap, this study aims to develop a Diabetic Retinopathy Health Education Programme (DRHEP) to improve patients’ knowledge, self-care, and visual function quality of life among T2DM patients. Knowledge transfer through technology aid acts as a communication medium.
In this study, the Diabetic Retinopathy webpage was established to provide health education among T2DM patients based on diabetes complication risks. The webpage was designed based on clinical research recommendations for T2DM and Diabetic Retinopathy Screening.4,5 Both guidelines provide as main reference in the provision of the webpage.
This study describes the development of a webpage based on the Intervention Mapping (IM) protocol and usability testing of the DRHEP webpage.
Methods
A mixed-method pilot feasibility study was employed using a semi-structured interview followed bythe System Usability Scale (SUS) questionnaire containing tenquestions from April to September 2021.6 The study involved 15 experts and 16 patients with T2DM from the ophthalmology clinic, University Malaya Medical Centre, Selangor, Malaysia.
The inclusion criteria for the experts were as follows: various backgrounds, including ophthalmology, endocrinology, and nursing. Meanwhile, the inclusion criteria for patients were: diagnosed with T2DM, aged 18 years and above, diagnosed with NPDR until moderate NPDR (NPDR), and have cognitive capability. Blindness, bedridden, dementia, and cognitively impaired patients were excluded from this study.
The study was carried out in 4 steps: experts’ interview, webpage development, and validation process of experts and patients.
Steps 1: Experts’ interview
Five experts (group A) were involved in the interview process before the DRHEP webpage testing. The aspects of evaluation include the topic, structure, content, video presentation, and overall effect sizeof the webpage. The panel of experts advised on the alignment with the current Clinical Practice Guideline of Management Type 2 Diabetes Mellitus 6th edition 202064 (Expert 1) and the addition of the signs and symptoms of DR to enhance the understanding of the changes in the patient’s eyes during the progress of the disease (Expert 2). Expert 3 suggested that selecting a proper color scheme and clear font for the subtitle could ease the audience’s experience while watching the educational video. The treatment plan should be updated in line with the diabetic retinopathy guideline. Experts 4 and 5 recommended using layman’s terms in the subtitle to overcome the language barrier for those who did not understand Malay. The overall comments from the experts were acknowledged for amendment and revisionwith information technology (IT) personnel before being implemented.
Step 2: Webpage development
The development of the webpage was based on IM, which involved developing and implementing a basic health education programme.7 The DRHEP webpage is available in English and Malay. It was developed as a web-based application with an adjustable desktop and mobile phone design. The users were given the link to the webpage by the researcher. They were directed to the home page of the webpage, from where they can navigate to other pages. Users can view the choices on the home page, which include health education in PDF form, a PowerPoint presentation, or a video of an ophthalmologist. Patients were given access to the link for a week.
Usability testing
The usability testing in this study applied the SUS questionnaire for experts and patients. The items of the questionnaire and the process are detailed below.
Usability questionnaire
The SUS was invented by John Brooke in 1986 with a 10-item version scored on a 5-point Likert scale, from 1 (strongly disagree) to 5 (strongly agree). The Malay version of SUS (named SKAMA) was translated and validated by 10 experts.The efficacy of the questionnaire was further assessed by testing face validity on 10 mobile phone users, followed by reliability testing involving 54 mobile phone users.8 A system or product that received a score of 68 and above is considered to have good usability. The items are listed below:
Item 1: I would use this app often; Item 2: I found the app unnecessarily complex; Item 3: I found the app easy to use; Item 4: I think help from technical support would be needed to be able to use this app; Item 5: I thought the different functions of the app were well integrated; Item 6: I thought there was great inconsistency in this app; Item 7: I would imagine that the majority of people would quickly learn how to use this app; Item 8: I found the app very hard to use; Item 9: I felt confident using this app; Item 10: I needed to learn a series of things before I could use the app.
The following is the formula to calculate the total score of SUS: X=Sum of the points for all odd-numbered questions–5. Meanwhile, Y=25 – the sum of the points for all even-numbered questions. Thus, the SUS score = (X+Y) × 2.5. The total scores of SUS between 0 to 100 are as follows:> 80 (excellent), 68-80.3 (good), 60 (okay), 51-68 (poor), and <51 (awful).9
Step 3: Experts’ usability testing
The usability test for the Diabetic Retinopathy webpage was carried out among 10 experts (group B) to evaluate the implementation for the usersin the study. The panel of experts encompassed 2 with IT speciality in webpage, 5 medical and health science lecturers, and 3 postgraduate students with experience developing webpage from different universities.
Step 4: Patients’ usability testing
A total of 16 T2DM patients participated in the study. The inclusion criteria include patients with T2DM, aged 18 years and above, diagnosed with moderate NPDR, and have cognitive capability. The exclusion criteria for this study were blindness, bedridden, dementia, and cognitive impairment.
The Institutional Review Board (IRB number: 202068-8726) approval was issued and obtained from the University Malaya Research Ethic Committee, Malaysia.
Statistical analysis
No formal hypothesis was carried out in this analysis, and descriptive statistics were consistent with the applied qualitative study methods of usability testing. Data were presented using counts, proportions (%), and means. All statistical analyses were conducted using Statistical Package for the Social Sciences for Windows, version 26 (IBM Corp., Armonk, NY, USA).
Results
A total of 16 patients and 10 experts participated in the study. Table 1 reveals the SUS value scores of the DRHEP webpage from the 10 experts (group B). The highest score for SUS given by the experts is 88, and the lowest is 80. The average SUS score is 83. The results indicate that the webpage is acceptable, good, and highly usable.
- The System Usability Scale score by 10 experts on Diabetic Retinopathy Health Education Programme webpage.
Table 2 reveals the SUS value scores of the DRHEP webpage from the 16 patients. The highest SUS score given by the patients is 85, and the lowest is 58. The average SUS score is 72. It indicates that the webpage isacceptable, good, and highly usable.
- The System Usability Scale score by 16 participants on Diabetic Retinopathy Health Education Programme webpage.
Discussion
Several interventions are used to disseminate health information on DR and other components like self-care, self-management, peer support, and group-based interventions using traditional or online approaches. Patients with no DR to moderate NPDR were chosen because they would be the influential group for the prevention utilizing DR education in the study. Hence, the practicality and effectiveness of T2DM patient implementation approaches depend on them. The precise elements of instructional content that permit analysis are crucial from a research standpoint.
Diabetes management involves lifestyle modification, medications, and patient education to encourage self-care and empowerment.10-12 Rapid changes, especially in the modern era alongside the COVID-19 pandemic, create challenges for keeping education up-to-date and technological usage to spread information.4
In line with the current worldwide pandemic, online health education has become more relevant due to limited physical interaction between patients and healthcare providers. Web-based diabetes research via the Facebook platform to deliver information, cure-focused innovations, and negative sentiment refutation is relevant among users.3 Meanwhile, smartphone application programmes for T2DM patients to monitor physical activity, hemoglobin A1c, blood pressure, waist circumference, waist-to-hip ratio, and interventions that suit the patient’s preference and feasibility issues need to be considered.13
Of 3 studies using mobile apps, only 2 were conducted in person. They acquired verbal education and visual feedback of their retinal images for 3 months, with 2 hours of face-to-face instruction every 2 months, focusing on prevention and self-management strategies, nutrition, physical activity, health-seeking behavior, and counselling.14
Meanwhile, Sight BookTM is a free smartphone app that evaluates patients with DR vision at home using the Amsler grid and visual acuity.15 The Diabetes Carer App, which comprises 3 components, such as the diabetic component, self-management, and patient community, is combined with or without self-monitoring of blood glucose (SMBG) and has an impact on glycaemic control in diabetes patients.16 Baseline retinal imaging as a real-time patient education/engagement tool and telehealth screening are 2 of the 4 technology treatments that make up telehealth. The comments/feedback by the patients regarding goals, obstacles, and change-motivating factors are recorded. An electronic decision-support tool for cardiovascular disease and diabetes based on current recommendations serves as a communication channel between healthcare professionals, T2DM patients, and their caregivers.17
Similar to the findings in this study, other researchers and designers have stressed the need to use an iterative strategy while creating webpages to comprehend end-users’ wants and enhance the feasibility and usability of the apps. Therefore, it is vital to focus on the significance of conducting usability tests on webpages that will be utilized in clinical and patient environments. When creating mobile health apps, user testing is crucial, especially when altering actual patient behavior or influencing patient outcomes.
Study limitations
This study was carried out in a teaching hospital due to logistic issues; thus, the possibility for generalization may be restricted. Reaching out to a wider population/venue might overcome this issue, as it would provide differences in cultures and resources of each health system, particularly in the private sector practice. The interaction with patients and experts was carried out online during the COVID-19 pandemic. Therefore, focus group discussion with the experts is recommended in developing an app.
In conclusion, the information on the DRHEP webpage is valuable to the end-users, such as patients with T2DM. The results of this study highlight the association between effective health application sand how their design might hamper their effectiveness in changing patients’ behavior. However, more personalized and interactive elements, such as providing the fundus images of the patient, should be imparted in future app development. A future version of an interactive Diabetic Retinopathy webpage will be a valuable tool for all diabetes patients about blindness-related risks.
Acknowledgment
The authors gratefully acknowledge ProofreadersUnited for the English language editing.
Footnotes
Disclosure. Authors have no conflict of interests, and the work was not supported or funded by any drug company.
- Received September 13, 2023.
- Accepted November 5, 2023.
- Copyright: © Saudi Medical Journal
This is an Open Access journal and articles published are distributed under the terms of the Creative Commons Attribution-NonCommercial License (CC BY-NC). Readers may copy, distribute, and display the work for non-commercial purposes with the proper citation of the original work.
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.