Abstract
Objectives: To validate and assess the reliability of the new version of an Arabic speech intelligibility rating among different raters.
Methods: This cross-sectional analysis was carried out between December 2018 and January 2019. Thirty cochlear-implant (CI) children (study group) and 30 subjects (control group) were enrolled. Study candidates’ speech skills were evaluated using the translated Arabic SIR by parents and original SIR by professions such as speech-language pathologists (SLPs). Inter-rater agreement, test–retest reliability, pre- and post-intervention score (responsiveness test), patient versus control score comparison (discriminant validity), and cross-validation of Arabic SIR have all been assessed.
Results: There was a good sense of agreement between the post-operative SIR parents’ assessments and the professional SLPs’ assessments (r=0.920, p<0.001). The mean of study subjects pre- and post-implantation score of Arabic SIR showed a statistically significant difference (p<0.001).
Conclusion: The Arabic SIR demonstrated excellent reliability with strong consistency. It showed its clinical ability in distinguishing healthy subjects from patients along with follow up of speech development skills over time. The Arabic SIR can be used by parents to evaluate post-CI progress of their children.
Severe-to-profound hearing loss in early childhood interferes with normal development of speech and language and had negative impact on child’s education.1 An ultimate outcome of cochlear implant (CI) is intelligible speech acquisition and language development.2 A rating scale provides a feasible method to monitor patients over a long period of time. However, a rating scale can only be used by different raters when they ascertain its inter-rater reliability. Good inter-rater reliability implies that the differences can be attributed to the behaviors of the patients being evaluated rather than the variability in scoring by the different observers.3 Inter-observer reliability is a statistical method to measure the extent of confounding errors that affect the perceived scoring, measured using intra-class correlation coefficient (ICC).4 High ICC values indicates high agreement among the raters.3 Speech intelligibility rating (SIR) is an adapted scale that allows both experienced listeners, including audiologists or speech-language pathologists (SLPs), and normal-hearing listeners with no prior expertise to assess speech perception before and after CI.5 It is comprised of 5 categories as shown in Table 1. Speech intelligibility is higher as the classification number increases.3 There is an increased need for a reliable validated Arabic version of SIR which will provide substantial clinical implications toward the practice of healthcare providers among Arabic-speaking populations. Moreover, this enhances the delivery of accurate assessments and monitoring the progress of CI patients. The availability of a validated Arabic SIR (that shows good reliability between professional and non-professional raters) also gives families a daily assessment tool to monitor their children’s performance and helps to set home training targets for speech rehabilitation. A thorough search in the literature demonstrated no existing validated Arabic SIR.
Categories of speech intelligibility rating (SIR) scales.
The purpose of our study is therefore to validate and estimate the inter-observer reliability (between SPLs and parents) of Arabic SIR.
Methods
Development of an Arabic SIR
The newly designed Arabic SIR were developed. The recommendations for standard cross cultural adaptation were followed in this study.6 The original SIR was translated by 2 professional linguistic advisors (mother tongues are Arabic), and the Arabic SIR was evaluated by 3 SLPs who speak both languages and 10 special-needs teachers. As per their advice, minimal changes have been implemented for cultural adaptation purposes. The Arabic version was then translated back to English by another 2 linguistic advisors. Back translated version was compared to the original version of SIR and yielded high similarity, ensuring that the original meaning was preserved.
The ethical committee of College of Medicine, King Saud University, Riyadh, Saudi Arabia approved the research protocol, and informed consent was taken from subject’s parents. This cross-sectional study was conducted at King Abdullah Ear Specialist Center, Riyadh, Saudi Arabia between December 2018 and January 2019. The study included 30 patients and 30 control subjects. Group of patients were enrolled from post-CI pediatric speech rehabilitation clinic, while the group of controls (with normal hearing) were enrolled from clinics of pediatrics in the same center. The study enrolled every third patient arriving at both clinics. The inclusion criteria were pediatric prelingually deaf patients who received CI surgery. The exclusion criteria were adults (aged 18 and above), any patient who has post-lingual deafness, and those with psychological or neurological diseases.
The process of validation
Every subject underwent an interview and was assessed by a SLP and one of their parents, using Arabic SIR, for inter-rater reliability. The meaningful use of speech scale (MUSS) is a test that assess the speech development in children. It has 3 domains: voice control, communication skills, and spontaneous use of language and this test was carried out for all subjects by SLPs for cross-validation with the parents’ scores of their children for Arabic SIR.7 To assess internal consistency of Arabic SIR, the evaluations were repeated after 14 days for test-retest reliability using intraclass correlation coefficient (ICC). Post-implantation Arabic SIR scores were compared with pre-implantation SIR scores. The patients’ Arabic SIR scores were also compared with the Arabic SIR scores in the control group for discriminant validity.
Statistical analysis
The statistical analysis was performed by the Statistical Package for Social Sciences version 23.0 (Inc, IBM, Armonk, New York, USA) was used to analyze the data. The expression of date was numbers and percentages for categorical data and mean, standard deviation, and range for continuous date. For the aim of comparing 2 categorical data Chi-square tests was used. And to compare 2 different means we utilized the Fisher’s exact test and t-test. The statistics of Kappa were implemented to discover any discrepancy among different tests. For determining the level of agreement between the tests in this study test-retest statistics, including the Spearman’s test, inter-item and inter-class correlation, and Cronbach’s alpha, were calculated. A p value of <0.05 was considered statistically significant.
Results
Patient group included 30 CI children with a mean age of 66.68 months (SD=37.1), while control group included 30 children with normal hearing with 80.28 months as a mean age (SD=32.57) (Table 2). The mean age of 2 groups demonstrated no statistically significant difference (p=0.153). Approximately half of the mothers (no:15) of children with CI held bachelor’s degrees (50.0%) followed by 11 mothers with high school (36.7%) and 3 with diploma degrees (10.0.%). 20 (66.7%) of the patients were implanted bilaterally simultaneously.
Children demographic information (N=60).
Table 3 shows the agreement rates of the Arabic SIR scores between the mothers and the SLPs. There was a positive agreement between the post-operative Arabic SIR mothers’ and the professional SLPs’ assessments (r=0.920, p<0.001), with 20 (66.7%) in total agreement. There was variation in the agreement percentage across the SIR categories, and the lowest was in category 5 (13.3%). However, Arabic SIR showed a high Spearman’s correlation coefficient for test and retest evaluation (r=0.93, p<0.001), with an excellent Cronbach’s alpha level and an intra-class coefficient value of 0.958.
Speech intelligibility rating scales categories discrepancies between speech intelligibility rating scales and mother’s assessment.
In the responsiveness validity test, there were significantly higher Arabic SIR scores reported for post-CI children, at 2.97 (+1.497), compared to their pre-operative Arabic SIR ratings of 1.07 (+0.254), with a statistically significant level of p<0.001. However, discriminant validity showed significantly lower Arabic SIR scores in the patients’ group, at 1.07 (+0.254), compared to the control group, at 5.00 (+0.01), with a statistically significant level of p<0.001. A positive correlation was found between the post-operative Arabic SIR and post-operative MUSS, which indicates the positive concurrent validity of the Arabic SIR (Figure 1).
Concurrent validity between speech intelligibility rating (SIR) and meaningful use of speech scale (MUSS). Pearson correlation=0.588, p<0.001
Discussion
The purpose of this study is to create an Arabic version of SIR. Developing such a tool will help professionals as well as non-professionals, including parents and schoolteachers, to gain more insight on speech development of children with CI, which will fill the gap between those who are working and dealing with this group of children in order to achieve better outcomes.
During the process of validating Arabic SIR, it showed a strong test-retest reliability as evidenced in the results section that confirms its internal consistency and reliability. On the other hand, the inter-rater reliability of Arabic SIR has been assessed by comparing the ratings of SLPs and of the mothers of CI children following implantation. This data has been evaluated through both kappa statistics and correlation analysis. A high correlation coefficient value of 0.9 has been reported and there was also a high agreement between the mothers and SLPs regarding the Arabic SIR ratings, as indicated by the kappa value of nearly 0.6. However, there were a few discrepancies reported for category 5, with a tendency toward higher scores being reported by the mothers compared to those reported by SLPs. This discrepancy might be explained by the greater familiarity of mothers with their children’s speech, which can make them overestimate this item of the scale and consider their child’s speech to be more intelligible. Variation in SIR scores between patients could be attributed to multiple factors such as duration since surgery, differences in rehabilitation, and personal cognitive differences between candidates. Similar findings of inter-rater reliability were reported by Allen et al7 in their study of developing SIR with a high correlation coefficient and agreement between raters for most of the SIR categories. High inter-rater agreement has also been reported in other related studies that investigated the reliability of the speech intelligibility rating in hearing-impaired and CI children.3,8,9 These findings confirm the validity of Arabic SIR scale being used in the assessment of speech of CI children not only by professionals but also by others who may lack the experience in listening to children with impaired hearing. Moreover, current study results proved discriminative validity of the Arabic SIR demonstrated by significant difference in SIR ratings between pre and post implant and between CI patients and control subjects. (p<0.001). Meaningful use of speech scale is a widely used test for evaluating speech development of children with CI.10 Significant positive correlation between MUSS and Arabic SIR was demonstrated which indicates the concurrent validity.
A validated Arabic SIR helps to improve accuracy of assessment of CI patients among Arabic speaking healthcare providers. Moreover, this test can be used by parents, school and special education teachers, and non-professional who are in direct contact of CI patients to monitor their progress. Such a tool will encourage parents to participate more in rehabilitation of their implanted children. The study findings demonstrated that Arabic SIR is valid and can be used by professional and nonprofessional native Arabic speakers, who are in direct contact with CI-fitted children, to ensure improved quality in post CI rehabilitation monitoring. A validated Arabic category of auditory performance CAPII has been previously developed by the group.11
The limitation of this study is the relatively small sample size. There is no other validated Arabic SIR test in the literature. Considering the increased numbers of CI surgeries performed worldwide on children with hearing loss, the need for the development of validated versions of SIR in different languages would be an essential requirement.
In conclusion, the Arabic SIR appears to be a reliable and valid tool for the assessment of spoken language development in Arabic-speaking children with hearing impairment post CI. Developing an Arabic SIR allows for the opportunity to assess a child’s speech abilities not only in the clinical setup but in everyday situations.
Acknowledgment
The authors are grateful to the Deanship of Scientific Research, King Saud University, Riyadh, Kingdom of Saudi Arabia for funding through Vice Deanship of Scientific Research Chairs. This research was recommended by the Saudi Otorhinolaryngology Society. Also, the authors gratefully acknowledge Scribendi (www.scribendi.com) for English language editing.
Appendix 1
Footnotes
Disclosure. This study was approved by the Deanship of Scientific Research, King Saud University, Riyadh, Kingdom of Saudi Arabia through Vice Deanship of Scientific Research Chairs.
- Received June 20, 2020.
- Accepted August 16, 2020.
- Copyright: © Saudi Medical Journal
This is an open-access article distributed under the terms of the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.