Abstract
Objectives: To validate the web weight gain-based WINROP (weight, insulin-like growth factor I, neonatal, retinopathy of prematurity [ROP]) algorithm retrospectively to identify type 1 ROP in a Saudi cohort of premature infants.
Methods: The records of preterm infants (>23 and <32 weeks gestation) born between August 2013 and October 2018, were reviewed. Birth weight, gestational age, and weekly weight measurements of the premature infants were entered online. Based on weekly weight gain, the WINROP algorithm alerted clinicians whether infants were at high-risk for vision-threatening type 1 ROP. Sensitivity, specificity, positive and negative predictive values were calculated.
Results: The median gestational age of the infants at birth was 28 weeks, with median birth weight at 1085 g. Of the 175 infants included in the study, 13 (7.4%) developed type 1 ROP. WINROP positive alarm was triggered in 70.9% (124/175) of all infants and 100% (13/13) of those treated for type 1 ROP. The specificity of the algorithm was 31.5%. Positive predictive values was 10.5% and negative was 100%.
Conclusion: The general WINROP sensitivity in identifying type 1 ROP was 100% similar to that reported in developed countries; however, its specificity was low at 31.5%. Tweaking of the algorithm based on the population may increase the specificity and promote the practical utility of this non-invasive screening tool for ophthalmologists and neonatologists in this population.
Retinopathy of prematurity (ROP) is a vision-threatening vascular proliferative disorder among the preterm infant population.1 The frequency of ROP based on the Cryotherapy for Retinopathy of Prematurity Cooperative Group is estimated to occur in 66% of infants under 1,250 g, reaching up to 82% among infants under 1000 g.2 Retinopathy of prematurity should be effectively and quickly detected because, if left untreated, it can lead to visual impairment or even vision loss.3 The effective screening program should identify infants with ROP that require treatment. Currently, the gold standard test for the diagnosis of ROP is ophthalmological examination, but it is a stressful and painful procedure.4 In contrast, the WINROP algorithm (weight, insulin-like growth factor [IGF], neonatal, ROP) is a non-invasive, simple, and inexpensive method of predicting ROP risk.5,6
The online WINROP algorithm7 can be used to identify infants with an increased risk for severe ROP.5,6 It was developed in Gothenburg, Sweden based on their studies of IGF-1 showing a correlation between a prolonged period of low-level serum IGF-1 and ROP.5,6 WINROP, calculated based on birth weight (BW), postnatal weight gain, and gestational age (GA) determines the approximate IGF-1 levels and, in this way, identifies the risk for vision-threatening ROP.5,6
The WINROP surveillance system has been validated in several studies.8-12 In highly developed countries (Sweden and United States),5,13 the accuracy of the test was higher, as opposed to that in less and moderately developed countries (Taiwan, Turkey, Korea).14-16 This study aims to retrospectively validate the diagnostic accuracy of the readily available online WINROP algorithm in the detection of vision-threatening type 1 ROP in a cohort of premature infants from Jeddah, Saudi Arabia.
Methods
The study included preterm infants from the Neonatal Intensive Care Unit of a tertiary university hospital in Jeddah, Saudi Arabia born between August 2013 and October 2018. All infants who underwent screening for ROP, had their weight measured every week, and had survived the final ophthalmologic ROP assessment were included. Exclusion criteria were gestational age at delivery <23 or >31+6 weeks’ gestation, incomplete weight entry, and no final ROP outcome documentation. Ethical clearance for the study was given by the Hospital’s Research Ethics Committee, which also conformed to the Declaration of Helsinki.
Infants demographics, GA, BW, and serial weight measurements were noted. In our center, infants were weighed every day until discharge. Each weekly weight had been entered into WINROP measured at exactly 1, 2, and 3 weeks, and so forth, after the child’s birth date. Also documented were the infant’s worst stage of ROP in either eye, ROP zone, presence/absence of plus disease, and ROP treatment.
Retinopathy of prematurity screening
At our institute, infants who were eligible for ROP examination, according to the American Academy of Pediatrics guidelines,17 were examined by pediatric ophthalmologists or vitreoretinal surgeons and categorized following the International Classification of ROP.18 All treatments were administered following the Early Treatment for Retinopathy of Prematurity Study guidelines.19 Retinopathy of prematurity screening was continued until treatment was required or complete vascularization of the retina occurred.
WINROP screening
Based on weekly weight gain, the WINROP algorithm alerted clinicians whether infants were at risk for vision-threatening type 1 ROP.20 Data entered into the WINROP algorithm included BW, GA, and weekly weight measurements until an alarm was signaled or a corrected gestational age of 35-36 weeks was reached.
The WINROP program categorizes infants into 2 groups: positive alarm and no alarm. An alarm identifies infants who are at high risk for type 1 ROP requiring treatment (based on the ETDRS guidelines). Alarm timing was recorded for all infants, and an infant was labeled to be at minimal risk for developing type 1 ROP (non-type 1 ROP) if no alarm was signaled by the time the initial ROP examination was performed. When the accumulated differences between the expected postnatal weight gain and the expected preset values, taking into account the BW and GA at birth, is exceeded, the alarm is signaled.
Statistical analysis
The study data was analyzed using IBM SPSS Statistics for Windows version 23 (IBM Corp., Armonk, N.Y., USA). Quantitative variables were expressed as mean ± standard deviation (SD) or median ± interquartile range, whereas qualitative variables are presented as numbers and percentages. Fisher’s exact test and Chi-square tests were used to check if there is a difference between categorical variables. While comparing 2 group means, an Independent t-test was used. These tests were carried out with the assumption of normal distribution. Sensitivity and specificity of WINROP alarm in predicting type 1 ROP were calculated based on actual ROP outcomes. Prevalence of type 1 ROP in the cohort was further used to calculate positive and negative predictive values of WINROP. Confidence intervals (CIs) were constructed at a 95% confidence level. The results of the ophthalmologic examination were adopted as the “gold standard”. In addition, the level of accuracy between the WINROP algorithm results and the final diagnosis were calculated using Cohen’s kappa coefficient κ. In all analyses, the differences were defined as significant when the p-value was less than 0.05.
Results
Retinopathy of prematurity screening was carried out for a total of 297 infants during the study period. Ninety-one were excluded for lack of weekly entry of weight measurements. Five were excluded due to missing data, and 26 for being <23 weeks or >32 weeks GA. Overall, 175 infants were included. Median BW was 1085 g (552-1920 g) and GA was 28 weeks (23-31 weeks). Among the 175 screened, none developed stage 4 or higher ROP. Sixty-eight patients (38.9%) developed any grade of ROP (non-type 1 ROP) and 13 infants (19.1%) developed vision-threatening type 1 ROP requiring treatment based on ophthalmic assessment.
WINROP outcome
Among the 175 screened, no alarm was triggered in 51 infants (29.1%). Of those, none developed type 1 ROP. The median BW in these infants was 1300 g (range, 792-1920 g) and GA was 29 weeks (range, 24-31 weeks). A positive alarm was triggered in 124 infants (70.9%). Of these, 13 developed type 1 ROP. The median BW in this cohort was 986.5 g (range, 552-1380 g) and GA was 28 weeks (range, 23-31 weeks) (Table 1). The mean time from the alarm alert to start of treatment was 4.83 ± 3.7 weeks.
Relationship of WINROP alarm signal to case characteristics and ROP stage.
Retinopathy of prematurity outcome versus WINROP algorithm (test characteristics)
The WINROP algorithm sensitivity in detecting ROP requiring treatment was 100%, with a specificity of 31.5%. Positive predictive value was 10.5% and negative was 100.0% (Table 2). The algorithm identified 13/13 infants who required treatment but missed 12 infants who developed any stage of ROP in whom no alarm was signaled on WINROP. The mean age at detection of ROP was 29.83 ± 1.6 weeks corrected gestational age. The accuracy rate of the WINROP algorithm was calculated to be 36.6%. Using WINROP, ROP screening could have been reduced by 29.1% (51/175) in our sample based on the number of infants who did not signal an alarm and who turned out non-type 1 ROPs.
Sensitivity, specificity, positive and negative predictive value in predicting type 1 ROPs (those requiring treatment) using the WINROP algorithm.
The weekly weight curve of infants developing any stage of ROP and whom the WINROP algorithm did not identify as high-risk with respect to other infants is shown in Figure 1. Infants with type 1 ROP had a significantly lower weight than those with non-type 1 ROP or the missed patients (those who did not signal an alarm but developed any stage ROP).
Graph of the weekly bodyweight of the missed infants (triangles) who developed any stage of retinopathy of prematurity (ROP) in whom the WINROP algorithm did not signal a high-risk alarm with respect to other infants. Boxes represent the weekly weight of infants with non-type 1 ROP, and diamonds represent that of infants with type 1 ROP. The asterisks represent significant differences (p<0.05).
Discussion
It would be helpful for clinicians to screen and prevent ROP if they could identify postnatal factors that may potentially affect ROP severity. Based on clinical studies, an association exists between the infant’s low serum IGF-1 and poor postnatal weight gain and the risk of developing more severe ROP.5,21-23 Earlier studies from Sweden5 and North America,13 reported that all infants with stage 3 ROP were recognized by inserting only the postnasal weight measurements, further simplifying the WINROP system. Similarly, in this report of a Saudi cohort of premature infants, WINROP identified correctly all 13 infants (100%) with type 1 ROP. This is what distinguishes our study from those conducted in other developing countries that showed lower sensitivity ranging from 80 to 90%.12,14-16,20,24-27 Sensitivity has also been demonstrated to be lower in heavier infants (>1000 grams) in developed countries, which was not the case in our series.14
The overall specificity was low due to a high false-positive rate at 31.5%. Large variations in WINROP specificity have been reported in cross-sectional studies that validated the WINROP.8,10-12,14 Variability in cases, neonatal care, survival rates, and screening practices could all account for those discrepancies. However, since preventing retinal detachment and, ultimately, blindness is the goal of ROP screening, the sensitivity and NPV of ROP screening examinations are thought to be more relevant than its specificity.
This study validated the WINROP algorithm in a Saudi sample of preterm infants. It is novel in that it is the first study of this type conducted in a Middle Eastern cohort. WINROP may help clinicians determine whether infants are at risk for vision-threatening ROP. Given that this algorithm is completely based on weight measurements which are routinely documented in clinical practice worldwide, it is a non-invasive, useful adjunctive tool that can be used in addition to our national screening guidelines, which takes into account our geographic variabilities and local factors. Therefore, several attempts have been made to allow this tool to function optimally in developing countries. The NPV was 100%, which is ideal to confidently reduce the ROP screening examination for infants with no alarm.
Study limitations
Firstly, one limitation is its retrospective design. The use of this algorithm, which is primarily designed for prospective use, may have affected the results. A multicenter prospective study including more premature infants will improve the value of the data. Secondly, there is a substantial subjective element in documentation and diagnosis of ophthalmoscopic findings. It was previously documented that the agreement on plus disease diagnosis among pediatric ophthalmologists is low.28-30 Thirdly, WINROP is not currently available for infants ≥32 weeks of gestation. This has implications for our population, where older and heavier premature infants have been demonstrated to develop severe ROP in developing countries.31 Finally, our study is limited by small sample size. A study involving a larger number of infants is required to allow for regression analyses and to improve the accuracy of this screening tool.
In conclusion, this is the first report to document the use of a weight gain-based algorithm for the prediction of ROP from Saudi Arabia. The sensitivity of this algorithm in identifying type 1 ROP was 100%, and specificity was 31.5%. Our results are in line with those of previous reports that suggest using the WINROP algorithm to complement rather than substitute the current screening tools. As with any algorithm, researchers should consider customizing it taking into account the regional and demographic differences in premature newborns at risk of developing ROP.
Acknowledgment
We would like to thank Editage for English language editing services.
Footnotes
Disclosure. Authors have no conflict of interests, and the work was not supported or funded by any drug company.
- Received March 17, 2020.
- Accepted May 19, 2020.
- Copyright: © Saudi Medical Journal
This is an open-access article distributed under the terms of the Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
