Comment on: Prediction of retinopathy of prematurity using the screening algorithm WINROP in a Saudi cohort of preterm infants

To the Editor

With great interest, I read the article by Raffa et al.1 The authors used the web weight gain based WINROP (weight, insulin-like growth factor I, neonatal, retinopathy of prematurity [ROP]) algorithm to identify infants with type 1 ROP. The authors found that the sensitivity of WINROP in identifying type 1 retinopathy of prematurity was 100%; however, its specificity was 31.5%.

Retinopathy of prematurity is one of the leading causes of blindness in preterm infants.2 Routine screening for ROP is invasive, time-consuming, and usually performed by an experienced pediatric ophthalmologist. It is imperative to search for other avenues to screen for ROP as entertained by Raffa et al.1 Therefore, the authors had the credit to address this important cause of blindness, which can be prevented and successfully treated if detected early.

The design of this study is similar to the previous retrospective study done by others; however, there are few points that require further discussion in the methodology, the outcome and the applicability of the current study.3

Firstly, the retrospective nature of this study limits its impact. Some of the previous studies on the WINROP algorithm were retrospective. However, others were prospective. I would have expected the author to choose a prospective method to replicate previous work.3 This would encourage neonatologists to adopt the study recommendations with confidence.

Secondly, almost one-third of the screened infants were excluded from the study simply because of the lack of weekly entry of weight measurements. This is surprising in a tertiary neonatal intensive care unit where the guidelines suggest that the weight of the extreme preterm infants be checked daily. The authors may wish to clarify the basis of exclusion of this significant number of screened infants.

Thirdly, the authors reported that 26 infants were excluded from the study because for being less than 23 weeks gestation or more than 32. As the inclusion criteria clearly stated that infants recruited if their gestational age <32 or >23 weeks, infants below or above this cut off age should not be part of the targeted study population in the first place and therefore are not eligible for inclusion or exclusion. The exclusion of a large number of participants weakens any study.

Fourthly, the WINROP algorithm is based primarily on weight gain, which is affected by many factors, including the associated comorbidities like necrotizing enterocolitis and sepsis, which are fairly common in preterm infants. Therefore, the inclusion of the associated comorbidities in this retrospective study could have possibly explained the low specificity of the WINROP algorithm in the Saudi cohort in this report. Furthermore, subgroup analysis of the performance of the high-risk infants with significant comorbidity in the WINROP algorithm may assist neonatologists in the use and interpretation of the algorithm for the high-risk population.

Fifthly, the algorithm identified all the 13 infants with type 1 ROP who required treatment, giving 100% sensitivity; however, it missed 12 infants who developed other stages of ROP. Moreover, the specificity of the algorithm was low in this Saudi cohort. Generally, screening is defined as early detection of at-risk subjects before the development of symptoms and signs, which allows therapeutic intervention.4 The sensitivity of the screening tool is considered more important than its specificity, as this would permit the correct identification of at-risk subjects with the additional cost of identifying false positive participants.5 I expect the authors to discuss in depth the interpretation of their findings in the context of the sensitivity and specificity of the algorithm. Also, further discussion as to why the specificity of the algorithm was low in this particular Saudi cohort is warranted. For example, some genetic and environmental factors peculiar to Saudi may have played a role.

Lastly, and based on their findings, I wonder would the authors implement the WINROP algorithm in their routine practice in their neonatal intensive care unit.

Also, the WINROP algorithm is available online. However, it would be more appropriate to inform the readers whether permission to study the algorithm was obtained from the original authors.

Sarar Mohamed

Department of Pediatrics Prince Sultan Military Medical City Riyadh, Kingdom of Saudi Arabia

Reply from the Author

We would like to thank Professor Sarar Mohamed for his interest and comments on our recently published article.1 He has raised several interesting points worth discussing.

A retrospective design was favored by the authors for one main reason. The retrospective inclusion of cases allowed for a relatively large sample to be collected and analyzed over a 5-year period. We favored a retrospective design because we wanted to first explore the applicability of this algorithm in our region, which has not been previously studied. The importance of interpreting the results of this article with caution was highlighted by the authors. Based on our results, and considering this limitation, the need for a multicenter prospective study to validate the accuracy of the WINROP algorithm in Saudi cohort of preterm infants is warranted. This would improve the value of the data and address any uncertainties in the applicability of this screening tool on our population.

Since daily weight measurements are part of routine clinical practice in neonatal care units around the world, WINROP is considered a convenient tool.6 Unfortunately, in our study, missing weight entries were mainly from those infants included between 2013 and 2014 when archived paper files were found to be missing for many of these infants, accounting for the large number of patients excluded. This was, fortunately, not the case for infants followed up during recent years.

For clarification, all infants who underwent screening for retinopathy of prematurity (ROP) at our institute were included in our study. Following the American Academy of Pediatrics guidelines for ROP screening, some of our screened infants included those with older gestational age who had an unstable clinical course and were thought to have a high risk of developing ROP. Infants who did not meet the inclusion criteria for the WINROP algorithm were excluded from our registry; these infants all happened to be born at more than 32 weeks’ gestation. This is a limitation of the WINROP algorithm, especially in developing countries, where more mature infants have been observed to develop treatable ROP.7 Therefore, it is necessary to explore different algorithms to detect infants at high risk of treatable ROP, which are more generalizable to the developing world.8

The WINROP algorithm does not take into account other health related issues that may affect the infant’s weight measurements. We concur with Prof. Sarar that some neonatal risk factors such as intraventricular hemorrhage, hydrocephalus, necrotizing enterocolitis, and sepsis could induce excessive weight gain that would affect the algorithm’s ability to detect those at risk for treatable ROP. In our study, 11 of the infants who developed any stage of ROP and signaled low-risk alarm had at least one or more of the following: respiratory distress syndrome, sepsis, necrotizing enterocolitis, or intraventricular hemorrhage. Thus, even if we had to use the WINROP algorithm to complement our national screening schedules, clinical judgment must always supersede the WINROP alarm outcome.4 Unfortunately, our sample size did not allow us to perform subset analyses. In future studies, if the sample size allows, it would be interesting to explore how certain neonatal risk factors could be used to fine tweak the algorithm to better suit our population.

It would also be interesting to investigate the factors affecting the variability in accuracy of this screening tool in Saudi Arabia compared to other countries; however, it was not the primary objective of our study. An in-depth interpretation of the reasons explaining the discrepancies in the accuracy of this tool is not possible. The authors can only speculate since this algorithm was set for the Swedish population, where the “normal expected weight gain curve” may differ from that expected in other parts of the world, high false positive rates can result. Differences in the availability of resources, neonatal care standard, ethnical background, and socioeconomic diversity may also account for the low specificity observed in our study. Fortunately, the high sensitivity and negative predictive value of ROP screening examinations (at 100%) are thought to be more relevant than their low specificity, which will merely lead to over-screening infants who falsely trigger the alarm.

We have not yet implemented the WINROP algorithm as part of our routine at our institute. We believe that this study paves way for future projects to explore this algorithm, amongst others, in large prospective multicenter studies to find the best screening tool that fits our population. As with any algorithm, customization to reflect regional differences in premature neonates at risk of developing ROP is needed.

WINROP is a free readily accessible online screening tool, which users can access after filling out a form on the authors’ website.