Abstract
Objectives: To develop an externally validated multivariable prognostic model for an underprivileged dialysis population.
Methods: This was a multicenter retrospective cohort study of 5 years duration from January 2013 to December 2017. A total of 758 patients (37.5% female; mean±SD age, 44.26±14.77 years) were enrolled for construction of the prognostic model. The data were analyzed using a proportional hazards model to identify predictors of survival. Three risk groups were identified at the 25th and 75th percentiles of the resultant prognostic index. The model was externally validated with another dataset of 622 dialysis patients.
Results: The prognostic index included 5 predictor variables: hemoglobin, serum potassium, interdialytic weight gain, serum albumin, and duration of dialysis, which had good predictive performance on the calibration and discrimination aspects of the model (Harrell’s c statistic: 0.748, Gonen and Heller k statistic: 0.647, Somers’ D statistic: 0.496, calibration slope: 1.156). There were significant interaction effects between weight and hemoglobin, weight and albumin, albumin and potassium, and albumin and hemoglobin.
Conclusions: We developed an externally validated model that contained 5 routinely collected prognosticators and confirmed its calibration and discrimination abilities in obtaining reliable prognostic estimates in developing countries. The model will assist clinicians in deciding the prognosis of dialysis patients. The application of this model in different clinical settings of developing countries can indicate interesting findings regarding public health.
Generally, prognostic studies predict a patient’s potential future progression via clinical and demographic characteristics. A prognostic model consists of a combination of prognostic factors that are informative about an individual patient’s treatment, the likely course of a disease, and the risk of a future outcome. A prognostic model must not be used in clinical practice without statistical validation to appropriately predict a related independent dataset and a proper fit for the purpose of prediction.1,2 In recent years, noteworthy developments have been made in dialysis treatment, but the annual death rate fluctuates between 10% and 25% globally.3 Mortality rate among undernourished hemodialysis patients is nearly 30%, while in non-malnourished patients it ranges between 10 to 15%.3 The increased prevalence rates of chronic kidney disease (CKD) and the costly treatment of dialysis indicate the importance of this health issue worldwide.1 Poor survivorship of dialysis treatment is still a concern for the treatment planning of patients with this chronic disease. A reliable prognostic model is needed for risk prediction and to improve the prognostication strategies. The prognostic model would be helpful to nephrologists initiating dialysis and assessing its suitability. Generally, the International Prognostic Scoring System is used with 3 “prognostic indicators” to predict the progression of the patient’s disease by prognostic groups. These prognostic risk groups do not predict how a patient’s disease will respond to treatment, but instead, it predicts the patient’s expected disease development over time without treatment.
The application of prediction models to clinical care is conditioned with external validity. Existing prognostic models are either not validated or only internally validated.2,4-15 Although most of these studies reported accurate predictive ability, none were externally validated; thus, they have limited clinical utility and have not been implemented in clinical practice.
In this research work, an initial dataset of 1215 patients was divided into 2: data from 758 patients used to build a prognostic model for CKD patients and data from 457 patients used to internally validate the model. A different dataset of 622 patients, collected at different times and from different regions, was used for external validation of the model.
Methods
The target population for the initial dataset comprised dialysis patients admitted to the dialysis units of government hospitals in 2 divisions of Punjab province during the period from January 2013 to December 2017. Data for 1215 patients was retrospectively collected from 6 dialysis units for model building, after approval of study and with consent of hospital administrators. For external validation, data of 622 dialysis patients, of a different region was collected. Each patient was followed-up until the end of the study period of 5 years or death, whichever occurred sooner. (For this research work the ethical clearance of study from departmental review committee of International Islamic University Islamabad has been taken, moreover, all the ethical values/rules were observed in accordance with relevant guidelines and regulations during the study. Data used in this study was collected retrospectively from hospital records and informed consent from dialysis units had been taken before data collection).
From retrospective data, patients who met the inclusion criteria were randomly enrolled. To avoid selection bias inclusion/exclusion criteria were stated clearly from the beginning of the study. Only prevalent chronic dialysis patients who had been maintained on hemodialysis for at least 3 months and survived for 90 days after first undergoing treatment were considered eligible for inclusion in the study.
For each patient, information on the following quantities was obtained from medical records: age, gender, date of the start of dialysis, age at the start of dialysis, duration of dialysis at entry, the clinical data contained serum albumin, pre-dialysis urea, pre-dialysis creatinine, inter-dialytic weight gain serum potassium, serum phosphate, hemoglobin, duration of dialysis per session, frequency of dialysis, average dialysis dose Kt/V (1.2 ±0.216), incidence of hepatitis (B/C) at entry, type of hospital acquired hepatitis, causes of dialysis, comorbidities. Laboratory variables were measured as time-averaged data, and each patient’s data was updated every 3rd month, as recommended by Kalantar-Zadeh et al.16 All patients were treated by in-center HD in Government hospitals, usually twice a week with treatment duration of 3 hours per session for relatively stable patients. Critical patients received dialysis treatment thrice a week with same duration of 3 hours.
Statistical analysis
The Cox proportional hazards model is a well-known regression technique and widely applied modelling strategy for time to event data. This study was executed to test the hypothesis that there are significant differences in the prognosis of patients caused by the levels of interacted predictors. Clinically relevant significant interaction terms were included in the final Multivariate Cox regression model. The Cox model’s proportionality assumption was examined for each covariate by using an approximate score statistic and Schoenfeld partial residuals17 and simulations.18 The assumption of linearity for the continuous prognostic factors was examined by using lowess19 smoothed residual plots. To assess the model adequacy, the indication of correctly fitted survival model is the display of the straight line with unit slope and zero intercept, like exponential distribution.
The prognostic index of the derivation dataset was calculated as the sum of the parameter estimates from the Cox regression multiplied by the covariate value of each patient and the underlying baseline survival. To build prognostic groups, the prognostic index of the derivation dataset was categorized into 3 groups at the 25th and 75th percentiles of the model’s risk score, denoted as low, intermediate, and high mortality groups. Similarly, 3 risk groups were established from the PI constructed for external validation (which is a new prognostic index) at the 25th and 75th percentiles, to develop a model from one dataset and validated by another. The calculation of the regression parameters from the validation model will then yield a new model instead of validating the existing model.1 The predictive performance of the model was evaluated by measures of discrimination and calibration.2 Statistical analysis was conducted using STATA 13 and SPSS 20.
Results
Patient survival time is definitely the main concern in dialysis therapy. The median survival time for the derivation dataset was approximately 13 months and 18 months for validation dataset. Univariate analysis was carried out using Cox regression and Kaplan-Meier estimates. Univariate analyses suggested that gender, current age, hospital-acquired hepatitis, weight gain, creatinine, potassium, phosphate, hemoglobin, serum albumin, and dialysis duration were significant (p<0.05) prognostic predictors for overall survival. Female patients were inclined to survive longer than male patients. Younger patients tended to survive longer than older patients. Hospital-acquired hepatitis (both B+ and C+) proved to be negatively associated with the survival of patients. Survival of those with a greater duration of dialysis at entry tends to be better than those with a shorter duration at entry. Elevated interdialytic weight gain, potassium urea, and phosphate were associated with shorter survival times, whereas hemoglobin and serum albumin were positively associated with survival. The final regression model is shown in Table 1.
- Multivariable Cox prognostic model for the derivation dataset of dialysis patients (N=758).
The Kaplan-Meier survival estimates for the prognostic groups computed from the prognostic index of the derivation dataset are presented in Figure 1. The Kaplan-Meier survival estimates for the prognostic groups constructed for internal validation are shown in Figure 2. The Kaplan-Meier survival estimates for the prognostic groups constructed from the prognostic index of the external validation dataset are presented in Figure 3.
- Risk groups of the derivation dataset, based on Cox model.
- Risk groups for internal validation of prognostic model, based on Cox model.
- Risk groups for the external validation dataset, based on Cox model.
For a prognostic model to be clinically useful, a necessary (but not sufficient) requirement is internal validity. In Figure 2, the risk groups appeared to be separated, although not so clearly, at early follow-up times and after the 20th month from the initiation of dialysis. After the fifth month of being dialyzed, the separation of the prognostic groups was clearer. This explains a modest amount of the variation in the prognoses of patients at the beginning of dialysis therapy. From Figures 1-3, Kaplan-Meier survival curves showed that prognosis was poor in the high mortality group and good in the low mortality group.
Predictive ability measures of the model have been calculated by using the measure of concordance Harrell’s c index, Gonen & Heller k statistics and Somers’ D statistic, presented in Table 2.
- Predictive ability measures of prognostic model evaluated in derivation and validation datasets
The value of Harell’s c index was 0.6977 for the derivation data, indicating that survival time for pairs of patients can be correctly ordered 70% of the time on the basis of a set of prognostic factors. In the external validation data, Harrell’s c index was estimated to be 0.7483, demonstrating that, by using the set of predictors in the model, the order of survival times for pairs of patients can be correctly identified around 75% of the time. Values of Harrell’s c index, the Gonen and Heller k, and Somers’ D statistics were similar in size in both datasets, indicating good predictive ability and validity. The hazard ratios between risk groups (Table 2) were well maintained in both datasets, which confirms the discrimination between risk groups presented in Figures 1 & 3. The calibration ability of the model was assessed by calibration slope as 1.1562, indicating no evidence of over-fitting of the prognostic model. Hence, calibration appeared to be preserved.
In the current study, immediately after the initiation of dialysis, worse survival was demonstrated due to the presence of some negative prognostic factors at that time (such as diabetes, hypertension, high interdialytic weight gain, hypoalbuminemia and chronic anemia, and so forth). This explains the stronger positive association between the event and the risk at initiation of dialysis. As dialysis is a violent therapy with its own complications, the accomplishment and proper maintenance of compulsory target ranges of clinical profiles are essential for better prognosis of prevalent patients receiving dialysis.
The median survival time in the derivation sample for the low, intermediate and high mortality risk groups was 22, 18, and 13 months, respectively, after beginning dialysis treatment, whereas in the validation sample, the median survival time was 12, 6, and 4 months, respectively. In underprivileged populations, a lack of early detection of disease and late referral to nephrologists, and ultimately very late commencement of treatment at ESRF stage 4 or 5 is common behavior. Consequently, survival times for such critical patients are very short. Patients in poor resource countries can have a better outcome by early detection of abnormal patient characteristics.
Discussion
Nephrologists are generally hesitant to respond to patients and rarely convey prognostic-related facts to patients and families. A reason for this behavior is uncertainty about the accuracy of existing prognostic models. The primary goal of the current study was to derive a validated prognostic scoring system based on clinically correctable factors that could be utilized to define the risk groups of patients receiving hemodialysis.
In the present study, a derivation dataset was used to build a prognostic model for the overall survival of dialysis patients, and external validation of that model was investigated using a second, independent dataset from a relevant but different population (diverse in terms of patient characteristics, region, and time). Compatible prognostic variables in both datasets were taken, with a maximum follow-up time of 5 years. In the derivation (n=758), internal validation (n=457), and external validation (n=622) datasets, there were 481, 276, and 412 events, respectively. It is recommended that at least 10 events per covariate be included in the model to avoid any instability in the fitted Cox model.20 The overall risk score expresses the expected progression of the disease and assigns each patient to a particular risk group. High mortality group can have more complications in a short time span and require more intensive treatment. This study contains original work in developing and validating a new prognostic model for the survival of hemodialysis patients. To the best of our knowledge, such a model for these patients and clinical settings has not been previously developed. It is a clinically useful risk model with well-defined and measurable variables. The proposed additive model is a clinical decision aid that produces risk predictions. Often, inspection of the interactions of variables provides important information and a better understanding of the mechanism of action to reduce morbidity and mortality and improving dialysis management. For instance, in our analysis, potassium and albumin had significant interactions, and a detailed analysis of the combined effects of covariates at each level proposed that a higher level of potassium, regardless of the albumin level, increased the hazard risk. Results showed that high levels of albumin, high hemoglobin, low interdialytic weight gain, and low/medium potassium were associated with increased average survival time. Significant interaction effects between interdialytic weight and hemoglobin (p=0.009) and albumin and potassium (p=0.008) were the major determinants for survival of hemodialysis patients and were modifiable factors. Therefore, improving variables such as albumin and hemoglobin, and simultaneously keeping interdialytic weight gain and potassium in the recommended range, would reflect a better outcome and improved prognosis. Prediction models that contain only main effects and ignore interactions may be unable to clarify more complex realities and may give fewer accurate predictions. Only Miskulin et al14 and Mauri et al4 retained significant interactions in their prognostic models. Due to the usual supposition that the risk remains constant for different combinations of variables, interaction effects analyses are found infrequently in the literature. Although the inclusion of interactions made the model more challenging to interpret, it nevertheless delivered better inferences and proved to be a more realistic and informative model.
Previous published4,5,11,12 prognostic models for dialysis patients lack external validation, and patients were randomly separated into both development and validation cohorts. Geddes et al9 performed temporal validation. Barrett et al13 executed validation and recalibration of a risk score with a merge of temporal and external validation datasets. Geddes et al9 presented a comparison of retrospective and prospective cohorts to assess model performance. Miskulin et al12 and Miskulin et al14 only established a model and rectified the overall performance of the model. Recently, Schmidt et al21 externally validated a prognostic model for dialysis patients of developed country. In that study, 3 clinical predictors of functional status Karnofsky Performance Score, response to the Surprise Question, and age appeared to be significant predictors of mortality.21 Inclusion of such rare predictors in the validation model will result in impractical tools for clinical implementation by nephrologists. Contrary to the previously reported models, our study is based on routinely measured clinically controllable factors that are easily managed for the population of low-income countries.
Study limitations
The application of the current model in a clinical setting depends upon the composition of the model sample size used to establish the model, predictive ability, and generalizability of the model to other populations of dialysis patients. Precisely, a prognostic model has been constructed based on a multicenter dataset of dialysis patients with multiple laboratory measures in which treatment, follow-up, and endpoints were measured in all patients according to similar criteria. The current multivariable prognostic model has been externally validated using another independent dataset, which offers generalizability of the model to other populations with similar conditions and clinical utility. However, the generalizability of this model to more developed nations is limited due to the relatively short lifespan of the study population. The developed externally validated prognostic model has the potential for use by clinicians in Pakistan and in situations with a similar health care environment. As in most developing nations like Pakistan, patients usually receive dialysis for 3 hours twice weekly for financial reasons; hence, this prognostic model may not be representative for other populations of advanced countries where the patients’ health conditions are not as fatal as those in this dataset. The other limitations of this study are related to the retrospective nature of the data. Future prospective studies may be able to identify combinations of other predictors. Nutritional status was not assessed in this study; however, clinicians can assess malnutrition patients according to predictors of nutritional status, such as serum albumin.
In conclusion, this study constructed an externally validated prediction model for the survival of hemodialysis patients in Pakistan, and the predictive ability of the model is encouraging regarding the model’s future utility. Assessments demonstrated that the discrimination and calibration of the prognostic model were broadly comparable across the 2 independent datasets used for its development. The developed model has the potential to provide useful information about the survival of dialysis patients in appropriate populations. The risk score models cannot be a substitute for clinical judgment, but can play a vital role by determining prognostic information and helping physicians in clinical decision. As we have experienced low survival rates in our cohort, early management of these prognostic factors is essential to improve the survival rate in our patients.
Acknowledgment
We would like to thank Scribendi (www.scribendi.com) for English language editing.
Footnotes
Disclosure. Authors have no conflict of interests, and the work was not supported or funded by any drug company.
- Received February 8, 2021.
- Accepted May 4, 2021.
- Copyright: © Saudi Medical Journal
This is an Open Access journal and articles published are distributed under the terms of the Creative Commons Attribution-NonCommercial License (CC BY-NC). Readers may copy, distribute, and display the work for non-commercial purposes with the proper citation of the original work.
References
In this issue
Jump to section
Related Articles
Cited By...
- No citing articles found.