Our protocol presents different methods of evaluation and validation of competing risk nomogram;which considers the presence of competing events in survival analysis. This protocol serves as a supplemental to the risk regression package in R.Such as the calculation of C-index and the internal validation, using bootstrap method. To begin for C-index discrimination, fit the matrix cob into the competing risk model mod CRR and get a predicted matrix SUV by executing the command.
Get the cumulative incidences in a certain month from SUV and calculate the C-index with the function R-core sends. For AUC discrimination, score the predictive performance of the competing risk model, using the function score, risk regression package. Then extract the AUC from the score by executing the command.
To obtain calibration curves with a 95%confidence interval of the competing risk model, get a data frame with the cumulative incidences of each individual in a certain failure time. Then divide the cohort according to the estimated cumulative incidences into five subgroups and calculate the average predicted cumulative incidences of each subgroup. Calculate the observed cumulative incidences that is the actual cumulative incidences using the function cuminc.
And then get the observed cumulative incidences with a 95%confidence interval in a certain failure time by executing the command. Plot the calibration curve with the predicted cumulative incidences as the X axis and the observed cumulative incidences as the Y axis. Using the function gg plot.
For calibration curve with risk scores of the competing risk model, valuate each level of all the variables and obtain the total RS by executing the command. Count the frequencies and calculate the observed cumulative incidences of the different total risk scores. Set the range of the X axis and calculate the predicted cumulative incidences of the total risk scores.
Then plot the calibration curve with risk scores by executing the command. To get the average predicted cumulative incidences using the bootstrap method, re-sample the original dataset, dataset with replace, to generate the bootstrap dataset. Dataset in.
Then establish a competing risk model:mod NCRR with the bootstrap dataset and use the function predict CRR to predict mod NCRR in loop B times to generate SUV all in. Next, get the average predicted cumulative incidences in a certain month. Calculate the C-index using interval cross validation with the function R-core sends.
For calibration, using external validation, get the predicted cumulative incidences using external data and cumulative incidences with the matrix of external data variables:Code-X by executing the command. Then calculate the C-index using external validation By executing the command. Two nomograms were obtained using the direct method and the weighted method, demonstrating that the points of each level of variables and the probabilities corresponding to the total points were almost the same.
While some slight differences were observed. The calibration curve for the competing risk model was close to the equivalence line and the 95%confidence interval of the observed frequency, fell into the equivalence line in each group. Indicating the accurate calibration ability of the model.
Calibration curves using internal and external validation were shown, indicating that the constructed model had a good calibration ability in the internal validation;but a poor one in the external validation. The results of the decision curve analysis of the competing risk nomogram were obtained. Demonstrating the changes in net benefit with increasing threshold probability.
Re-sampling the original dataset was replaced to generate the bootstrap dataset as important, in performing the internal validation of the competing risk nomogram. Besides the bootstrap method, the randomized display and K4 method can also be performed to generate data sets used for internal validation. Using our R based landscape validation of the competing risk model, clinicians can perform a prognosis analysis in real world considering the competing risk more easily.
