purpose. To evaluate sampling strategies used to estimate survival after uveal malignant melanoma that exclude some patients who would be censored from the analysis.

methods. Simulation was performed on a population-based data set of 133 patients who had an eye enucleated because of uveal melanoma. One thousand bootstrap samples of 80 patients were drawn, without replacement, according to three sampling strategies: a random draw (conventional strategy), a draw limited to patients who died of the tumor or survived at least 10 years without metastasis (“late-censoring” strategy), and a draw modified so that 40 patients died of melanoma and others survived at least 10 years without metastasis (“fifty-fifty” strategy). The bias in the Kaplan-Meier analysis and Cox proportional hazards regression was quantified.

results. The late-censoring strategy decreased the proportion of censored patients from 53% to 42%, whereas the fifty-fifty strategy assigned 50% of patients to this group. The former strategy overestimated mortality, the excess being 5.2% and 3.7% at 10 and 20 years, respectively. The latter strategy underestimated mortality, the bias being 1.6% and 4.6% at 10 and 20 years, respectively. The bias differed according to categories of explanatory variables so that the log-rank test statistic was inflated a median of 1.08 times (range, 0.73–1.87) and 1.14 times (range, 0.87–1.84), and the Wald χ^{2} statistic of the Cox regression was inflated a median of 1.18 times (range, 0.79–2.13) and 1.16 times (range, 0.71–2.02), respectively, when the late-censoring and fifty-fifty strategies were applied.

conclusions. Sampling strategies that exclude on purpose a proportion of patients who would be censored produce biased statistics, because they violate assumptions of survival analysis. Only random sampling from an underlying population produces unbiased survival estimates.

^{ 1 }

^{ 2 }

^{ 3 }

^{ 4 }

^{ 5 }

^{ 6 }

^{ 7 }In these studies, the data sets were designed to include an unequal

^{ 5 }or an equal

^{ 1 }

^{ 2 }

^{ 3 }

^{ 4 }

^{ 6 }

^{ 7 }number of patients who either died of uveal melanoma or survived at least for a specified time after treatment. Patients who would have been censored for any reason before the specified time, typically 10 to 15 years after treatment, were excluded.

^{ 4 }

^{ 5 }

^{ 6 }

^{ 7 }The data were analyzed according to standard procedures. Usually these studies did not comment on the use of unconventional sampling strategies.

^{ 1 }

^{ 2 }

^{ 4 }

^{ 5 }

^{ 6 }

^{ 7 }

^{ 8 }

^{ 9 }was reanalyzed to asses the effect on Kaplan-Meier and Cox regression analysis of different sampling strategies. This population-based set consists of 167 consecutive patients treated from 1972 to 1981 when enucleation was the only treatment for all but the smallest uveal melanomas.

^{ 8 }Median follow-up of survivors was 17 years (range, 0.2–26.9).

^{ 8 }

^{ 9 }The variables chosen were gender; age at enucleation (divided in tertiles for Kaplan-Meier analysis); involvement of ciliary body by the tumor (not involved, involved); height and largest basal diameter of the tumor (LBD; divided in three categories for Kaplan-Meier analysis); presence of epithelioid cells (absent, present); grade of pigmentation (weak, strong); presence of microvascular loops and networks, consisting of at least three back-to-back loops,

^{ 10 }analyzed as an ordered categorical variable that considers networks to be an advanced stage of loops

^{ 8 }(no loops, loops without networks, loops forming networks); and microvascular density (MVD) obtained from the most highly vascularized area (hot spot) of the tumor (divided in quartiles for Kaplan-Meier analysis and square-root transformed for Cox regression).

^{ 9 }

^{ 11 }

^{ 12 }

^{ 13 }An average survival curve representing each sampling strategy was obtained by plotting the survival for all 80,000 patients in each bootstrapped set. It can be shown that this curve closely approximates the survival curve that would be obtained by calculating point by point the average of the 1000 individual survival curves. Patients judged to die of causes unrelated to uveal melanoma were censored at the time of death. Confidence intervals were calculated according to Greenwood.

^{ 14 }Empiric confidence limits were obtained by plotting the individual survival curves for the first 100 replications in each bootstrapped set.

^{ 13 }Bias related to hypothesis testing was quantified by calculating the ratio of the average log-rank test χ

^{2}statistic, obtained as the mean of all replications in each set, to that of the conventional strategy. The probability corresponding to the average χ

^{2}statistic was obtained by computer (StaTable, ver. 1.0.1; Cytel Co., Cambridge, MA).

^{2}statistic

^{ 13 }

^{ 15 }for each independent variable, obtained as the mean of all replications in each set. In addition, a previously derived multivariate model was fitted.

^{ 9 }Bias related to HR was quantified by calculating the ratio of the corresponding HR estimate to the estimate obtained from data on the 133 patients in the starting data set. Bias related to hypothesis testing was quantified by calculating the ratio of the corresponding Wald χ

^{2}statistic to that obtained by the conventional strategy.

^{2}statistic of the log-rank test by a median of 1.08 times (range, 0.73–1.87), and the fifty-fifty strategy inflated it by a median of 1.14 times (range, 0.87–1.84), with corresponding undue improvement in the probability in both cases (Table 2) .

^{2}statistic by a median of 1.18 times (range, 0.79–2.13), and the fifty-fifty strategy inflated it by a median of 1.16 times (range, 0.71–2.02), with corresponding undue improvement in the probability compared with the conventional strategy (Table 3) . With the multivariate model, the χ

^{2}statistics of explanatory variables were also generally inflated (Table 3) .

^{ 5 }The bias was greater for subgroups with high mortality, and statistical significance probably was inflated. Similarly, studies that have used the fifty-fifty sampling design

^{ 1 }

^{ 2 }

^{ 3 }

^{ 4 }

^{ 6 }

^{ 7 }probably underestimated mortality by 3% to 5%. The bias for subgroups with low mortality was greater, which again inflated statistical significance to a variable extent. In addition, the fifty-fifty strategy is expected to increase the hazard ratio by a mean of 1.03 times. It is impossible to predict exact bias for individual covariates, however.

^{ 7 }According to a table published in the report of that study, the follow-up times of those who died of tumor (1–150 months) and those who were censored from the analysis (186–245 months) were very disparate.

^{ 7 }These observations uncover a fifty-fifty sampling strategy.

*P*= 0.035).

^{ 7 }The corresponding χ

^{2}statistic is 4.45. In the simulation, survival curves that similarly did not cross each other yielded χ

^{2}values that were 1.07 to 1.84 times larger than those obtained by conventional sampling. The correct χ

^{2}value consequently was probably between 2.42 and 4.16, corresponding to a true probability of 0.13 to 0.041.

^{ 1 }

^{ 2 }

^{ 4 }

^{ 5 }

^{ 6 }

^{ 7 }except once when they thought that variables associated with prognosis would be easier to spot and multiple covariates easier to compare.

^{ 3 }One might presume that they wanted to conduct a case–control type of study, which would be proper if the data were analyzed by logistic regression,

^{ 16 }a method that has been appropriately used to model survival in uveal melanoma.

^{ 17 }Another reason may be that a new assay was being tested on previously collected specimens and there were not enough resources to run the assay on every one.

^{ 18 }Logistic regression returns the log odds ratio of dying at a specified time point. Cox regression yields the relative hazard of death, which is assumed to be constant over time, unless time-dependent covariates are included in the model.

^{ 15 }

^{ 18 }Cox regression makes more efficient use of time-to-event data than does logistic regression, which disregards both data of patients who would be censored before the time point analyzed and the subsequent survival experience of patients who were alive at that point. Logistic regression is also indifferent to the exact time when death occurred. Survival analysis remains the most valid and the most efficient method to analyze time-to-event data.

^{ 19 }Proper construction depends on three factors: correct recording of the time of entry, recording of the time of death or censoring, and the assumption that the chance of being censored is unrelated to the risk of dying.

^{ 18 }

^{ 19 }The survival curve is calculated on the basis of patients who are at risk of dying on each successive day.

^{ 18 }

^{ 19 }Censoring occurs when the time to death is unknown because of termination of the study, loss to follow-up, or withdrawal for other reasons.

^{ 18 }Censored observation should be incomplete only because of random factors, so that, conditionally on the values of all explanatory variables, the prognosis for any patient who has survived a certain time should not be affected if he or she is censored.

^{ 15 }

^{ 20 }In particular, censoring must be unrelated to future lifetime.

^{ 21 }Whether a patient is included in a survival study in the first place must also be determined before knowing the outcome.

^{ 14 }

^{ 15 }

^{ 21 }These factors also distort the Cox regression analysis.

^{ 22 }

^{ 23 }

Data Set | Early Period^{*} | Late Period^{*} | Entire Sample | |||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Events | Censored | Events | Censored | Events | Censored | |||||||||

N | n (%) | n (%) | n (%) | n (%) | n (%) | n (%) | ||||||||

Underlying population | 133 | 49 (66) | 25 (34) | 14 (24) | 45 (76) | 63 (47) | 70 (53) | |||||||

Simulated data | ||||||||||||||

Conventional strategy | 80 | 29 (67) | 14 (33) | 9 (25) | 27 (75) | 38 (48) | 42 (52) | |||||||

Late-censoring strategy | 80 | 36 (100) | 0 (0) | 11 (24) | 33 (76) | 47 (58) | 33 (42) | |||||||

Fifty-fifty strategy | 80 | 31 (100) | 0 (0) | 9 (18) | 40 (82) | 40 (50) | 40 (50) | |||||||

Published data | ||||||||||||||

Late-censoring strategy^{ 5 } | 132 | N/R | N/R | N/R | N/R | 73 (55) | 59 (45) | |||||||

Fifty-fifty strategy^{ 7 } | 36 | 16 (100) | 0 (0)^{, †} | 2 (10) | 18 (90)^{, †} | 18 (50) | 18 (50) |

**Figure 1.**

**Figure 1.**

**Figure 2.**

**Figure 2.**

Explanatory Variable^{*} | df | Conventional Strategy | Late-Censoring Strategy | Fifty-fifty Strategy | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

χ^{2} | P | χ^{2} | Ratio^{, †} | P | χ^{2} | Ratio^{, †} | P | |||||||

Gender^{, ‡} | 1 | 1.64 | 0.20 | 1.20 | 0.73 | 0.27 | 1.42 | 0.87 | 0.23 | |||||

Age at enucleation | 2 | 6.16 | 0.046 | 11.53 | 1.87 | 0.0031 | 11.31 | 1.84 | 0.0035 | |||||

Ciliary body involvement | 1 | 9.64 | 0.0019 | 10.51 | 1.09 | 0.0012 | 11.71 | 1.21 | 0.00062 | |||||

Tumor height | 2 | 10.20 | 0.0061 | 14.52 | 1.42 | 0.00071 | 15.94 | 1.56 | 0.00035 | |||||

Largest basal tumor diameter | 2 | 14.18 | 0.00083 | 20.04 | 1.41 | <0.00001 | 19.13 | 1.35 | 0.00007 | |||||

Epithelioid cells | 1 | 15.12 | 0.00052 | 16.46 | 1.09 | 0.00027 | 16.92 | 1.12 | <0.00001 | |||||

Grade of pigmentation | 1 | 7.38 | 0.0066 | 7.98 | 1.08 | 0.0047 | 7.93 | 1.07 | 0.0049 | |||||

Microvascular patterns^{, ‡} | 2 | 13.27 | 0.0013 | 15.31 | 1.15 | 0.00047 | 14.93 | 1.13 | 0.00057 | |||||

Microvascular density | 3 | 18.25 | 0.00039 | 19.47 | 1.07 | 0.00022 | 20.84 | 1.14 | 0.00011 |

Explanatory Variable | Population Estimate Hazard Ratio | Conventional Strategy | Late-Censoring Strategy | Fifty-fifty Strategy | ||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Hazard Ratio | Wald Test | Hazard Ratio | Wald Test | Hazard Ratio | Wald Test | |||||||||||||||||||||

Estimate | Ratio | χ^{2} | P | Estimate | Ratio^{*} | χ^{2} | Ratio^{, †} | P | Estimate | Ratio^{*} | χ^{2} | Ratio^{, †} | P | |||||||||||||

Univariate analysis | ||||||||||||||||||||||||||

Gender^{, ‡} | 1.322 | 1.407 | 1.06 | 1.53 | 0.22 | 1.237 | 0.94 | 1.21 | 0.79 | 0.27 | 1.245 | 0.94 | 1.09 | 0.71 | 0.30 | |||||||||||

Age at enucleation^{, §} | 1.027 | 1.027 | 1.00 | 4.71 | 0.03 | 1.039 | 1.01 | 10.04 | 2.13 | 0.0015 | 1.041 | 1.01 | 9.50 | 2.02 | 0.0021 | |||||||||||

Ciliary body involvement^{, ∥} | 2.532 | 2.719 | 1.07 | 8.19 | 0.0042 | 2.636 | 1.04 | 9.50 | 1.16 | 0.0021 | 2.862 | 1.13 | 9.72 | 1.19 | 0.0018 | |||||||||||

Tumor height^{, §} | 1.133 | 1.139 | 1.01 | 8.23 | 0.0041 | 1.183 | 1.04 | 14.00 | 1.70 | 0.00018 | 1.202 | 1.06 | 14.57 | 1.77 | 0.00014 | |||||||||||

Largest basal tumor diameter^{, §} | 1.127 | 1.136 | 1.01 | 8.75 | 0.0031 | 1.149 | 1.02 | 12.51 | 1.43 | 0.00040 | 1.150 | 1.02 | 11.40 | 1.30 | 0.00073 | |||||||||||

Epithelioid cells^{, ¶} | 3.089 | 3.389 | 1.10 | 12.63 | 0.00038 | 3.287 | 1.06 | 15.13 | 1.20 | 0.00010 | 3.504 | 1.13 | 14.51 | 1.15 | 0.00014 | |||||||||||

Grade of pigmentation^{, #} | 3.144 | 3.790 | 1.21 | 5.95 | 0.015 | 3.500 | 1.11 | 6.70 | 1.13 | 0.0096 | 3.890 | 1.24 | 6.37 | 1.07 | 0.012 | |||||||||||

Microvascular patterns^{, **} | 1.833 | 1.905 | 1.04 | 10.28 | 0.0013 | 1.861 | 1.02 | 12.05 | 1.18 | 0.00050 | 1.957 | 1.07 | 11.92 | 1.16 | 0.00056 | |||||||||||

Microvascular density^{, ††} | 1.338 | 1.360 | 1.02 | 12.04 | 0.00052 | 1.363 | 1.02 | 14.31 | 1.19 | 0.00016 | 1.390 | 1.04 | 14.18 | 1.18 | 0.00017 | |||||||||||

Multivariate Analysis | ||||||||||||||||||||||||||

Largest basal tumor diameter^{, §} | 1.105 | 1.114 | 1.01 | 5.56 | 0.018 | 1.117 | 1.01 | 6.94 | 1.25 | 0.0084 | 1.115 | 1.01 | 6.07 | 1.09 | 0.014 | |||||||||||

Epithelioid cells^{, ¶} | 2.437 | 2.807 | 1.15 | 7.70 | 0.0055 | 2.627 | 1.08 | 8.73 | 1.13 | 0.0031 | 2.908 | 1.19 | 8.98 | 1.17 | 0.0027 | |||||||||||

Microvascular patterns^{, **} | 1.348 | 1.451 | 1.08 | 3.55 | 0.060 | 1.430 | 1.06 | 3.92 | 1.10 | 0.048 | 1.373 | 1.02 | 2.81 | 0.79 | 0.094 | |||||||||||

Microvascular density^{, ††} | 1.231 | 1.248 | 1.01 | 6.07 | 0.014 | 1.262 | 1.03 | 6.85 | 1.13 | 0.0089 | 1.281 | 1.04 | 6.56 | 1.08 | 0.010 |