Abstract
Purpose:
To validate the Liverpool Uveal Melanoma Prognosticator Online (LUMPO) in a cohort of patients treated at the University of California-San Francisco (UCSF).
Methods:
A retrospective chart review was performed of 390 patients treated between 2002 and 2007 for choroidal melanoma at UCSF. Similar patients (n = 1175) treated at the Liverpool Ocular Oncology Centre (LOOC) were included in the study. The data were analyzed using the model previously developed for LUMPO, an online prognostication tool combining multiple prognostic factors. Main outcome measures included all-cause mortality and melanoma-specific mortality. Reliability of the survival estimates in each group of patients was indicated by the C-indices of discrimination and Hosmer-Lemeshow test.
Results:
Patients treated at UCSF tended to be younger with thicker tumors, and were more likely to receive proton beam radiotherapy as primary treatment compared to patients at LOOC. There were no significance differences with respect to ciliary body involvement, melanoma cytomorphology, and mitotic counts between the two groups. Death occurred in 140/390 (35%) patients from UCSF and 409/1175 (34%) patients from LOOC, with no difference in overall mortality by Kaplan-Meier analysis (log rank test, P = 0.503). For all-cause mortality and melanoma-specific mortality, the C-index of discrimination and Hosmer-Lemeshow test at 5 years after treatment indicated good discrimination performance of the model, with no statistically significant difference between observed and predicted survival.
Conclusions:
Despite differences between the two cohorts, external validation in patients treated at UCSF indicates that LUMPO estimated the all-cause and melanoma-specific mortality well.
Approximately 45% of patients with uveal melanoma will die from metastatic disease within a 15-year period after treatment of their primary tumor.
1 Uveal melanoma spreads hematogenously, with the most common site of metastasis being the liver. Metastases are rarely detectable at the time of primary ocular treatment and become evident months or years after apparent good health, which indicates the need for estimating if and when metastatic disease is likely to develop. Such prognostication, if sufficiently reliable, is reassuring for patients with minimal risk for metastasis.
2,3 Although systemic surveillance for metastases does not have a clear survival benefit,
4 identifying patients at high risk for metastasis is important for clinical trial entry and any further treatments for metastatic disease that may become available in the future. There is also some evidence that quality of life is improved by prognostication, even when the survival probability is found to be poor.
2
There are many prognostic factors for metastatic death from uveal melanoma. Anatomic predictors, which form the basis of the American Joint Committee on Cancer (AJCC) Tumor, Node Metastasis (TNM) staging, include largest basal tumor diameter, tumor thickness, ciliary body involvement, and extraocular extension.
5–9 The TNM staging system is imprecise and is therefore applicable only to large groups of patients and not to individual patients.
10 Histologic predictors include epithelioid melanoma cytomorphology, high mitotic count and certain extravascular matrix patterns, particularly closed loops, and tumor microvascular density.
11–15 Genetic predictors include partial or total chromosome 3 loss (“monosomy 3”), chromosome 8q gain, lack of chromosome 6p gain, and class 2 gene expression profile (GEP).
16–21 Uveal melanomas were also reported to fall into two classes based on GEP, with a 92-month survival probability of 95% in class 1 vs. 31% in class 2, and with nearly all metastatic deaths occurring among class 2 patients.
17 However, it was later found that a small percentage of class 1 tumors also give rise to metastasis, and class 1 tumors were subsequently subdivided into class 1A (“low risk”) with 2% risk of metastasis, and class 1B (“intermediate risk”) with 21% risk of metastasis.
22 A recent study found that that
PRAME (preferably expressed antigen in melanoma) confers increased metastatic risk in class 1 or disomy 3 tumors.
23
Although genetic typing is highly accurate in determining which patients will develop metastatic disease, it does not predict when such disease is likely to develop. Survival time is better estimated by combining anatomic, histologic, and genetic predictors.
24 However, a problem with such multivariate analysis is bias caused by missing data. Bias also arises because of competing causes of death unrelated to the uveal melanoma.
25 Because these are censored in Kaplan-Meier analysis, this tends to exaggerate the apparent metastatic mortality, especially in elderly patients, who are more likely to die of other causes.
1,16,26
The Liverpool Uveal Melanoma Prognosticator Online (LUMPO) is an online tool that was developed to estimate survival probability after treatment of choroidal melanoma.
10 It combines clinical, histologic, and genetic data to enhance reliability of prognostication in individual patients and to estimate survival time as well as risk of metastasis. It minimizes bias from missing data by extrapolating from other predictors. To avoid bias from competing risks, LUMPO estimates relative survival using all-cause mortality in patients with choroidal melanoma and compares it to mortality from that of the general British population matched for age and sex using census data. This prognostic tool was developed and validated in 2012 with data on 3653 British patients, with histologic data in 1778 patients and genetic data in 738. A total of 1235 patients had died, and the model was calibrated using all-cause mortality.
10
The LUMPO consists of two models, one of which uses only anatomic data (the “clinical” model) and the other also including histologic and genetic data (the “laboratory” model). These models were validated internally at the Liverpool Ocular Oncology Centre (LOOC) by bootstrapping.
10 Bootstrapping is a method commonly used for internal validation in which a bootstrap sample is selected from the original sample where each subject has equal probability of being selected, and each can be selected more than once. This is repeated several hundred times, and the variance of statistics is analyzed for internal validation of a prediction model. However, to our knowledge, LUMPO has not previously been validated externally at another institution, and it is not known whether it is generalizable to patients treated elsewhere.
The aim of our study was to validate LUMPO in a cohort of patients treated at the University of California-San Francisco (UCSF).
A retrospective chart review was performed on all patients treated for choroidal melanoma at UCSF between 2002 and 2007. Patients were excluded if they had bilateral choroidal melanoma, or if there was missing information regarding baseline tumor diameter and tumor thickness. Patients were also excluded if they were known to have metastases at the time of diagnosis or if they did not receive treatment.
Patient demographics included age and sex. Anatomic data collected included ciliary body involvement (i.e., pars plicata and pars plana), extraocular spread, and ultrasonographic measurements of largest basal tumor diameter and thickness. Histologic information included the presence or absence of epithelioid cells and mitotic cell count (number of mitoses per 40 high-powered fields). Presence or absence of closed extravascular matrix loops was not available in the pathology reports from UCSF during this time period. None of the patients in the UCSF dataset had genetic data available because tumor biopsy and prognostic genetic typing had not yet been adopted.
Survival data were gathered from the UCSF Cancer Registry, which includes all cancer patients diagnosed and/or treated at UCSF. Patient vital status was updated at least annually through readmissions to the hospital, contact with primary care physicians, Department of Motor Vehicles records, contact with the patient's family, and information from the state registry. The UCSF Cancer Registry is required to update survival status at least every 15 months in a minimum of 80% of patients. Survival data for this study were obtained in 2014. The 2002–2007 time period was selected to allow for greater than 5 years of follow-up for all patients, and adequate time to gather sufficient survival information through these multiple methods. Due to the difficulty of obtaining paper charts from before 2002, this was selected as the starting point.
Data were obtained from the LOOC database using the same inclusion and exclusion criteria. The Liverpool Ocular Oncology Centre notified the U.K. National Health Service Cancer Registry of all patients with ocular melanoma at the time of their initial diagnosis. Survival information was then collected from the Cancer Registry, which automatically informed LOOC of the date and certified cause of death of all patients residing in mainland Britain (i.e., Scotland, England, and Wales but not Northern Ireland). Genetic data were available at LOOC beginning in 1999 with fluorescence in situ hybridization (FISH) performed on all patients undergoing local resection or enucleation, and on all consenting patients from 2007 using multiplex ligation-dependent probe amplification (MLPA) and/or microsatellite analysis (MSA).
The LUMPO model has been described by Eleuteri et al.
10 In brief, an accelerated failure time model was used,
27 and missing data points were estimated using the alternating conditional expectations algorithm,
28,29 which predicts each missing data point as a function of the other available pieces of information. If histologic or genetic data were available, the laboratory model was used. If both histologic and genetic information were not available, LUMPO would default to the clinical model.
Demographics and tumor characteristics of patients treated at LOOC were compared with those of UCSF by Fisher's exact test and Wilcoxon rank sum test. Survival of patients at LOOC and UCSF were compared using Kaplan-Meier analysis and log rank test. The performance of the model was assessed in two ways: discrimination using the Harrell C-index
30,31 and calibration using the Hosmer-Lemeshow test,
32,33 which are commonly used to assess prediction performance in survival analysis and to test goodness of fit in logistic regression, respectively. The Harrell C-index of discrimination measures the separation of two survival distributions, and any value of 0.7 or above within the 95% confidence interval indicates good performance. The Hosmer-Lemeshow test is a goodness-of-fit test for logistic regression. For this test, the null hypothesis is that there is no difference between observed and predicted mortality, so a
P value greater than 0.05 indicates that observed survival is not significantly different from predicted mortality.
30–33 All-cause and melanoma-specific mortality were analyzed, where melanoma-specific mortality was obtained by subtracting patient survival from control survival.
The Institutional Review Board of UCSF prospectively granted approval for this study (No. 13-11313), which followed the tenets of the Declaration of Helsinki.
The 5-year time period of 2002–2007 was selected to provide sufficient patient numbers, follow-up times, and deaths for statistical analysis. We excluded patients whose tumor did not involve choroid because these are rare, so they were also excluded from the LUMPO models.
Patients with missing tumor dimensions were excluded because it would not have been possible to determine whether inaccurate survival estimates were the result of incomplete data or failures of LUMPO, so validation of this predictive model may have been compromised. Patients who did not receive treatment were excluded, as prognostication begins at the time of treatment, and tumor characteristics such as dimensions and cell type are not arrested in these patients but may continue to change.
A 5-year time point was chosen for statistical analysis to allow sufficient follow-up time for all patients within the study group.
It will be important to perform external validation including genetic data once sufficient follow-up time has passed to allow survival analysis. Prognostic biopsies are now routinely being performed and genetic data have been gathered prospectively at UCSF beginning in 2013, and the inclusion of genetic data in future studies should only improve the performance of LUMPO. Upcoming studies could also include a longer follow-up time point, such as 10-year survival, once adequate time has passed. In addition, efforts are in progress to improve LUMPO using a larger dataset and more sensitive methods of genetic analysis. There is, as well, scope for further external validations at different centers to determine whether LUMPO can indeed be used widely. This indicates a need for more standardization in the way tumor size, extent, and grade of malignancy are measured and reported.
The GEP class 2 group of tumors includes most metastatic deaths; however, it is now known that a small percentage of class 1 tumors also metastasize—the class 1B or “intermediate risk” tumors.
22 PRAME has recently been found to confer increased metastatic risk in class 1 or disomy 3 tumors.
23 Mutations such as
SF3B1,
EIF1AX, and
BAP1 also correlate with survival and may therefore have prognostic importance in uveal melanomas with disomy 3 and partial monosomy 3.
41 It is likely that future versions of LUMPO will incorporate these and other new predictors for survival once sufficient data become available.
The authors thank Ann Griffin, PhD, of the University of California-San Francisco Cancer Center Registry for her assistance.
Disclosure: S.W. DeParis, None; A. Taktak, None; A. Eleuteri, None; W. Enanoria, None; H. Heimann, None; S.E. Coupland, None; B. Damato, None