Reproducibility of Endothelial Assessment during Corneal Organ Culture: Comparison of a Computer-Assisted Analyzer with Manual Methods

Abstract

purpose. To investigate the reproducibility of endothelial assessment of organ-cultured corneas with the computer-assisted Sambacornea analyzer in comparison with manual methods.

methods. Seven observers of two eye banks determined the endothelial cell density (ECD) of 30 corneas through a grid overlay placed on endothelial photographs using two manual modes, unaided (naked-eye) and pointing (point-out). ECD was measured with the analyzer, first in automated mode, where analysis was completely machine determined, and then in touched-up mode, where the observer selected the analysis zone and corrected poorly drawn cell borders. Interobserver variability of ECD for the different methods was compared. Reproducibility of morphometry parameters was determined for the touched-up mode.

results. Interobserver variability was ±19.2% (95% confidence interval [CI], 13.0–25.4) and ±17.6% (95% CI, 11.9–23.3) for the naked-eye and point-out mode, respectively, whereas the touched-up mode gave the least variability of ±9.6% (95% CI, 6.5–12.7), confirmed by the highest intraclass correlation coefficient of 0.95 (95% CI, 0.91–0.97). Interobserver variability increased with worsening image quality. Manual modes underestimated ECD (naked-eye by a mean 10.7% [SD, 2.9%]; point-out by a mean 6.9% [SD, 2.3%]), whereas the automated mode overestimated ECD by a mean 14.7% (SD, 24.3%). Reproducibility of morphometric parameters by the touched-up mode was acceptable but was influenced by endothelial pleomorphism.

conclusions. Manual counting shows systematic underestimation of ECD with high interobserver variability. The analyzer in automated mode overestimates ECD and is absolutely unreliable. Detection of cell contours by the specific algorithm, combined with manual correction by a skilled technician, appears to be the most reliable method of ECD and morphometry determination.

Endothelial cell density (ECD, expressed in cells/mm²) is the principal quantitative criterion of the suitability of corneas stored in eye banks for graft. In Europe, where organ culture is the preferred method of storage, ECD evaluation is performed with the use of a light microscope. ^{1 2} The first step, contrary to specular microscopy evaluation, is to render the endothelial cells (ECs) visible thanks to an osmotic dilation of intercellular spaces using generally 0.9% sodium chloride. ³ Several counting methods exist and can be divided into 2 main categories, manual and computer aided. 

Throughout Europe, manual counting is more commonly performed by observation through a calibrated reticule fitted in the eyepiece or by printouts of endothelial images, with cells pointed out on a grid overlay of a known surface area. Manual methods are subject to instrument- and observer-related variations. We previously reported on the unacceptable lack of reliability of manual counts in 21 French eye banks with interbank and intrabank (i.e., intertechnician) variations; 59% of the counts varied more than 10% of the actual ECD, and the deviation ranged between 82% overestimation and 42% underestimation. This was attributed mainly to a lack of microscope calibration and the use of different counting strategies. ^{4 5}  

Computer-assisted analyzers with digitized light microscopy images have been used for endothelial evaluation since the mid-1980s. ^{6 7 8} Recent years have seen the introduction of digital image acquisition and improved computer algorithms, which allow a reliable estimation of endothelial parameters. ^{9 10} Recently, a fully automated analyzer based on Fourier transformation of endothelial images without cell contour recognition was validated and is an encouraging sign for the modernization of eye bank endothelial assessment techniques. ¹⁰ The tri-image analyzer Sambacornea (Sambatechnologies, Meylan, France), which was based on a prototype developed at our research laboratory in collaboration with the Centre of Medical Engineering of Saint Étienne, measures ECD by automatic cell contour recognition. ⁹ Application of this algorithm, in combination with observer-mediated manual corrections tested on standard microlithographic mosaics with mathematically predetermined cell densities mimicking the human endothelium, has been shown to be accurate. ¹¹ Nevertheless, regarding routine application on organ-cultured corneas, the usefulness of the analyzer to reduce the interobserver variability and its comparison with manual counting methods remained to be evaluated. Given that automated and semiautomated counting methods exist in the analyzer, the purpose of this study was to investigate their reproducibility in comparison with the two manual counting methods. 

Materials and Methods

Endothelial Imaging

Corneas stored in organ culture at 31°C in medium (CorneaMax [Eurobio, Les Ulys, France] or Inosol [Baush and Lomb Chauvin Opsia, Labége, France]) were studied before the deswelling process. The endothelial surface was incubated with 0.9% sodium chloride (Aguettant, Lyon, France) for 4 minutes for visualization of the cell borders by a process of gradual dilation of the intercellular spaces. This is a dynamic and transient phenomenon limited by an equilibration between the intracellular and the extracellular spaces, after which cell borders again become less obvious. Endothelium was viewed through a long working distance 10× objective under a direct optical microscope (DMLB; Leica Microsystems GmbH, Wetzlar, Germany). When cell borders were optimally discernible, endothelial photographs were acquired using a monochrome CCD video camera (XC-ST50CE; Sony, Tokyo, Japan) installed on the microscope and digitized using a video frame grabber (DT-3155; Data Translation, Marlboro, MA). Three wide-field (1000 μm × 750 μm) images of three randomly chosen nonadjacent zones of the endothelium contained within the central 8-mm diameter were taken at a resolution of 768 × 576 pixels in 8-bit gray level and saved in bitmap (BMP) format. Image quality was graded by two observers on a three-level score, depending on the visualization of cell borders, background noise, and surface area of the image with visible ECs. ¹² The score was deemed good if the cell borders were very well visualized, if the background noise was low or nonexistent, and if the cells were visible on two thirds or more of the image area. It was deemed average if visualization of the cell borders was good, if the background noise was moderate, and if the cells were visible on one third to two thirds of the image area. It was deemed poor if the cell borders were difficult or impossible to visualize, if the background noise was high, and if cell borders were visible on less than one third of the image area. Thirty corneas with images of different scores (good, 12 [40%]; average, 9 [30%]; poor, 9 [30%]) were chosen for the study so as to represent the routine eye bank practice. The sample size was chosen as a balance between the feasibility of involving multiple technicians of two eye banks in a research procedure conducted in addition to their routine clinical activities and the minimum number required for a valid statistical calculation. High-quality laser printouts of the images on premium matte photograph paper (HP Q6549A; Hewlett-Packard, Geneva, Switzerland) were obtained at resolution of 600 dpi in A4 format (Laser Shot LBP-1120; Canon Europe, Uxbridge, UK) for manual counting purposes. A certified micrometer (Leitz GmbH, Postfach, Germany) was also photographed in the same conditions and served as a reference scale for the endothelial images. A grid overlay simulating the microscope reticule, with each vertical column designated from A to H and each horizontal line from 1 to 6, was designed with software (Photoshop version 7.0.1; Adobe Inc., San Jose, CA) and printed on A4-sized transparency sheets (Epson, Long Beach, CA) using the same printer. The dimensions of the grid were similar to the visual appearance of the grid placed in the eyepiece—the square of 120 μm × 120 μm (external dimension), with a line thickness of 2.6 μm—taking account the 10× objective and the internal magnification factor of that microscope. 

Manual Counting: Naked Eye and Point-Out

Seven skilled observers, each having performed more than 500 counts and affiliated with either of two eye banks (Grenoble and Saint Étienne), participated in the study. Because our study required multiple observers of different eye banks to perform counts on the same cornea, manual counting in real time using the light microscope was not feasible. Hence, the eye bank counting procedure—printouts of endothelial images with a transparent calibrated grid overlay (Fig. 1) —was simulated for study purposes. In the naked-eye mode, the printed grid was accurately aligned over each endothelial photograph, and 10 nonadjacent reticule units, where cells were most clearly visible (four on image 1 and three each on images 2 and 3) were chosen and their coordinates noted. Accurate alignment, though theoretically not necessary for the naked-eye mode, was essential to ensure counting of identical zones with the second method. Cells were counted with the fixed-frame technique, whereby those within the reticule unit (except those touching the adjacent right and bottom borders) were included. Counting was based on a visual impression using only a manual counter, without any pointing device. Counting was repeated using another method, the point-out mode, whereby cells contained in the same 10 reticule units on the three images were counted using the fixed-frame technique, this time pointing out individual cells with a marker. All 30 corneas were serially counted first with naked-eye mode and then with the point-out mode to avoid any observer bias caused by memorization. Between 250 and 400 cells (25–40 cells per reticule unit) were counted for each cornea, depending on the ECD, to maintain uniformity with the computer-assisted analysis. The mean of the 10 reticule unit counts for a particular cornea was obtained, and from that the ECD was calculated as follows: ECD (cells/mm²) = (mean cell count × 10⁶)/(120–2.6)². No quantitative morphometric evaluation was possible for either manual method. 

Computer-Assisted Counting: Automated and Touched Up

The tri-image eye bank analyzer Sambacornea, consisting of a personal computer–compatible 1.3-GHz processor (Pentium 4; Intel, Santa Clara, CA) with 256 MB RAM and equipped with a 21-inch CRT monitor, was used for computer-assisted counting. We used an upgraded version (version 1.2.10) of the prototype software earlier described, ⁹ which permits detection of probable errors in contour recognition such as cells abnormally small (<100 μm²) or large (>1000 μm²) or of widths greater than twice the length; the last two probably indicate poorly separated cells. Instrument calibration was performed with the same certified micrometer (Leitz). The same three images of each cornea were viewed simultaneously, and a minimum of 300 cells (approximately 80–120 cells per image) was analyzed using two different strategies, both based on a variable frame technique. 

First, each image was analyzed by the automated mode, where the algorithm itself selected the counting zone and subsequently performed thresholding and contour recognition, technical details of which are mentioned elsewhere. ⁹ The procedure required no human participation. Cells with probable errors in contour recognition were eliminated by the software from the final analysis. Second, images were analyzed with the touched-up mode, where the observer selected the counting zones as free forms, carefully avoiding the stromal folds caused by swelling in organ culture storage and liable to produce an artifactual counting error. This zone was next subjected to manual threshold adjustments and contour recognition by the algorithm, followed by manual correction of poorly recognized cell contours, if necessary. For each method, the software calculated the mean cell area from which the ECD, the SD, and the coefficient of variation (CV) of cell area were calculated (Fig. 2) . Numbers of cells with six neighbors gave the percentage of hexagonality. 

Statistical Analysis

Because our purpose was to compare the analysis methods, assuming each cornea was evaluated seven times, in the absence of a true known value for ECD or a morphometric parameter among the seven observers, we used a method similar to that described by Bland and Altman. ¹³ Regarding ECD determination with each method of counting, we plotted the difference of each observer with the mean of seven (expressed in percentages) against the mean of the seven observers for each cornea (reference value). The interval containing 95% of the values was defined as the limit of agreement and was calculated as mean ± 2 SD of the mean difference. This interval was considered a direct illustration of the variability of the method studied. Considering the sample size of 30 corneas, the 95% confidence interval (CI) of the bounds of agreement was calculated. We also calculated, for each counting method, the interobserver agreement expressed by the intraclass correlation coefficients (ICCs) with their 95% CIs. ¹⁴ Agreement was considered poor for ICCs lower than 0.40, fair for those from 0.40 to 0.59, good for those from 0.6 to 0.74, and excellent for those greater than 0.74 according to the criteria of Cicchetti and Sparrow. ¹⁵ Endothelial cell density obtained by the different methods was compared using a nonparametric test for paired data. 

To assess the interobserver agreement for analysis of morphometry parameters (CV and hexagonality) determined only for the touched-up mode with the analyzer, the same Bland-Altman–like method was used whereby for each cornea, the variation against the mean (in percentage) for each observer was plotted against the mean for seven observers. ¹³ The limits of agreement, calculated as mean ± 2 SD of the difference, delineated the interval containing 95% of the values. Statistical data were then analyzed (SPSS 11.5; SPSS Inc., Chicago, IL). 

Results

ECD by Manual Methods and the Analyzer

Interobserver Agreement.

Manual methods were associated with the highest interobserver variability, with a variation of ±19.2% (95% CI, 13.0–25.4) for the naked-eye mode and ±17.6% (95% CI, 11.9–23.3) for the point-out mode. Regarding counting with the analyzer, no variability was logically found for the automated mode. For the touched-up mode, variability was ±9.6% (95% CI, 6.5–12.7). The automated mode was not reliable, especially for low ECDs with a net tendency for overestimation. Persistent variability, twice as low as that for manual counts, remained in the touched-up mode because of the poor quality images in this limited set of 30 corneas (Fig. 3) . Interobserver variability was ±7.8% (95% CI, 3.6–12.0) for images of good quality (n = 12), ±8.1% (95% CI, 2.9–13.3) for images of average quality (n = 9), and ±12.8% (95% CI, 4.4–21.2) for images of poor quality (n = 9). Interobserver agreement determined by ICC confirmed these results. Excluding the automated mode logically showing zero variation, the touched-up mode was associated with the highest interobserver agreement of 0.95 (95% CI, 0.91–0.97) compared with 0.82 (95% CI, 0.56–0.92) for the point-out mode and 0.79 (95% CI, 0.54–0.90) for the naked-eye mode. 

Accuracy of Counting Strategy.

The mean ECD of the seven observers for each of the 30 corneas obtained with the automated and two manual methods was compared to determine their respective reliability, taking the touched-up mode as the reference given that this demonstrated the highest interobserver agreement and that its exactness has been demonstrated. ¹¹ All three were significantly correlated with the touched-up mode (assessed by the determination of the Pearson correlation coefficient R; P < 0.001 for all), with the automated mode showing poor correlation (R = 0.65) and both manual modes showing excellent correlation (R = 0.99 for both). ECD was underestimated by both manual methods (means, 10.7% [SD, 2.9] for naked-eye mode and 6.9% [SD, 2.3] for point-out mode) and was overestimated by a mean 14.7% (SD, 24.3) in the automated mode. Moreover, an important drift was noted for the automated mode that showed increasing overestimation when the ECD decreased. Such a drift was not observed with either manual mode. 

Reproducibility of Morphometry Assessment with the Touched-Up Mode

Considering that ECD evaluation by the automated mode of the Sambacornea analyzer was not reliable, only morphometry data obtained with the touched-up mode is presented here. Interobserver variations of ±6.9% (95% CI [4.6–9.1]) for CV and ±7.0% (95% CI [4.7–9.3]) for hexagonality were observed. No correlation was found between image quality and variation of CV or hexagonality (data not shown). 

Discussion

In our study, manual counting, still the most frequent practice in European eye banks, significantly underestimated the ECD (by nearly 10%) and had high interobserver variability. The computer-assisted analyzer in touched-up mode reduced the interobserver variability by half. We have demonstrated ¹¹ the exactness of the touched-up mode in an application using standard microlithography mosaics with mathematically predetermined ECD, and our present results show its reliability for measuring endothelial parameters for routine eye bank corneas. 

The automated mode of the analyzer Sambacornea produces significant overestimation and high SD, especially for lower ECDs, and is definitely not reliable, as has also been noted in previous specular microscopy studies using automated modes. ^{16 17 18} The lack of reliability may be attributed to several factors. Software, especially for images with poor quality and lack of sufficient contrast, fails to correctly identify the cell borders, resulting in the recognition of cells of abnormal size (too big or too long) that actually represent poorly separated cells. Zones with the best contrast are favored by the software for analysis; this often includes areas of corneal folding, which give an overestimation because of parallax error. Images used for analysis with the computer algorithm are widefield (1000 μm × 750 μm with the microscope used here and depending on microscope type) and tend to include areas with stromal folds. Ruggeri et al., ¹⁰ using images with a still wider field (1256 μm × 940 μm), reported good correlation between manual and fully automated counts. The authors used only good quality images for correct functioning of their algorithm. They also used a modified manual count technique, taking into account an analysis zone seven times larger (approximately 314 μm × 235 μm) than the routine zone as the reference in their study. 

Semiautomated counting with the analyzer, with the observer selecting the counting zone, choosing the best threshold levels, and retracing the cell contours, overcomes most of the drawbacks observed with the automated and manual techniques. Unlike what occurs with the automated mode, with the semiautomated mode the counting zone is delineated by the technician as a free form. Therefore, choice of an optimally visible zone (or zones considered representative of the rest of the central cornea, even if it is not optimally discernible) and circumvention of stromal folds is possible, limiting the necessity of manual retouch and improving the performance of the algorithm. Manual correction of cell contours incorrectly drawn by the algorithm further improves the reliability of the procedure; hence, human judgment and intervention contribute to reducing the software-induced errors. 

Underestimation with manual fixed-frame counting compared with an endothelial analyzer using a semiautomated approach with manual correction has been demonstrated. ^{7 9} Unacceptable interbank and intrabank variations in manual counts noted in our previous studies were attributed to improper microscope calibration and differences in counting strategy. ^{4 5} Both factors were neutralized in the present study because, for the manual counts, images taken after appropriate microscope calibration were analyzed by all observers with the same fixed-frame counting strategy. For computer-assisted analysis, the same three images were used, as was the touched-up mode, whose exactness had already been established. ¹¹ Moreover, we counted more or less uniform samples (250–400 cells) for manual and semiautomated counts, which should have reduced the variability. In spite of these, the differences in reproducibility observed between the two manual modes and the semiautomated mode were important and could be logically explained by several factors. 

One factor could have been the choice of counting zone used by various observers for the different counting modes. This, however, seems less contributive because in all three modes, zones in which the cells were optimally visible were selected by the observers and were nearly identical because of the careful avoidance of stromal folds. This factor is supported by the high degree of correlation found among the seven observers for the different modes (0.95 for semiautomated versus 0.82 for point-out versus 0.79 for naked eye). 

Another factor lay in the counting strategy itself, which was directly related to the human factor and likely played a major role. The fixed-frame strategy was used for manual counting and was only a convenient approximation of the real count (designed in the beginning to count numerous floating cells in a hemocytometer and later adapted to contiguous objects ¹⁹ ). The fixed-frame technique is thought to be associated with an inherent variability given the small counting unit (<10,000 μm²), and it is particularly liable to show underestimations for corneas with high ECDs. ^{4 20} For manual counts, errors occur even for expert personnel, whether they count in real time under a light microscope or in a deferred manner on endothelial images displayed on a computer monitor. The fixed-frame strategy is influenced by inconsistencies on the part of the observer regarding the inclusion of cells lying close to or touching the reticule margins. This is important because missing even a single cell out of an average 25 to 40 cells in the 100 μm × 100 μm reticule would result in an underestimation of 2.5% to 4%. 

The Sambacornea algorithm makes use of a variable-frame technique, where the analysis zone is selected manually as a free form, contrary to the principle used in other endothelial analyzers where only a fixed frame ^{7 8} or the entire field ¹⁰ is analyzed. This variable frame totally suppresses the specific source of variability described, but the differences in extent and nature of manual corrections between observers, especially with images of poor quality and corneas with irregular endothelia, are also liable to induce new variations. In our study, the interobserver variability for ECD assessment increased with the worsening of image quality. This is in agreement with findings reported by the Cornea Donor Study Group. ^{21 22} Image quality is dependent on multiple factors, such as response of ECs to osmotic challenge, corneal folding, and microscope field-depth problems. Ruggeri et al. ¹⁰ advocated completely automated cell counts with their analyzer using only good quality images after excluding images that could not be processed. To enable its application in eye banks, where poor quality images are not uncommon, the touched-up mode of the analyzer, which allows human intervention to compensate for deficiencies in image quality, is a definite advantage. 

Another advantage of the Sambacornea analyzer over manual modes is the possibility of morphometry assessment (CV and percentage hexagonality), which is considered a reflection of the functional properties of the ECs. ^{23 24 25} This is the only analyzer described in the published literature, other than that described by Barisani-Asenbauer, ⁷ that allows morphometric evaluation because the algorithm functions on the principle of contour detection. However, it remains susceptible to interobserver variations, especially for corneas with high pleomorphism and polymegathism. Though this susceptibility may be partly ascribed to differences in manual corrections between observers for poorly visualized but relatively regular mosaics, for corneas with polymegathous endothelia, the calculation of CV has been known to deviate between 6% and 8% from the mean, even if a large sample (nearly 100 cells) is counted. ²⁶ In the absence of any absolute cutoff levels for morphometric criteria of eye bank corneas, the variations shown by the touched-up mode could be considered to be within clinically acceptable limits. 

In conclusion, we recommend a semiautomated, not a fully automated, approach to obtain reliable endothelial assessment. We insist on the role of an eye bank technician to guide the evaluation process, especially in the choice of an analysis zone and the correction of cell contours, if necessary. We also recognize the importance of image quality because that influences the reliability and, hence, the need to optimize image quality using the best settings of the microscope and the imaging device. Further research is under way to explore endothelial assessment using three-dimensional corneal imaging, which will allow a reliable and less observer-dependent evaluation by facilitating counting on the slopes and depths of corneal folds. 

 Submitted for publication September 4, 2006; revised November 16 and December 18, 2006; accepted March 5, 2007.

 Disclosure: N. Deb-Joardar, None; G. Thuret, None; Y. Gavet, None; S. Acquart, None; O. Garraud, None; H. Egelhoffer, None; P. Gain, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Figure 1.

 

Diagram showing the manual count set-up consisting of the endothelial image printout and the transparent grid overlay used for naked-eye and point-out modes. Each square of the grid measured 120 μm × 120 μm (external area). For the naked-eye mode, cells were counted to avoid the stromal fold (arrow) and without any aid. For the point-out mode, these were indicated on the transparency sheet with a marker (dots). Here we show the illustration of the point-out mode of a cornea graded as good. In some cases, the decision to include a cell touching the lower (black arrowhead) or the right (white arrowhead) reticule border was questionable, thus increasing the likelihood of error.

Figure 2.

 

Final computer-assisted endothelial analysis report of a cornea in touched-up mode. The blue line shows the analysis zone selected by the observer as a free form. Cell contours obtained after threshold adjustments and manual corrections are shown in green. Cells in orange represent the smallest and the biggest cells, facilitating the ultimate detection of probable gross errors. The section on the right shows the identification details on top and the cellular parameters below: ECD (calculated on 328 cells comprising 111, 117, and 100 cells on images 1, 2, and 3, respectively; minimum/maximum/mean cell area and SD; CV of cell area; and percentage of hexagonal cells (calculated on 263 cells after exclusion of cells intersected by the blue selection line). Images were graded as good (see Fig. 3C ), and this cornea showed a high interobserver variation for CV because of the high polymegathism.

Figure 3.

 

Endothelial images of a cornea with (A, first line) good image quality and lowest interobserver variation for measurement of ECD (–2.5% to +2.5%; mean of absolute values, 1.5%) and CV (–0.6% to 2.2%; mean of absolute values, 0.71%). (B, middle line) Poor image quality and highest variation for ECD (–19.0% to +24.7%; mean of absolute values, 11.5%) and CV (–13.7% to 17.6%; mean of absolute values, 7.6%). (C, last line) Good image quality and high variation for CV because of high polymegathism (–11.0% to 5.2%; mean of absolute values, 5.6%).

The authors thank the researchers and personnel (Marie-Claude Legal, Laurence Puillet, Christian Theilliere, Chloé Manissolle, Souad Herrag) of the Saint Étienne and Grenoble eye banks for their valuable contributions and Arnaud Meulle for coordinating the intercenter evaluation.