Automatic Recognition of Corneal Nerve Structures in Images from Confocal Microscopy

Abstract

purpose. To devise a method for automatically tracing corneal nerves in confocal microscopy images.

methods. Images were acquired with a confocal microscope. They were normalized and enhanced in luminosity and contrast. The nerves were recognized by applying a novel tracing algorithm, which includes Gabor filtering to enhance nerve visibility and postprocessing procedures to remove false recognitions and to link sparse segments into continuous structures. A prototype of the algorithm was implemented in commercial software and run on a personal computer.

results. A retrospective evaluation of the automatic procedure was performed on a data set containing 90 images, from normal and non-normal subjects. The average percentage of correctly recognized nerves length with respect to total manually traced lengths of visible nerves was 80.4% in normal subjects and 83.8% on non-normal subjects; the average rate of false nerve length recognition (with respect to the total automatically traced length) was 6.5% in normal subjects and 9.1% in non-normal subjects. Correlation coefficients between manual and automatic lengths on the same image were 0.94, 0.95, and 0.86 in all, normal, and non-normal subjects, respectively. A further evaluation was performed on an independent set of 80 normal subject images, resulting in a correlation coefficient of 0.89 between manual and automatic nerve lengths.

conclusions. Automatic and manual length estimations on the same image were very well correlated, indicating that the automatic procedure is capable of correctly reproducing the differences in nerve length between different subjects.

In vivo confocal microscopy of the cornea (see, e.g., Ref. ¹ ) allows acquisition in a rapid and noninvasive way of images of the various corneal layers and thus extraction of important clinical information on the health status of the cornea. In particular, the images collected at a specific depth, the subbasal layer, allow the visualization of the nerve structures present in this section of the cornea. They are narrow and elongated structures lying flat inside a 10-μm layer (Fig. 1A) . These structures have been shown to be quite important in providing clinical information related to changes caused by ageing ² or prolonged contact lens wear ³ ; by corneal surgical interventions, such as LASIK, PRK, ^{4 5} or transplantation ⁶ ; or by such diseases as dry eye/Sjögren’s syndrome, ^{7 8} keratoconus, ^{9 10} or herpes keratitis. ¹¹ An important link has been shown between nerve tortuosity and the severity of diabetic neuropathy, one the most common and serious long-term complications of diabetes. ¹²  

At present, all these analyses are based on a tedious manual tracing of the nerves and thus the derived clinical parameters take a long time to retrieve and are subjective. An automatic tool, capable of reliably extracting the nerve course and layout and quantitating their geometric features (e.g., length, density, tortuosity), would provide a much easier, objective, and clinically usable procedure. It would thus be expected to improve the reliability of derived clinical parameters and to enhance their diagnostic value. This is a report of our work on the development of such a tool. 

Methods

Ninety images of corneal subbasal epithelium from normal (n = 76) and non-normal (n = 14) subjects were made available by Nidek Technologies (Padova, Italy) from their own image database. The privacy of the patients was protected in compliance with the Declaration of Helsinki. The images were acquired with a confocal microscope (ConfoScan4; Nidek Technologies, Padova, Italy), covering a field of 460 × 350 μm² at 40× magnification, and saved as JPEG compressed, monochrome, 768 × 576 pixel digital images. The central regions of corneas were examined using a coupling medium (gel) between eye and front surface of the lens. The front surface of the lens was advanced with a joystick attached to the main body of the instrument until the anterior layers of the cornea were visualized and then image acquisition began when the objective lens was properly positioned on the corneal apex by the instrument autoalignment module. On average, total duration of the examination was 2 minutes, and image acquisition time was 30 to 60 seconds. Informed consent was obtained by all subjects. 

All the 90 images of this dataset are publicly available for download. ¹³  

Preprocessing

Acquired images do not usually have a uniform luminosity and contrast, exhibiting, for example, darker areas in the peripheral regions of the image. This lack of uniformity is due to many factors, including the spherical shape of corneal layers, which causes a nonuniform reflection of illumination light in the different corneal areas, and the different attenuation of light along the various illumination paths. To compensate for this, a specific equalization procedure, which we originally developed to normalize luminosity and contrast in retinal images, ¹⁴ was applied. As this procedure increases the amplitude of noise as well, a median filter was then applied to reduce this artifact. In preprocessed images, nerve structures have a higher contrast with respect to background and appear more visible, even in the peripheral areas of the image (Fig. 1B) . 

Nerve Tracing

The algorithm starts by identifying a set of seed points, to be used as starting points for a nerve-tracking procedure. A line grid of equally spaced rows and columns (one every 10 pixels) is drawn over the image and its pixels are analyzed by looking for variations in their gray-level intensity that may suggest an intersection with nerves. A detection threshold was empirically set at 0.8 times the average gray level over the whole image and all analyzed pixels exceeding this threshold were considered as seeds (i.e., pixels belonging to nerves). On average, 600 seed points per image were detected. Lower values of the threshold, which actually provided more seed points, however, did not yield overall better tracing results. 

Starting from each seed point, the tracking module detects the nerve direction and then moves along a nerve by drawing successive segments perpendicular to the nerve direction (cross sections). Pixels on each cross section are analyzed, with a Fuzzy C-Mean clustering technique ¹⁵ applied to their gray-level intensities, to classify them into “nerve” and “background” pixels. Consecutive nerve pixels are used to detect the nerve part of the cross section (nerve profile), whose length represents the nerve caliber. The direction connecting the midpoints of two consecutive nerve profiles is used to estimate the new nerve direction. It allows the tracking procedure to proceed along the nerve, until a termination situation is met (e.g., when the end of the nerve fiber in the image is reached, or when a section of the nerve has such a low contrast with respect to background that it cannot be further recognized). 

As our nerve tracing strategy starts to track nerves from quite a few seed points all over the image, it may result in the splitting of a single nerve into two or more segments, because of the presence of termination situations where the tracking algorithm has stopped. These segments belonging to the same nerve are likely to have end points that are close to each other and quite similar with respect to direction, width and gray-level intensity. We have thus developed a strategy that evaluates all possible pairs of segment end points, selects the candidates for connection based on their proximity, and decides whether to connect these facing end points by minimizing a weighted combination of the differences in direction, width, and gray-level intensity of the two segments under examination. 

Improving Nerve Tracing

To boost correct connections between nerve segments of the same nerve, while at the same time limiting the incorrect ones, we devised an original strategy, in which all remaining pairs of end points that are still candidate for connection are evaluated by a more specific analysis. In each pair of end points, five different arcs are drawn between the two end points, simulating possible connections (Fig. 2) . The arc composed by image pixels with the brightest average gray-level intensity is selected as the candidate connection. Two more arcs, the “carabinieri” (policemen), are then drawn, one on each side of the candidate connection, at a predefined distance, with the aim of controlling (thus the name) the candidate one. If the difference between the (average) gray-level intensity of the candidate connection and the (average) ones of the carabinieri is larger than an empirically determined threshold, then the candidate connection is accepted as a true connection; otherwise, it is rejected. 

The whole tracking technique described above was then applied also to a version of the images obtained by preprocessing them with a Gabor filter, ¹⁵ in place of the equalization preprocessing described earlier. With this algorithm, nerve visibility is remarkably enhanced and this overcame some of the situations in which the tracking procedure applied to the equalized images was not able to detect a nerve fully. However, as in some other situations, the original equalization technique still yielded better results, we decided to take advantage of both preprocessing procedures, by merging the tracking results obtained by each of them. 

Improvements were also implemented to reduce the number of false nerve recognitions. Most of these false recognitions are due to keratocytes being incorrectly identified as short segments of nerves, as they both appear as bright structures over a darker background. To identify possible keratocytes, we segmented the original image by a simple threshold binarization and the resulting blobs were then morphologically dilated and eroded. In this segmented image, white blobs may represent either keratocytes or high-luminosity segments of nerves (e.g., nerve beads). Tracked segments inside the former should be deleted, whereas the ones inside the latter should be kept, as they belong to nerves. To this end, tracked segments wholly contained inside white blobs were deleted (as they were assumed to be false nerve recognitions inside keratocytes), whereas longer segments that extend outside the white blobs were confirmed (as they were assumed to be true nerve recognitions, in high-luminosity segments). 

A prototype of the whole algorithm described herein was implemented in commercial software language (Matlab; The MathWorks, Natick, MA). 

Results

An evaluation of the proposed algorithm was retrospectively performed on the dataset containing 90 images from normal and non-normal subjects previously described (dataset 1, see the Methods section). The manual detection of nerves was performed by tracing all clearly visible nerves with a manual drawing module that we developed ad hoc in the software language. The same images were then analyzed with the proposed algorithm, to provide the automatic detection of nerves. 

Because of the curvature of cornea layers and the possibly inaccurate alignment of the instrument on the corneal apex during image acquisition, many of the images also include parts that do not belong to the subbasal layer (i.e., stroma or epithelium), as shown, for example, in the lower right part of Figure 1A . To limit the effect of this situation, the detection procedure was then repeated for each image on a user-selected ROI, which was drawn to include as accurately as possible only the portion of the image actually containing nerve structures. 

Table 1reports the statistics for the length of recognized nerve structures, detected with the manual or the automatic method on whole images or ROIs, for all 90 images and then separately for the normal and non-normal images. Dividing the detected nerve lengths by image (or ROI) areas, nerve density (in micrometers per square millimeter) were computed and are reported in Table 2 , for all 90 images and then separately for the normal and non-normal images. 

The total lengths (densities) of nerves are very different in the various images, ranging, for example, from 0 (0 μm/mm²) to 2,522 (18,177 μm/mm²) μm by a manual method on whole images. Therefore, reporting absolute values of nerve lengths (densities) or of differences between manual and automatic methods is not very informative for quantitative assessment of the performances of the system. A more significant comparison may be performed by considering the percent lengths, namely the ratio of correctly recognized nerves length as a percentage of total manually traced length of visible nerves, which is just the sensitivity of the method and the rate of false nerve length recognitions with respect to the total automatically traced length. The latter may be assumed as representative of the specificity of the method, which cannot be computed as the true-negative nerves are undefined. The results of percent lengths are reported in Table 3 , still for all 90 images and then separately for the normal and non-normal images. On average, more than 80% of the nerves actually present in an image are correctly recognized, and fewer than 7% of false recognitions are present in the nerves detected by the automatic procedure, with slightly more correct recognitions but also more false recognitions in non-normal images. When the nerve tracing procedure is applied to user-selected ROIs, results are marginally better than on whole images, especially as regards the reduction of false recognition in non-normal subjects. 

Figures 3 4 and 5show the scatterplots of manually versus automatically recognized nerve length in whole images, for all normal, and non-normal images. From this graphic representation of the results, the correlation between the automatic and manual method can be appreciated and quantitatively described by the correlation coefficient, which was 0.94 for all images and 0.95 and 0.86 for normal and non-normal images, respectively. To display the extent of agreement between the two methods, Bland-Altman plots ¹⁶ of differences versus average for each pair of manual and automatic lengths in whole images are shown in Figures 6 7 and 8for all, normal, and non-normal images, respectively; the lines indicate the average difference and the 95% limits of agreement. 

Average length differences confirm the moderate underestimation of the automatic method with respect to the manual one: −127 μm (−13%) on all images; −134 μm (−14%) and −88 μm (−8%) in normal and non-normal images, respectively. These differences are uniformly distributed over the whole range of possible lengths, with 95% confidence intervals of −455 μm ÷ 200 μm for all images, and −453 μm ÷ 184 μm, and −463 μm ÷ 287 μm for normal and non-normal images, respectively. 

Two representative examples of the results obtained by the proposed algorithm are shown in Figure 9 . 

To confirm these results on images from an independent source, not related to the development of the algorithm presented here, we applied the automatic procedure to 80 images from normal subjects (dataset 2), provided, together with the manually determined nerve lengths, by colleagues from the Mayo Clinic College of Medicine (Rochester, MN). A scatterplot and Bland-Altman plot of manually versus automatically recognized nerve lengths in whole images from this dataset are shown in Figures 10 and 11 , respectively. The correlation coefficient between the automatic and manual method for these images is 0.89. 

As regards processing time, the prototype computer program (MatLab; The MathWorks) required approximately 1 minute per image on a computer (Pentium 4 CPU; Intel, Mountain View, CA) to trace all image nerves. 

Discussion

The algorithm we propose for nerve recognition is fully automatic, requiring no user intervention. Only if the user wishes to restrict the analysis to a specific ROI is a manual selection of the ROI necessary. The advantage of working on ROIs is that slightly fewer false nerves are detected, especially in non-normal eyes, at the expenses of a negligible decrease in the percentage of true nerve detection. The overall advantage, however, is quite marginal and, moreover, using a different user-selected ROI in each image would strongly bias the nerve density values. 

A very important characteristic of the automatic method is its capability of correctly recovering the differences in nerve length between the various subjects. As shown in Figures 3 4 5 and 10 , automatic and manual length estimations in the same image were very well correlated. This ensures that, despite the moderate underestimation of the automatic method with respect to the manual one, shown in Figures 6 7 8 11 , the former can reliably differentiate between subjects characterized by different nerve lengths. 

The performances of the algorithm are affected by the overall quality of the image (e.g., related to luminosity contrast between nerves and background and image noise), and by the possible presence of information partially coming from other layers, whose cell structures (keratocytes, epithelium cells) may be erroneously recognized as segments of nerves. A careful custom setting of the instrument lamp power, an accurate alignment of the system and, to a lesser extent and with the drawbacks mentioned, the adoption of the ROI analysis can improve the performance in these respects. As regards the processing time, implementation of the algorithm with a more efficient computer language (e.g., C++) will reduce the analysis time to few seconds per image. 

In view of a clinical application of the algorithm, the possibility of allowing the user to perform some manual touch-up of the automatic results to increase the correct nerve detection may also be considered and the proper tools developed. In this way, a manual editing session (e.g., a few tens of seconds) might result in performances close to 100% of true nerve recognition. 

To the best of our knowledge, the system presented herein is the only ever proposed for the automatic detection of the corneal subbasal nerve structures. With its application, important clinical parameters such as total length of nerves in the image, nerve density, and nerve tortuosity (e.g., as evaluated as in Ref. ¹² ) could be readily derived in an easy, quantitative, and reproducible way. Work is in progress to develop additional computer programs to derive and evaluate these clinical parameters. A significant advantage in the clinical assessment of patients can thus be reasonably expected, even if extensive clinical studies, involving a large number of subjects and diseases, should be conducted to assess fully the overall clinical benefit. 

 Presented at the annual meeting of the Association for Research in Vision and Ophthalmology, Fort Lauderdale, Florida, May 2006.

 Supported in part by Nidek Technologies, Italy.

 Submitted for publication March 21, 2008; revised May 29, 2008; accepted August 28, 2008.

 Disclosure: F. Scarpa, Nidek Technologies (F); E. Grisan, None; A. Ruggeri, Nidek Technologies (C)

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Figure 1.

 

A representative image of the cornea subbasal layer obtained with the confocal microscope: original (A) and preprocessed equalized version (B).

Figure 2.

 

The two-carabinieri (policemen) procedure is used to enhance the connection between nerve segments: original image with untracked nerve section (top left); five arcs to simulate possible connections (top right); the carabinieri arcs (black), drawn at a predefined distance on both sides of the candidate connection (bottom right); and the final accepted connection (bottom left).

Table 1.

 

Lengths of Nerve Structures in the Images of Dataset 1, for Whole Images or ROIs, Detected by the Manual or Automatic Method

	Nerve Lengths	Whole Images			ROIs
				Manual Method	Automatic Method	Manual Method	Automatic Method
All subjects (n = 90)	Average	983	856	974	837
		SD	483	440	480	436
	Min	0	0	0	0
	Max	2,522	2024	2,503	1,959
Normal subjects (n = 76)	Average	963	829	956	815
		SD	504	451	501	444
	Min	0	0	0	0
	Max	2,522	2,024	2,503	1,959
Non-normal subjects (n = 14)	Average	1,092	1,004	1,071	959
		SD	345	352	346	381
		Min	501	539	491	421
	Max	1,584	1,532	1,569	1,523

 All data are in micrometers.

Table 2.

 

Density of Nerve Structures in the Images of Dataset 1, for Whole Images or ROIs, Detected by the Manual or Automatic Method

	Nerve Densities	Whole Images			ROIs
				Manual Method	Automatic Method	Manual Method	Automatic Method
All subjects (n = 90)	Average	7,088	6,171	9,033	7,772
		SD	3,483	3,172	3,153	2,961
	Min	0	0	0	0
	Max	18,177	14,593	20,859	16,001
Normal subjects (n = 76)	Average	6,943	5,973	8,741	7,463
		SD	3,631	3,251	3,188	2,935
	Min	0	0	0	0
	Max	18,177	14,593	20,859	16,001
Non-normal subjects (n = 14)	Average	7,875	7,240	10,621	9,445
		SD	2,491	2,539	2,490	2,605
		Min	3,609	3,887	6,017	3,968
	Max	11,416	11,044	15,064	12,285

 All data are in micrometers per square millimeter.

Table 3.

 

Percentage of Correctly and Falsely Detected Lengths of Nerve Structures

	%	Whole Images			ROIs
				Tracked Nerves	False Tracking	Tracked Nerves	False Tracking
All subjects (n = 90)	Average	81.0	6.9	80.4	6.3
		SD	14.4	8.0	14.3	6.2
	Min	36.0	0.0	36.0	0.0
	Max	100.0	44.0	100.0	26.0
Normal subjects (n = 76)	Average	80.4	6.5	80.2	6.0
		SD	14.6	7.2	14.3	6.1
	Min	36.0	0.0	36.0	0.0
	Max	100.0	32.0	100.0	26.0
Non-normal subjects (n = 14)	Average	83.8	9.1	81.8	7.9
		SD	14.0	11.8	14.7	6.6
	Min	51.0	1.0	48.0	2.0
	Max	100.0	44.0	99.0	25.0

 Data are percentages of correctly detected structures, as a percentage of all manually detected nerves, and of falsely detected structures, as a percentage of total automatically detected nerves, in the images of dataset 1, for whole images or ROIs. All data are expressed as percentages.

Figure 3.

 

Scatterplot of nerve lengths from the manual versus the automatic method on whole images of all subjects (normal and non-normal, n = 90) from dataset 1.

Figure 4.

 

Scatterplot of nerve lengths from the manual versus the automatic method on whole images of normal subjects (n = 76) from dataset 1.

Figure 5.

 

Scatterplot of nerve lengths from from the manual versus the automatic method on whole images of non-normal subjects (n = 14) from dataset 1.

Figure 6.

 

Bland-Altman plot for nerve lengths from the manual and automatic methods on whole images of all subjects (normal and non-normal, n = 90) from dataset 1. It displays the difference versus average for each pair of manual and automatic lengths; dotted line: mean difference; dashed lines: 95% limits of agreement.

Figure 7.

 

Bland-Altman plot for nerve lengths from the manual and automatic methods on whole images of normal subjects (n = 76) from dataset 1. The configuration of the plot is as described in Figure 6 .

Figure 8.

 

Bland-Altman plot for nerve lengths from the manual and automatic methods on whole images of non-normal subjects (n = 14) from dataset 1 The configuration of the plot is as described in Figure 6 .

Figure 9.

 

Two representative results of the nerve tracing technique (original image in the top left thumbnail).

Figure 10.

 

Scatterplot of nerve lengths from the manual versus the automatic method on whole images of subjects from dataset 2, all normal (n = 80).

Figure 11.

 

Bland-Altman plot for nerve lengths from the manual and automatic methods on whole images of subjects from dataset 2, all normal (n = 80). The configuration of the plot is as described in Figure 6 .

The authors thank Renato Biagi (Nidek Technologies Srl, Padova, Italy), for kindly providing the images for dataset 1; Jay W. McLaren, Jay C. Erie, and William M. Bourne (Mayo Clinic College of Medicine, Rochester, MN) for kindly providing the images and the manually determined nerve lengths for dataset 2; and Jan Schroeter (Charité Campus Virchow-Klinikum, Clinic of Ophthalmology, Berlin, Germany) for assistance with image evaluation in dataset 1.