Removal of Reflections in the Photographic Assessment of PCO by Fusion of Digital Retroillumination Images

Abstract

purpose. Automated image-analysis systems for objective assessment of posterior capsule opacification (PCO) depend on good image quality. One major drawback is the existence of light-reflection artifacts (Purkinje spots) in retroillumination images of the posterior capsule. Therefore, a software algorithm was developed that removes these artifacts by fusion of two or more digital images from the same eye, photographed in slightly different directions of gaze.

methods. The image-fusion process comprises five steps: definition of a primary and a secondary image, automated segmentation of the region of interest and the Purkinje spots, manual selection of three pairs of corresponding points in both images, geometric registration and radiometric calibration of the regions to be inserted from the secondary image into the primary image. The program was tested with an image set of 30 eyes that had various degrees of PCO. A digital image acquisition system with a coaxial optical path was used to take retroillumination images from each eye in at least three different directions of gaze.

results. In 28 cases all light-reflection artifacts within the capsulorrhexis rim could be removed entirely. In two cases, small parts of single Purkinje spots remained visible, because the reflections were located too closely in the primary and the secondary images.

conclusions. Fusion of digital retroillumination images provides high-quality, reflection-free PCO images. This allows objective PCO assessment systems to analyze 100% of the posterior capsule, leading to more accurate results.

After-cataract or posterior capsule opacification (PCO) is still the most common long-term complication of modern cataract surgery. Many efforts have been made to investigate the genesis of PCO and to prevent its formation, including modifications in intraocular lens (IOL) design and material, modifications in surgical technique, and application of drugs. Most of these clinical trials use subjective scoring techniques or capsulotomy rates as outcome measures. ^{1 2} The former are often examiner biased and time consuming, the latter are dependent on patients’ demands and financial factors. Therefore, the validity of numerous trials is questionable, and comparison of results of different trials is difficult. 

To compare different interventions for preventing after-cataract, an objective and standardized PCO assessment method is required. This is especially important in the case of PCO, because masking of examiners is not possible, in that the IOL type is almost always detectable. However, to perform studies that are compatible with the criteria of evidence-based medicine, a masked evaluation of the main outcome variable—intensity of PCO—is mandatory. Therefore, automated and objective image-analysis systems have been developed. A preliminary version of one such system (automated quantification of after-cataract [AQUA]) has been introduced by our group in cooperation with the Institute of Electrical Measurement and Measurement Signal Processing at the Technical University of Graz. ³ This system is to be used for prospective, randomized, masked clinical trials assessing the effectiveness of surgical techniques, drugs, IOL design, and IOL material on preventing PCO. A similar image analysis system was developed by Barman et al. ⁴  

Standardized retroillumination photography is required for objective analysis of PCO with these evaluation methods. Different photographic techniques and assessment systems have been described by several investigators. ^{5 6 7 8 9 10 11 12} Conventional slit lamp photographs (also in retroillumination) have disadvantages. They show uneven illumination with loss of detail in the relatively darker area and have a large light-reflection artifact caused by the slit. Therefore, Pande et al. ¹³ have designed a retroillumination system with a coaxial optical path that leads to homogeneous illumination over the entire region of interest and causes much smaller light reflections. This system was adopted by us with slight modifications (digital coaxial retroillumination photography [DCRP]). ¹⁴ Still, the method produced disturbing light-reflection artifacts (Fig. 1) . Unfortunately, these artifacts (Purkinje spots) are mainly located in the central 3-mm zone of the IOL, which is of particular interest in clinical trials on PCO, because opacities in this area predominantly affect visual function. 

The purpose of this study was the development of a software algorithm for complete removal of light-reflection artifacts from digital retroillumination images of the posterior lens capsule. 

Methods

The concept behind the software solution for removal of light-reflection artifacts from digital retroillumination images is to eliminate the reflections by taking a series of images of the same eye, but in slightly different directions of gaze, and fusing (digital combination) two or more of these images. This is possible because the Purkinje spots change their positions with the various directions of gaze and cover different parts of the posterior capsule in different images of one series. Thus, a region covered by artifacts in one image can be replaced by the corresponding reflection-free area in another image of the same eye. 

However, several problems had to be solved in the development of an effective algorithm. First, not all visible structures reside in the same plane (especially parts of the anterior capsule). Besides, the image parts from one image that should replace the Purkinje spots in another image must be spatially adapted and scaled before they can be inserted, because both images were photographed in different directions of gaze and were, therefore, differently distorted. This correction step in the algorithm requires detection and definition of three pairs of exactly corresponding points that can be found in both images—for example, small Elschnig pearls. Another difficulty is radiometric adaptation. Because radiometry (color and brightness) is always slightly dissimilar in different images, especially when photographed in different directions of gaze, it must also be adjusted to obtain a good visual impression and to avoid additional artifacts by the fusion process itself (such as sudden changes in color and/or brightness between the original bit map and the inserted parts). This problem can be solved by computing average gray levels for small regions at the border of the inserted image parts and interpolating the change in gray level. A minor problem is the correct segmentation of the region of interest and of the Purkinje reflections. 

The fusion software was primarily developed for use with our digital image-acquisition system, which uses coaxial illumination for high-quality retroillumination images with relatively small light-reflection artifacts. ¹⁴ However, we have shown that it can be used with conventional slit-lamp images, as well. To record our image sets (three or more images per eye in different directions of gaze) in a standardized manner, we added a ring with three red fixation LEDs in front of our photographic setup. Three images are taken with the patient looking straight ahead, upward, and temporal (the latter two by fixating the corresponding LED). The LEDs can be switched on and off by the examiner. The active LED turns off automatically when acquiring the image, to avoid a reflection artifact by the red light. In some cases we used an additional variable fixation light in front of the contralateral eye (especially when the patient was unable to see the small LEDs on the ring). 

The current version of the image-fusion software is based on algorithms in a commercial software package (MatLab; The MathWorks, Natick, MA). The underlying principle is the assumption of a rigid affine transformation, which is applied to the bit map parts that should replace the Purkinje spots in the original image. After realizing that a fully automated approach to image fusion causes several serious problems, especially in finding correct point correspondences in the image pairs (due to variation of focus, illumination, and different directions of gaze that affect the apparent overlap of the capsulorrhexis and the posterior capsule), we used a semiautomated approach in this study. 

Steps in the Complete Fusion Process

Definition of a Primary and a Secondary Image.

First, the operator has to select a primary and a secondary image. The primary image contains the regions that should be replaced (the areas within and around the Purkinje spots), and the secondary image contains the corresponding regions that should be inserted into the primary image. Usually, the primary image is the image taken in primary gaze. The selection of the best image depends on contrast and focus and especially on the size and position of the Purkinje reflections. If one or more reflections are located in the same position in both images, it is not possible to remove all artifacts entirely. Consequently, image pairs with entirely different locations of the reflection artifacts are ideal for fusion. 

Segmentation of the Region of Interest and of the Purkinje Spots.

The approximate size and position of the region of interest (the area inside the pupillary aperture) are automatically detected by thresholding, morphologic operations, and intermediate symbolic representation (ISR) filtering (criterion, largest region). The areas that should be replaced (the Purkinje reflections) are segmented by thresholding as well and enlarged by morphology. 

Selection of Corresponding Points.

For the registration process, three pairs of corresponding points defining the subsequent affine transformation have to be selected manually, with the computer mouse in both the primary and the secondary images. For this purpose, parts of the image can be magnified by the operator, which makes it easier to find the same pattern in both images (e.g., a small Elschnig pearl). In this step, it is important to find exactly matching points in both images to attain good fusion results. 

Geometric Registration.

Registration in this context means the process of shifting, rotating, scaling, and distorting a bit map to get exactly matching image parts for fusion. The parameters (A to F) defining the affine transformation for the registration process (the adaptation of parts of the secondary image that should fit into the primary image) are calculated from the point correspondences by means of a least-square error (LSE) algorithm. The pixels of the regions containing light-reflection artifacts in the primary image (X _M , Y _M) are then replaced by pixels from the corresponding reflection-free region in the secondary image (X _S , Y _S), by using the following formula:  

\[\left(\begin{array}{l}X_{M}\\Y_{M}\end{array}\right)\ {=}\left(\begin{array}{ll}A&B\\C&D\end{array}\right)\ \left(\begin{array}{l}X_{S}\\Y_{S}\end{array}\right)\ {+}\left(\begin{array}{l}E\\F\end{array}\right)\]

 

Radiometric Calibration.

Radiometric calibration (adaptation of color and brightness of the inserted parts) is performed by statistical convergence of the area around the region of interest. The statistics of the region that should be inserted are scaled to the statistics of the corresponding region in the primary image. Only pixels at positions without artifacts in both the primary and secondary images are used for computation. In addition, a linear adaptation of color and brightness in relation to the distance of each pixel from the center of the Purkinje spot is applied to the inserted region and the original region in the primary image. 

These five steps lead to a reflection-free fusion image in most cases (provided that the reflection artifacts are differently located in the primary and the secondary image). The entire procedure takes approximately 2 minutes for one image (computation of the result lasts approximately 20 seconds on a standard computer). In some cases, additional fusion steps are necessary (images in three or more directions of gaze) for complete removal of all reflection artifacts. However, with the proper photographic technique, two images are sufficient. Figure 2 illustrates the image-fusion process. 

To test the new software, we created a set of digital retroillumination images of 30 eyes showing PCO of various degrees. The patients were not selected for this trial but were taken from a continuous cohort. The patients came to our outpatient department for follow-up treatment or for Nd:YAG laser capsulotomy and were included after giving informed consent, according to the provisions of the Declaration of Helsinki. Each eye was photographed in at least three different directions of gaze to make sure that all Purkinje artifacts could be completely removed later by image fusion. The two best images from each patient with different Purkinje artifacts were selected (usually, the first image, with the patient looking straight into the light, was chosen as the primary image) and the image-fusion software was applied to them. 

Because we wanted to find out whether the removal of the Purkinje reflections in PCO images would have a positive effect on the accuracy of PCO assessment with automated systems, we evaluated both the original and the fusion images with the AQUA software. The software calculates the entropy (the heterogeneity of texture) of a PCO image. This value is then converted to a score between 0 and 10 (AQUA score), where 0 stands for a clear posterior capsule and 10 for exceptionally severe PCO. ³ Regions with reflection artifacts are detected by thresholding and are ignored during PCO assessment. Therefore, only reflection-free images can be assessed entirely. 

We also evaluated an additional image pair (original and fusion image) with comparatively large Purkinje artifacts in the original (primary) image. Usually, we would not choose such an image for evaluation with the AQUA software. However, sometimes no better image is available. We expected the difference between the AQUA score for the original and for the reflection-free image to be higher in this case than with images showing only small Purkinje reflections. 

In each eye, the AQUA score was calculated for the central 3-mm area of the IOL only—once using the original (primary) image and once the reflection-free fusion image. The central 3-mm zone was chosen, because this is the area in which most light-reflection artifacts are located. At the same time, PCO in the central 3-mm region has the most influence on visual function, because this is the average pupil size under photopic conditions. Opacities in this area should therefore be accurately measured in clinical trials on PCO. The scores (AQUA score, 0–10) for the original images and the fusion images were compared, and the differences between the two results were calculated in each case. In the original images, the size of the reflections (percentage of pixels covered by Purkinje artifacts in the region of interest) was also measured. All statistical evaluation was performed on computer (Excel; Microsoft, Redmond, WA). 

Results

Figure 3 shows some examples from the test image set. In 28 cases, the Purkinje artifacts within the capsulorrhexis were eliminated entirely. In two cases, small parts of single reflection artifacts could not be removed entirely, because the artifacts were located too closely in all existing images of these eyes. In two other cases, an additional fusion step (a third image) was necessary to attain a reflection-free result. All other images could be made reflection free in only one step (from two images). 

Figure 4 shows the correlation between the AQUA results for the central 3-mm zone of the original images and the reflection-free images (scatterplot). The correlation coefficient was 0.99. The maximum difference was 1.1 (11%). In 29 of the 30 cases the evaluation of the original image showed a higher result than that for the fusion image, in one case there was no difference between the two results (mean difference: 0.56; confidence interval: ±0.1). The SD of all absolute differences was 0.28. The mean size of the Purkinje artifacts in the original images was 1.93% of the whole area of interest (maximum: 3.71%). 

The evaluation of the additional image pair with large reflection artifacts in the original image lead to a difference of 1.2 (12%). 

Discussion

Fusion of digital retroillumination images with the current version of the image-fusion software provided highly satisfactory results when a coaxial illumination system was used. ¹⁴ At least two images of one eye with differently located Purkinje spots, which must not cover the same location in both images (even in part) are needed. This can be easily attained by taking photographs in different directions of gaze. All disturbing reflection artifacts within the capsulorrhexis rim can be removed completely. It is then almost impossible to tell where the artifacts were located before removal. In some cases, it is possible to find the inserted region in the image; nevertheless there are no disturbing lines or sudden changes in color and/or intensity between the original image and the inserted bit map. The process of transformation and radiometric calibration leads to well fitting replacement parts which are additionally blended into the surrounding bit map to avoid new artifacts by the fusion process itself (such as lines or edges between the combined image parts). Even in cases with poor image quality of the secondary image (e.g., under- or overexposed, out of focus), the fusion process leads to surprisingly good results, if the primary image is of high quality and shows only small Purkinje spots. 

In the current version of the algorithm, three corresponding points have to be selected manually in the primary and the secondary images to define the affine transformation needed for image fusion. This may cause problems in cases with little or no visible PCO. However, we had no problems with the 30 image sets that were used for this trial and which also included cases with little PCO. Even in eyes with a clear posterior capsule there are usually some small landmarks (e.g., spots, inhomogeneities) which can be found in both the primary and secondary images. 

The evaluation of the central 3-mm area of all images with the automated AQUA software showed that the original images tended to produce a slightly higher AQUA score than the fusion images (mean: 0.56). This is most likely a result of the texture operations that were applied to the original image during the image-fusion process and which led to a somewhat smaller entropy in the resultant image. However, the correlation between the AQUA score for the original and the processed (reflection-free) images was very high (r > 0.99). The main reason may be that all reflection artifacts were relatively small with our coaxial image-acquisition system and cover only a small percentage of the central 3-mm area (mean: 1.93%). Besides, Purkinje spots were detected by the AQUA software by thresholding and were ignored during the assessment of PCO. Obviously, the use of unprocessed images did not have a negative effect on this image-analysis method, as long as there were only small light-reflection artifacts. However, when the area covered by reflection artifacts got larger, the evaluation results became less reliable. This is shown by the 12% difference between the two AQUA results for the image pair with larger Purkinje spots. With the introduction of reflection-free images, this weakness was eliminated. 

In contrast, because dedicated retroillumination photography systems are rare, most retroillumination images are taken with conventional photograph slit lamps. These images show uneven illumination, reduced contrast, and large reflection artifacts at the slit. Therefore, they cannot be assessed exactly with image-analysis systems, because large parts of the bit map have to be omitted. The image-fusion software can be useful, especially for this purpose. We are currently trying to adapt the software for use with slit lamp images. Two retroillumination images, with the slit decentered nasally and temporally, should be sufficient to get one reflection-free image with almost even illumination. Figure 5 shows an example that was generated with a slightly adapted version of the computer program (MatLab). Provided that there is a generally accepted standard for taking slit lamp images in retroillumination, objective PCO assessment methods could become more easily accessible with this method than with dedicated, expensive retroillumination image-acquisition systems. Also, the technique could be extended and adapted to images in cataract epidemiologic studies. 

In conclusion, digital image fusion is a quick and effective procedure that removes light-reflection artifacts from retroillumination images of the posterior capsule. In combination with a coaxial photographic setup it provides excellent PCO images. For the first time, these images can be entirely assessed by automated image-analysis software. There is now a sophisticated, objective, and accurate method for evaluating development of PCO after cataract surgery in clinical studies. 

 Supported by Grant FWF P14269-MED from the Austrian Fonds zur Förderung der wissenschaftlichen Forschung.

 Submitted for publication June 21, 2002; accepted August 1, 2002.

 Disclosure: O. Findl, None; W. Buehl, None; H. Siegl, None; A. Pinz, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

 

Figure 1.

 

Purkinje spots: P1, P3, and P4 denote Purkinje spots. R indicates the internal reflection caused by the optics of the system.

Figure 2.

 

Image-fusion process. Top left: primary image; top right: secondary image; bottom left: secondary image with regions to be inserted into the primary image (white); bottom right: reflection-free final image.

Figure 3.

 

Examples from the image set, showing original (left) and fusion images (right); bottom image: case with large Purkinje spots.

Figure 4.

 

Correlation between the AQUA score for the original and the reflection-free images. The regression line (solid) and the line with slope 1 (dashed) are depicted.

Figure 5.

 

Fusion of slit-lamp retroillumination images: top and middle: original slit images; bottom: fusion result.