Comparison of Objective Retinal Thickness Analysis and Subjective Stereo Fundus Photography in Diabetic Macular Edema

Abstract

purpose. To compare assessment of retinal thickening by stereo fundus photographs and by Retinal Thickness Analysis (RTA).

methods. Forty-degree stereo fundus photographs of the macular field, and RTA scans in 99 eyes of 53 diabetic patients were compared in both location and area of retinal thickening. The deviation in retinal thickness on RTA from the mean retinal thickness ±2 SD in healthy controls was mapped in a spreadsheet and printed, assuming the same size as the grid with the subjectively assessed retinal thickening. Location was evaluated as retinal thickening being present or absent in nine subfields. Area was one of four categories: No retinal thickening, <1 disc area (DA), <2 DA, or <3 DA. Statistics: κ-statistic.

results. Exact agreement on location was found in 632 of 891 observations (70.9%). κ = 0.17 (95% CI: 0.08–0.25). Pooling observations for subfields 2 to 5 (inner circle), and 6 to 9 (outer circle) showed exact agreement = 75.8% and κ = 0.10 (95% CI: −0.06–0.25) and exact agreement = 65.2% and κ = 0.24 (95% CI: 0.13–0.34), respectively. Twelve eyes, in which both methods assessed the same amount of retinal thickening, showed no agreement on location, and were compared only on location of retinal thickening. Exact agreement on area was found in 44 of 87 (50.6%) eyes. Weighted Kappa = 0.34 (95% CI: 0.23–0.45).

conclusions. The degree of agreement between subjectively (with stereo fundus photographs) and objectively (with RTA) assessed location and area of retinal thickening was poor, and fair, respectively.

Diabetic retinopathy accounts for a large part of visual impairment in the western world. Among patients with type 2 diabetes, macular edema is the most common cause of visual impairment and legal blindness. Macular edema is diagnosed by binocular view of the fundus performed by slit lamp biomicroscopy, indirect fundoscopy, or stereo fundus photography, and stereoptic vision is necessary for all three assessments. Clinical studies of diabetic macular edema require fundus photographs in stereo pairs, because this method is a generally accepted “gold standard” for the evaluation of macular edema, and has proven its prognostic value empirically. ^{1 2} However, obtaining stereo fundus photographs is fairly laborious and it takes skilled personnel for both photography and assessment of the stereo fundus photographs. In addition, patients generally find this procedure fairly uncomfortable. Objective techniques, such as Retinal Thickness Analysis (RTA), Optical Coherence Tomography (OCT), and scanning laser tomography decrease clinical work load, and minimize patient inconvenience; however these techniques are still in the process of being validated. RTA has previously proven useful in detecting various pathologies in the retina ³ and in quantitating diabetic macular edema. ^{4 5 6} Generally, RTA scans are sensitive enough to detect foveal involvement of diabetic macular edema. However, the subtle changes in macular thickness seen in cases of nonclinically significant diabetic macular edema are less likely to be detected by RTA. Nonclinically significant diabetic macular edema has been assessed with both OCT and stereo fundus photography, and good agreement is found between these two methods. ⁷ The scanning laser tomograph has been used for volumetric measurements ^{8 9} as well as for Z-signal profile width ^{10 11} assessments as investigational and diagnostic tools in diabetic macular edema. RTA has not previously been compared to stereo fundus photographs. The objective of this study was to compare subjective evaluation of retinal thickening in stereo fundus photographs to objective assessment of retinal thickening by RTA in diabetic patients with macular edema, and to evaluate the degree of agreement between the two methods. 

Methods

One hundred twelve eyes of 56 diabetic patients were studied. All patients were diagnosed as having diabetic macular edema less severe than clinically significant macular edema (CSME) ³ or as having nontreatable CSME in one or both eyes by slit lamp biomicroscopy (before data collection). Thirteen eyes were excluded for the following reasons: epiretinal fibrosis (n = 1), missing RTA scans (n = 2), poor photograph quality (no stereo effect) in Early Treatment Diabetic Retinopathy Study (ETDRS) standard field 2 (n = 4), poor photograph clarity due to lens opacification and poor quality of the RTA scans due to cataract (n = 2), unresolved retinal structures in RTA scans (n = 2), and corneal dystrophy (n = 2). Thus, a total of 99 eyes (48 right eyes and 51 left eyes) in 53 patients underwent analysis. Patient characteristics were as follows: mean age, 53.3 years (range, 24.5–72.5 years); female/male ratio, 11/42; type I/II DM ratio, 11/42; mean duration of diabetes (type I), 22.1 years (range, 14–32.3 years); type II, 13.5 years (range, 2–26.7). 

Two eyes were diagnosed on fundus examination with CSME meeting the ETDRS criteria of retinal thickening ≤ 500 μm from the center of the fovea. ² Two eyes had previously had focal laser photocoagulation for CSME, and both eyes were clinically assessed as still having retinal thickening. The remaining eyes were diagnosed on fundus examination with nonclinically significant macular edema (defined as retinal thickening present at or within 2 disc diameters (DD) from the fovea, but not meeting the criteria of CSME, n = 85), or with no definite retinal thickening (n = 10). 

The control group consisted of 51 eyes (27 right eyes and 24 left eyes) of 30 healthy controls; mean age, 48.4 years (range, 21–75 years); female/male ratio, 15/15. 

Stereo Fundus Photography

ETDRS 7-standard 40° field stereo fundus photographs were obtained in all patients with a Canon CF – 60 UVi fundus camera by a photographer certified for stereo fundus photography with this camera by the Fundus Photography Reading Center, University of Wisconsin, Madison, WI. Photographs in which sufficient stereo effect, clarity, and correct positioning had not been obtained (because of poor patient cooperation, excessive tearing, poorly dilated pupils, or unclear media) led to exclusion. 

The stereo color slides were mounted in transparent slide holders in stereo pairs, and retinal thickening was evaluated in ETDRS-standard field 2 (centered on the macula) of each photograph set. Retinal thickening was evaluated with a stereo viewer and assessed with respect to area and location of retinal thickening by the author (CS). These assessments were drawn on transparent maps, one map for each eye. For improving the assessment of the retinal thickening in the fundus photos, a detailed grid 1 , calibrated for 40° fundus photographs was used (kindly provided by the Fundus Photography Reading Center). 

Retinal Thickness Analysis

The principle of RTA has previously been described. ^{5 12} In short, RTA assesses retinal thickness noninvasively on dilated pupils. The RTA map depicts the retinal thickness with a depth resolution of 50 μm. ¹² For each fixation, ten vertical 2-mm slit photographs of the retina are taken. The front and back surfaces of the retina are found with a mathematical algorithm and retinal thickness is calculated for each 200 μm of the slit giving 100 values for each fixation in 0.2 seconds. Nine quadrants of 2 × 2 mm cover an area of 36 mm² of the posterior pole. In this study the Retinal Thickness Analyzer, version 2.13, analysis version 6.39 (Talia Technology Ltd., Mevaseret Zion, Israel) was used. In each eye a standard session of nine scans of 100 measuring points was obtained, yielding a total set of 900 measuring points in microns. The raw data set was extracted and each study eye was compared to the mean retinal thickness ± 2 SD of healthy controls. To ensure that anatomic differences in retinal thickness in the control eyes (e.g., nasal versus temporal, right versus left) would not interfere with the comparison to study eyes, right and left eyes were treated separately. Deviations in a study eye from the mean retinal thickness of healthy controls ± 2 SD was noted in a spreadsheet in exact numbers. The spreadsheet was printed, and areas of retinal thickening with positive values (i.e., measuring points exceeding the mean retinal thickness +2 SD in healthy controls) were defined as definitely thickened. This RTA deviation map, which depicted the deviation in retinal thickness in that particular study eye from the healthy control map, was subsequently compared to the transparent subjectively evaluated stereo fundus photography map, with the spreadsheet printed, matching the detailed grid exactly in size 1 . 

Comparison of Location of Retinal Thickening

The transparent subjectively assessed retinal thickening maps were compared to the RTA deviation map, and the area or areas definitely thickened on RTA were added to the transparent map. For the purpose of analyzing the degree of agreement on location of retinal thickening, the fundus was divided into 9 subfields as shown in 2 and as previously described by several authors. ^{13 14 15 16}  

Comparison of Area of Retinal Thickening

The degree of retinal thickening was arbitrarily evaluated as being one of four categories: no retinal thickening, <1 DA, <2 DA, and <3 DA. The area of retinal thickening was based on a sum of all areas of retinal thickening in one eye, and the assumption that 1 DD was equal to 1500 μm. No eyes had >3 DA of retinal thickening on stereo fundus photography or RTA. The RTA deviation maps and subjective evaluation of stereo fundus photographs were assessed by the same person with a 6-month interval between the two assessments. Subjective assessment of retinal thickening on fundus photographs took place before assessment of RTA deviation maps. 

Reproducibility of the RTA in one subfield of repeated scans showed a coefficient of variance of 9.3%, approximately 15–20 μm, which was within the same range as findings of other authors. ^{4 17 18} The highest accuracy of the assessment is obtained after correction for the refractive power of the eye, compared to the axial length. This analysis feature was not available in the current analysis version of the RTA, and as all eyes included were within ± 4.0 diopters, the effect of the correction of optical power and axial length would be of marginally importance. An updated analysis version of the RTA has become commercially available, since the analysis of these data. Test samples run with the new software version 3.2, analysis version 7.2, on the present study material and control group showed quantitatively differences between the two analysis versions from 5 to 10 μm. Qualitatively, no differences were found. 

Comparison of Objective and Subjective Methods

Comparison between an objective and a subjective method presents some problems. A major advantage of the RTA is the systematic procedure—covering the posterior pole—with equal distances between the measuring points, yielding a large number of calculated values. In this study, deviation of retinal thickness from healthy controls was used to assess areas with definite retinal thickening, rather than assessing the exact retinal thickness in the study eyes, and hereby allowing for a comparison to stereo fundus photography. Thus the numerical data of the exact retinal thickness was converted into categorical data, that is, exceeding the retinal thickness of healthy controls +2 SD or not. This compromise was taken, since assessment of the exact thickness is not possible in stereo fundus photographs. The standard deviations for each of the 900 values obtained for all nine fixations ranged from 12 μm to 45 μm with a mean of 24.7 μm in both right and left eyes. In the fovea, the standard deviations ranged from 20 μm to 30 μm. 3 shows the mean normal retinal thickness values of all left eyes of healthy controls. 

Informed Consent

The study followed the tenets of the Declaration of Helsinki. Informed consent was obtained from the subjects after explanation of the nature and possible consequences of the study. The research was approved by the local ethics committee. 

Statistics

The Kappa statistic (κ) is interpreted as the chance corrected proportional agreement between two groups. The weighted kappa (κ_(w)) is the weighted proportional agreement corrected for chance. ¹⁹ The usage of the κ_(w) is preferable to κ for the comparison of area of retinal thickening, because the degree of disagreement is taken into account. Thus in this data set, the categories are ordered and disagreements are given weights according to the size of the discrepancy. Weights applied were full for exact agreement, two-thirds for one-step disagreement, one-third for two-step disagreement, and no credit for three-step disagreement. However, the comparison of location of retinal thickening yielded only two categories. Thus, weighted kappa analysis was not required. Additionally, McNemar’s test was performed as a test for bias in symmetry of the disagreements. The Kappa statistic has a maximum of 1.00 when agreement is perfect, and κ = 0 indicates no agreement better than chance. Only guidelines to interpretation can be given, and those given by Landis and Koch ²⁰ suggest that κ = 0.61–0.80 indicates that agreement between groups is “substantial”, and κ = 0.81–1.00 indicates that agreement is “almost perfect.” Although the degree of acceptable agreement depends on circumstances, for most purposes κ or κ_(w) <0.5 will indicate poor agreement. 

Results

Comparison of Location of Retinal Thickening

A total number of 891 (nine per eye in 99 eyes) observations were made. Overall exact agreement on location of retinal thickening was found in 632 (70.9%) observations, κ = 0.17 (95% CI: 0.08–0.25). By subfield the exact agreement ranged from 60.6% to 78.8%, and kappa values ranged from 0 to 0.26. McNemar’s test for symmetry of the data showed that two subfields (2 and 7) had significant bias in the symmetry of the observations of discrepancies in favor of retinal thickening being present in stereo fundus photographs and absent on RTA. Four subfields (1, 4, 5, and 6) were also biased in favor of stereo fundus photographs on discrepancies, yet not significantly. One subfield (9) showed no disagreement on discrepancies, and in two subfields (3 and 8) the discrepancies were biased in favor of RTA, more often showing retinal thickening compared to stereo fundus photos, but not significantly. The results are shown in 1 . 

When pooling the observations for subfields 2 to 5 (inner circle), and 6 to 9 (outer circle), the agreement on location of retinal thickening remained low: inner circle: exact agreement = 75.8% and κ = 0.10 (95% CI: −0.06–0.25) and outer circle: exact agreement = 65.2% and κ = 0.24 (95% CI: 0.13–0.34), respectively 2 . Based on the fact that the majority of eyes have nonclinically significant macular edema, and no retinal thickening in the center of the fovea, and thus would skew a pooling analysis, subfield 1 was omitted. 

Twelve eyes with agreement on area but not on location were omitted because these data would improve the result on a false basis. On the comparison of area, these eyes would enter as “false positives”—hence they were analyzed with respect to location only. 

Comparison of Area of Retinal Thickening

Exact agreement on area of retinal thickening was found in 44 of 87 (50.6%) eyes, κ_(w)= 0.34 (95% CI: 0.23–0.45). The result of the assessment is shown in 2 . Two eyes with CSME showed retinal thickening with central (i.e., foveal) involvement on fundus photographs. Foveal thickening was detected by RTA in only one of these two eyes with CSME. Six eyes, which had no detectable retinal thickening as determined by stereo fundus photographs, showed no retinal thickening by RTA either. Additionally, 18 eyes showed definite retinal thickening on subjective evaluation of stereo fundus photography, yet no retinal thickening was detected on RTA. A scatter plot on area of retinal thickening is shown in 4 and the Spearman coefficient of correlation (r _s) was 0.46. 

Discussion

Objective measurement of retinal thickness is of importance in diabetic patients and could serve as a valuable screening tool, especially as an extension of automated image analysis of fundus photographs. In the present study, the Retinal Thickness Analyzer was compared to fundus photographs in patients with subtle degrees of diabetic macular edema. 

Exact agreement of location of retinal thickening was 70.9% and κ was 0.17. On area of retinal thickening, exact agreement was 50.6% and weighted κ was 0.34. Thus poor agreement was found between the two methods, and in 17 eyes no edema was found with RTA in contrast to the fundus photographs. The lack of sensitivity could be due to the control group, as the standard variation in healthy subjects is approximately 15%–20% of the mean retinal thickness. Part of this could be due to excessive scatter, particularly in elderly subjects where some degree of nuclear sclerosis will always be present even if the media is considered clear clinically. In the present study the variation in the most peripheral part of the slits was up to 44 μm. A considerable interpersonal variation has also been found by Landau et al. ²¹ and Neubauer et al. ²² Pulito and co-workers ²³ report a 34% waste of study eyes due to light scatter. On the other hand, the evaluation of fundus photographs may easily be biased by the presence or absence of hard exudates, and in some cases be overinterpreted. Additionally, in cases where the edema is already resolved, hard exudates may stay for many weeks before dissolution. 

Is RTA More or Less Sensitive in Detecting Retinal Thickening than Stereo Fundus Photos?

As indicated in 1 , McNemar’s test was performed to assess significant bias to one side. Only two subfields (2 and 7) showed significant bias. In subfield 2, it was in favor of fundus photographs showing retinal thickening more often than RTA, and in subfield 7, the case was the reverse. The table of frequencies on the comparison of area of retinal thickening 3 shows that in 23 eyes stereo fundus photographs revealed more retinal thickening than RTA, and in 20 eyes RTA showed more retinal thickening than what was found by subjective evaluation of the same eyes. This difference of three eyes between the two methods is hardly evidence of one method being more sensitive than the other. However, 17 eyes, in which fairly small focal edemas (<1 DA in size) were found by stereo fundus photos, no retinal thickening was found by RTA. Pires and co-workers ²⁴ report RTA to be a more sensitive tool in detecting localized increases in retinal thickness. The work of Pires and co-workers comprise 28 eyes where 36% had no clinically visible retinopathy; the increases in thickness measured with the RTA had a very high variance. With eyes suggested to have an increased retinal thickness as high as 73.5% in the fovea, it seems unlikely that these eyes with an increased retinal thickness, assessed with RTA, can maintain visual acuity of 20/20 as suggested by Pires et al. ²⁴  

A comparison was made between subjective evaluation by stereo fundus photographs and objective assessment by RTA. Having worked with both OCT and RTA, however, this study and previous work on comparison of stereo fundus photographs and OCT ⁷ suggest that OCT showed a better agreement with subjectively assessed retinal thickening by stereo fundus photographs on both location and area of retinal thickening than did RTA. 

A potential source of error in RTA may be the presence of hard exudates, which tends to induce reflexes in the RTA slit, making it impossible for the algorithm to resolve the width of the slit. Yoshida ²⁵ also reports that backscattering from hard exudates induces obscurity to the vitreoretinal interface in the RTA. Additionally, the algorithm will skip many hard exudates as “unresolved points.” A possible remedy to help solve this problem could be to analyze the slope on the intensity curve instead of making the algorithm trying to fit a peak of intensity profile of the retinal pigment epithelium that is not well defined. Scanning laser tomography employs a technique similar to that of RTA in using the reflectance profile of the retina which is postprocessed with a mathematical algorithm defining the anterior and posterior retinal border from predefined rules for the location of the specific surface in relation to the slope of the reflectance profile. However the two instruments have different optical systems, and scatter is not reported as a major problem with this scanning laser tomograph using a confocal aperture ^{8 9 10 11} ; therefore changing the optics in RTA may also avoid the scatter problem. 

Undersampling may also be a problem during analysis of RTA data. The information in one slit of 2 mm is smoothed by the software algorithm and given as only ten measuring points. Thus the true values of thickness might be undersampled by the smoothing mechanism in the algorithm. 

In this study only whites participated (controls and study patients), and often whites have a low content of pigment in the pigment epithelium. This can cause a low signal from the RPE and reduce the ability of the software algorithm to detect the posterior signal, which is the edge of the RPE, and thus the registration of an inaccurate width of the slit may occur. 

Lens opacities can potentially induce light scatter to the RTA slits and decrease the clarity of the fundus photographs. In RTA, nuclear and posterior subcapsular cataract seem to be of minor importance, whereas cortical cataracts induce light scattering, resulting in inaccurate assessment of the width of the slit. In this study, eyes with clinically significant lens opacities were excluded, due to both poor photograph quality and poor quality of the RTA scans. Insufficient lubrication of the cornea or corneal dystrophy also induce light scatter to the slit, resulting in a falsely high value of retinal thickness. One patient had mild corneal dystrophy in both eyes, and RTA in both eyes showed extremely high values of retinal thickness and an excessive amount of unresolved measuring points. Both eyes of this patient were excluded. 

In the present study it is possible that focal edema assuming a fairly small size in extent was not detected by RTA, due to the addition of 2 standard deviations to the mean value of healthy controls. However this leaves little explanation of the 20 eyes in which the RTA detected definite retinal thickening to a greater extent than what was assessed in the same eyes by subjective evaluation of stereo fundus photographs. It is a fair claim to make on an objective tool that it can indeed detect retinal thickening where it supposedly is present and detected by the means that are currently considered “gold standard” (fundus photos), irrespective of the extent of retinal thickening. It has even been claimed by Pires and co-workers ²⁴ that RTA can detect retinal thickening in eyes with no visible retinopathy (ETDRS retinopathy level 10) or mild retinopathy (level 20 or 35). It is impossible for the human eye to detect changes of <100 μm on stereo fundus photos ⁵ or in binocular slit-lamp biomicroscopy. ²⁶ Thus the subtle focal pathology seen in nonclinically significant macular edema must presumably assume a size (i.e., height) greater than 100 μm in order for the observer to become aware of its presence, and it seems to be a fair claim of an objective tool to be able to detect retinal thickening to a similar degree to that of the human eye. 

RTA is a noninvasive, patient- and operator-friendly technique, which has the advantage of quantitating retinal thickness in a standardized manner. It has been shown reproducible by several authors. ^{6 12} The results of this study showed that RTA assessment of retinal thickening corresponded poorly on location of retinal thickening to the subjective assessment of retinal thickening by grading of stereo fundus photos, and fair on area of retinal thickening. 

 Submitted for publication July 23, 2003; revised January 6, 2004; accepted January 13, 2004.

 Disclosure: C. Strøm, None; B. Sander, None

 The publication costs of this article were defrayed in part by page charge payment. This article must therefore be marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

Figure 1.

 

RTA deviation from controls, and subjective assessment. Red numbers indicate values of less than mean retinal thickness −2 SD; black numbers indicate values above +2 SD in healthy controls. The hatched area on the grid indicates the area definitely thickened, assessed subjectively. This was interpreted as retinal thickening being present in subfields 2, 6, and 9 (for the location analysis) and as <2 DA in area of retinal thickening.

Figure 2.

 

Schematic drawing of the fundus map used in right eyes. The map is divided into 9 subfields for comparison on placement of retinal thickening. The boundaries of the circles are: inner, 1000 μm; middle, 3000 μm; outer, 6000 μm. The map was flipped for use in left eyes.

Figure 3.

 

Mean retinal thickness in microns in left eyes of healthy controls. The inner circle indicates a radius from fovea of 1 disc diameter (DD), and the outer circle indicates a radius from fovea of 2 DD. The SD ranges for each grid subfield (according to the map shown in 2 ) are: subfield one, 20–25 μm; subfield two, 16–26 μm; subfield three, 23–34 μm; subfield four, 22–37 μm; subfield five, 23–32 μm; subfield six, 13–26 μm; subfield seven, 16–32 μm; subfield eight, 17–29 μm; subfield nine, 26–45 μm.

Table 1.

 

 

Frequencies of Location of Retinal Thickening

Table 1.

 

Frequencies of Location of Retinal Thickening

	RTA	Fundus Photos			McNemar’s Test
				+ RT	− RT	z	P
Field 1	+ RT	2	12	−0.2	0.84
	− RT	13	72
Field 2	+ RT	2	7	−2.2	0.028*
	− RT	18	72
Field 3	+ RT	8	13	−0.41	0.68
	− RT	11	67
Field 4	+ RT	3	8	−1.09	0.28
	− RT	13	75
Field 5	+ RT	2	10	−1.18	0.24
	− RT	16	71
Field 6	+ RT	26	16	0.67	0.50
	− RT	20	37
Field 7	+ RT	21	26	−2.08	0.04*
	− RT	13	39
Field 8	+ RT	10	15	0.58	0.56
	− RT	12	62
Field 9	+ RT	20	18	0.0	1.0
	− RT	18	43

* Significant; n = 99; Exact agreement: 70.9%; κ = 0.17; 95% CI: 0.08–0.25.

Table 2.

 

 

Frequencies on Pooled Location of Retinal Thickening (Inner and Outer Circle)

Table 2.

 

Frequencies on Pooled Location of Retinal Thickening (Inner and Outer Circle)

Pooling of Subfields	RTA	Photo Grading			Exact Agreement	Kappa (95% CI)
				+ RT			− RT
Subfields 2–5	+ RT	15	38
	− RT	58	285	75.8%	0.10 (−0.06–0.25)
Subfields 6–9	+ RT	77	75
	− RT	63	181	65.2%	0.24 (0.13–0.34)

Figure 4.

 

Scatter plot of area of retinal thickening divided into four categories: No retinal thickening (RT); <1 disc area (DA); <2 DA; and <3 DA. Spearman coefficient of correlation (r_s) is 0.46.

Table 3.

 

 

Frequencies on Area of Retinal Thickening

Table 3.

 

Frequencies on Area of Retinal Thickening

RTA	Fundus Photos							Total
		No RT	<1 DA	<2 DA	<3 DA
No RT	6	17	1	0	24
< 1 DA	2	28	3	1	35
< 2 DA	1	9	9	1	19
< 3 DA	1	3	4	1	9
Total	10	58	16	3	87

 n = 87; Exact agreement: 50.6%; κ_(w) = 0.34; 95% CI: 0.23–0.45; H(0) test: z = 9.8, P < 0.0001.