The Effects Of Image Misregistration On The ... - Semantic Scholar

1566

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, VOL. 36, NO. 5, SEPTEMBER 1998

The Effects of Image Misregistration on the Accuracy of Remotely Sensed Change Detection Xiaolong Dai, Member, IEEE, and Siamak Khorram, Member, IEEE

Abstract— Image misregistration has become one of the significant bottlenecks for improving the accuracy of multisource data analysis, such as data fusion and change detection. In this paper, the effects of misregistration on the accuracy of remotely sensed change detection were systematically investigated and quantitatively evaluated. This simulation research focused on two interconnected components. In the first component, the statistical properties of the multispectral difference images were evaluated using semivariograms when multitemporal images were progressively misregistered against themselves and each other to investigate the band, temporal, and spatial frequency sensitivities of change detection to image misregistration. In the second component, the ellipsoidal change detection technique, based on the Mahalanobis distance of multispectral difference images, was proposed and used to progressively detect the land cover transitions at each misregistration stage for each pair of multitemporal images. The impact of misregistration on change detection was then evaluated in terms of the accuracy of change detection using the output from the ellipsoidal change detector. The experimental results using Landsat Thematic Mapper (TM) imagery are presented. It is interesting to notice that, among the seven TM bands, band 4 (near-infrared channel) is the most sensitive to misregistration when change detection is concerned. The results from false change analysis indicate a substantial degradation in the accuracy of remotely sensed change detection due to misregistration. It is shown that a registration accuracy of less than one-fifth of a pixel is required to achieve a change detection error of less than 10%. Index Terms— Accuracy assessment, change detection, false change analysis, image registration, remotely sensed.

I. INTRODUCTION

I

N THE current decade, global environmental change has reached beyond the research domain and become a major national and international policy issue. Given the current techniques available, remote sensing provides the most feasible approach to regional and larger scale land surface change detection [1]. A considerable amount of data about the nature of the earth’s surface have been collected by remote-sensing devices. The volume and rate of these data are expected to increase rapidly as more and more high-resolution images are becoming commercially available, such as the NASA’s

Manuscript received September 19, 1997; revised May 1, 1998. This work was supported in part by the Coastal Change Analysis Program (C-CAP) of the National Oceanic and Atmospheric Administration (NOAA) funded through the North Carolina Sea Grant Program. The authors are with the Center for Earth Observation (formerly the Computer Graphics Center), North Carolina State University, Raleigh, NC 27695-7106 USA (e-mail: [email protected]). Publisher Item Identifier S 0196-2892(98)06849-1.

Mission to Planet Earth (MTPE) data. These remotely sensed data are being and will continue to be used to detect changes from time-varying image sequences [2], [3]. The usefulness of these remotely sensed data in change detection is dependent not only on the radiometric and spatial resolutions of the data, but also on the subsequent processing and the quality of the processed data [4]–[6]. Technological developments in the area of remotely sensed change detection offer more and more possibilities. A responsible use of the processed geodata is only possible if the quality of these data is known [6], [7]. Therefore, it is important to quantify the degree of error and determine the error sources and their propagation processes. The simple definition of change is the difference in the landscape between two times. Unfortunately, the definition of error is quite close. One of the important issues in change detection is to ensure that the changes detected are not confused with errors. This is a difficult proposition, and unlikely to be handled in a totally satisfactory manner [1]. There are many error sources that are associated with the land surface change products derived from remotely sensed data. These sources include data acquisition, processing, analysis, and conversion. The analysis of change detection accuracy is more complex than single image accuracy assessment. Any of the error sources that affect the accuracy of single image analysis will contribute to the change detection accuracy. There are also many other error sources that uniquely affect the accuracy of change detection, such as vegetational phenology differences, atmospheric effects, and viewing angular differences. Additionally, positional miscorrespondence of two images will significantly affect the change detection accuracy [8]. More importantly, the majority of current change detection techniques depend critically on the accuracy of geometric registration of two images since change analysis is generally performed on a pixel-by-pixel basis [9]. If accurate registration between images is not achieved, spurious differences will be detected merely because the land surface properties at wrong locations are evaluated instead of real changes at the same location between one time and another [4]. The effects of band-to-band misregistration on single-date multispectral classification were explored by several researchers [10]. The effects of misregistration on change detection accuracy were previously explored by only a few studies. A systematic research on this topic was first reported in the early 1990’s using degraded Multispectral Scanner (MSS) data and Thematic Mapper (TM) imagery [4]. However, many important consequences of misregistration on the quality of multitempora

0196–2892/98$10.00  1998 IEEE

DAI AND KHORRAM: IMAGE MISREGISTRATION ON REMOTELY SENSED CHANGE DETECTION

CHARACTERISTICS

OF THE

TABLE I SUDY AREAS

data analysis, especially on the accuracy of remotely sensed change detection, are still unknown. This research aims at contributing to further quantitative understanding of the magnitude and the propagation process of errors caused by geometric misregistration in remotely sensed change detection. This simulation study involves using semivariograms to investigate the band, temporal, and spatial frequency sensitivities of change detection to image misregistration. Furthermore, the impact of misregistration on change detection is evaluated in terms of the accuracy of change detection using the output from an actual change detection algorithm, called Ellipsoidal Change Detector (ECD), proposed in this work. The rest of the paper is organized as follows. In Section II, a brief description of the study areas and data sets is given. The quantification methodologies are described in Section III. The detailed experimental results and analysis are presented in Section IV. Finally, discussion and conclusions are summarized in Section V.

II. STUDY AREAS

AND

DATA SETS

Four study areas in North Carolina with different characteristics were chosen in this quantification research. They are representative of four important land cover types in the North Carolina coastal plain: forest land, agricultural land, pocosin land, and urban/residential area. In each study area, a test window of 200 200 pixels was extracted from the mosaicked TM scenes of North Carolina coastal plain used in a project called Coastal Change Analysis Program (C-CAP) of the National Oceanic and Atmospheric Administration (NOAA), Washington, DC. The multidate images of the same area were coregistered using common ground control points (GCP’s), and an rms error of less than 0.5 pixel at the GCP’s was achieved using the second-order polynomial transformation. The images were resampled and one pixel corresponds to a 28.5 m . Characteristics of the ground resolution of 28.5 study areas are listed in Table I. Their grayscale images are shown in Fig. 1(a)–(h).

AND

1567

IMAGE DATA SETS

III. QUANTIFICATION METHODOLOGIES A. Simulation of Misregistration and General Description of Quantification Methods The registration process can be decomposed into different types of transformations between images, such as scaling, rotational, translational, and skewing differences. By the same token, misregistration can be modeled quantitatively by the these differences, which we call affine misregistration. Mathematically, the affine misregistration can be simulated as be the original image and be the follows. Let misregistered image. Their relationship can be expressed as (1) and are the amount of where misregistration between the original image and the misregistered image. These six parameters describe the differences in scale, rotation, translation, and small nonlinear transformation between the two images. Statistically, the effects of three types of linear misregistrations are the same. However, different modeling approaches should be used in different cases. For example, it might be necessary to consider the effects of rotational misregistration when multiangular imagery is involved, such as in the bidirectional reflectance distribution function (BRDF). In this research, assuming that the misregistration noise is uniformly distributed spatially in a small neighborhood, only local translational misregistration needs to be considered in the simulation procedure. In this case, the misregistration process has two important properties: 1) the difference image has zero sample mean and 2) the registered image is only a shifted version of the original image [11]. The translational misregistration can be simulated by progressively sliding one image at some random directions to the other after the images are well registered using GCP’s. The experiments are then performed to evaluate the consequences of progressive misregistration of the images, focusing on the resultant changes in the differences. With the images in their original position with zero misregistration, it is assumed

1568


(a)

(b)

(c)

(d)

(e)

(f)

(g)

(h)

Fig. 1. Grayscale images of the test windows: (a) Carolina Bays, TM, 1988; (b) Carolina Bays, TM, 1994; (c) Pocosin, TM, 1988; (d) Pocosin, TM, 1994; (e) Wilson fields, TM, 1988; (f) Wilson fields, TM, 1994; (g) Wilmington, TM, 1988; and (h) Wilmington, TM, 1994.

that the differences between the images are a result of real differences on the ground between the images, although these changes between the multitemporal images could be a con-

sequence not only of changes in land cover between the two dates, but also of other differences, such as atmospheric effects and viewing angle changes. As the images are sequentially


(a)

1569

(b)

Fig. 2. Simulation of misregistration process: (a) discrete sliding process and (b) total misregistration.

misregistered, spurious differences will be added to the overall changes (commission error) and some real changes on the ground will be removed (omission error). The differences in changes can be evaluated by both semivariance (SV) increase and percentage of changed areas in land covers. The commission and omission errors introduced by misregistration can be quantified by the actual change detection process. To eliminate the effects of atmospheric and other false changes in the simulation process, each of the images is misregistered against itself and the resultant differences are then analyzed. At zero misregistration, the images are perfectly registered and any other spurious changes are eliminated. Consequently, the relationship between the TM bands, the spatial structure of the images, and the impact of misregistration can be more clearly examined when a duplicated version of the image is running over itself. In this simulation, we define the following three sensitivities of change detection to misregistration: band, spatial frequency, and temporal sensitivities. Band sensitivity is the difference in the responses among the TM spectral bands to misregistration. Band selection is important in both image classification and change detection. Different TM bands could have different sensitivities to misregistration in change detection. Spatial frequency sensitivity is defined as the difference in responses of imagery with different spatial details measured by SV. Temporal sensitivity is the difference in the responses of multidate images of the same area to misregistration. These three sensitivities can provide further insight into the impact of misregistration on change detection. To simulate the misregistration process, images need to be resampled and reconstructed at certain misregistration stages. The resampling process generally introduces interpolation effects. To eliminate the interpolation effects induced during the image resampling process, images are slid only in a discrete and directions so that an interpolation is fashion in both not needed. It is clear that this simplification will not affect the simulation results since a series of discrete points are enough for describing the trends of misregistration effects on change detection. This simulation approach can also be applied for subpixel misregistration. The outputs of any kind are then interpolated over a finer abscissa using spline interpolation to produce continuous results. Simulation of the misregistration process is shown in Fig. 2, in which the discrete sliding

process and the total misregistration are described in Fig. 2(a) and (b), respectively. Emphasis is placed on the output values close to onepixel misregistration since it is this region that has been least understood about the effects of misregistration [4]. For this reason, we develop and apply the Ellipsoidal Change Detection (ECD) method to the test areas while images are progressively misregistered. To simulate real applications, TM bands 2–4 are used in the change detection. For every study area and at every point of misregistration, the total changes detected, true changes present, false changes added by misregistration, and true changes removed by misregistration are identified based on the outputs from the ECD to evaluate the effects of misregistration on change detection accuracy. The false changes caused by misregistration, particularly at one-pixel misregistration, are analyzed. B. Image Preprocessing and Data Normalization The assumption to use remotely sensed data to detect changes is that there is a consistent relationship between remotely sensed brightness values after calibration and the actual surface conditions. However, the data are usually acquired at different dates with varying sensor system responses, phenological variations sun angle, atmospheric, and soil moisture conditions [12]. Due to these effects, there is a need to transform the data from multiple sensors and dates during the image preprocessing stage so that the grayscale of the data will have the same meaning for the data in the whole study and these nonsurface effects can be minimized or eliminated. The simplest normalization procedure is to match the firstand second-order statistics of the probability distribution of the source and reference images. We assume that the data have Gaussian distribution so that we just need to match the mean and standard deviation of the source data to those of the reference data. The procedure is defined as follows: (2) is the normalized pixel value of the source image, where is the original pixel value of the source image, and are the means of the source image and the reference image, respectively, and and are the standard deviations

1570


of the source image and the reference image, respectively. Rearranging this equation, we obtain (3) This procedure can be easily implemented in experiments. C. Simulation Study by SV and Variance (V) The effects of misregistration can be measured by the differences between the original image and the misregistered image in terms of SV’s. In the case of translationally misregistering is compared one image against itself, each pixel , where to a corresponding pixel is the misregistration error in direction and is the direction. The difference between misregistration error in the two image gray values is squared and summed for each pixel location, divided by the total number of comparisons, and then halved. The SV is defined as the halved mean-squared difference between the two image gray values

(4)

SV

is the number of columns and is the number of where rows in the overlapped part of two images. There is a close relationship between the SV and the image autocorrelation function (ACF). A few manipulations of (4) give ACF

SV

ACF

(5)

where

(6)

AFC

Considering that the ACF is the Fourier transform of the image power spectrum, the sensitivity of fine detail (i.e., wide frequency support) imagery to misregistration is immediately evident. If the images are stationary (any partial area across the image and its spatially shifted area have the same joint density, i.e., no specific patterns spatially across the image), in the process of sliding of one image over a copy of itself, the SV will gradually increase from a value of zero at zero ) and will asymptotically misregistration ( approach the variance (V) of the original image

(7)

V

is the mean of the image. At this level of misregiswhere tration, the pixels being compared are essentially uncorrelated and change signals are totally lost in the noise, so that any further increase in misregistration will have no effect. The number of pixels at which this occurs is called the range [4]. The SV can be normalized by the image V SV (normalized)

SV V

(8)

The normalized SV has several important characteristics: 1) it will approach 1.0 when the images are stationary and the range is reached; 2) it could exceed 1.0 or never reach 1.0 when the images are nonstationary; and 3) differences between the SV curves due to the different image V’s will be reduced and the form of the curve will be mainly due to the spatial structure of the image. For images with a substantial amount of high spatial frequency, the range will be small, whereas if there are a substantial amount of near-uniform areas of low spatial frequencies, the SV’s will rise slowly and should gradually reach the maximum value. By the same token, for multitemporal images and , the SV is defined as

(9)

SV

and , the images are perfectly When registered and the SV at this point represents the actual differences between the two images. In practical simulation, a perfect registration is assumed so it can be used as a basis to evaluate the total false changes introduced by misregistration. The percentage increase in SV ( ) at a misregistration compared with the SV (SV ) at zero misregistration is calculated as follows: SV

SV SV

(10)

This quantity can be used to evaluate the relationship between the percentage of SV increase caused by misregistration and the amount of misregistration. D. Ellipsoidal Change Detection Technique and False Change Analysis To evaluate the impact of misregistration on the accuracy of change detection, a method that we call ellipsoidal change detection is proposed and used to subjectively evaluate the impact of misregistration on the change detection accuracy. In change detection, image differencing is commonly used to describe the changes between two images. Therefore, the multispectral differenced image between corresponding pixels of two images can be used to construct the feature space describing changes. In the case of multidimensional Gaussian feature space, a hyperellipsoidal decision boundary between change and nonchange is more representative of changes than any other kind of decision boundaries, such as hypercubic. This can be noticed in Fig. 3, in which three example plots of the feature space related to the differenced images of TM bands 2–4 are shown. Based on the feature space plots, we notice that the distributions of the differenced images for all TM bands are approximately symmetric and the density contours in the multidimensional feature space are approximately hyperellipsoidal. Therefore, the change signal can approximately be modeled by a Gaussian distribution and the Mahalanobis distance function of -dimensional difference image can be used as the discriminant function between change and nonchange.


(a)

1571

(b)

(c)

Fig. 3. Example feature space plots describing changes. Features 1–3: difference values of TM bands 2–4 of the two images, respectively: (a) change distribution in Feature 1–Feature 2 space, (b) change distribution in Feature 1–Feature 3 space, and (c) change distribution in Feature 2–Feature 4 space.

Mahalanobis distance function can be expressed as follows: (11) where is the differenced image with bands and and are the mean vector and the covariance matrix, respectively. Further insight of the Mahalanobis distance modeling of changes can be obtained by vector diagonalization of the covariance matrix [13]. The eigenvector transformation is an orthonormal transformation that diagonalizes the covariance matrix (12) is the orthonormal transformation matrix consisting where of the eigenvectors of the covariance matrix. Based on the eigenvector diagonalization, we can obtain (13) where (14) are the eigenvalues of the covariance matrix. and Therefore, the Mahalanobis distance can be expressed in the transformed space as

(15) where

(16)

From the equation above, it is clear that the locus of all points at a given Mahalanobis distance is a hyperellipsoid. That is why we call this technique ellipsoidal change detection.

Furthermore, it is known the principal axes of this ellipsoid are aligned with the eigenvectors. This distance is different from the Euclidean distance in that it is weighted by the eigenvalues of the covariance matrix of the difference images. Therefore, the Mahalanobis distance based discriminant function for change detection can be expressed as follows: threshold threshold

unchanged changed

(17)

is the Mahalanobis distance at a specific pixel. where There are at least two reasonable ways to determine the threshold to identify change and nonchange based on the Mahalanobis distance image. First, we can use the point where the derivative of the histogram of the Mahalanobis distance image reaches the minimum as the threshold since this point has the steepest change in the number of pixels. This idea is described in Fig. 4, where a portion of the histogram of the Mahalanobis distance image (0–40 for 8-bit unsigned data) distribution is shown in Fig. 4(a) and with an approximate the maximum point of the negative derivative of the histogram of the Mahalanobis distance image can be used as the threshold to identify change and nonchange, as shown in Fig. 4(b). Another method is to predefine a percentage of change based on other criteria and use this percentage to derive the threshold. Based on this technique, a reasonable and fixed threshold that can approximately estimate the percentage of changes (i.e., 10% of the total area) is used for all experiments in the false change analysis. Since only a relative amount of changes are of interest in this work, the absolute value of the threshold is irrelevant to the results of the analysis and the impact of the different thresholds on false change analysis results can be ignored. Based on the output from this change detection technique, a true change ( ) can be defined as the amount of change detected at zero misregistration, i.e., when two multitemporal images are perfectly registered. At every misregistration point, while we slide one image over another, we apply the ECD method to the overlapped area to detect the overall changes ( ) between these two misregistered images. At the same time, the true changes ( ) can be detected over the over-

1572


(a)

(b)

Fig. 4. Distribution of the Mahalanobis distance image and its negative derivative for determination of the threshold in change detection: (a) histogram of the Mahalanobis distance image and (b) negative derivative of (a).

lapped area using zero-misregistered images. The true changes detected ( ), when images are misregistered, are then the and intersection of (18) ) (false posiThe false changes added by misregistration ( tive or commission error) can be expressed as and

(19)

The true changes removed due to misregistration ( negative or omission error) can then be defined as and

) (false

(20)

Based on these definitions, the following evaluation quantities can be defined to evaluate the effect of misregistration on the overall change detection accuracy: 1) percentage of true changes detected ( ) when misregistration exists (21) 2) percentage of false changes added ( istration

) due to misreg-

(22)

3) percentage of true changes removed ( tration

IV. EXPERIMENTAL RESULTS A. Band Sensitivity Band selection is an important procedure using multispectral satellite imagery. Since every band has its own characteristics, selection of one specific band or a combination of multiple bands for a specific problem is always critical in most remote-sensing applications [14]. Therefore, it is important to investigate the sensitivity of TM bands to misregistration in change detection. In this research, the six nonthermal bands of the Wilmington image were used in the experiments. The six nonthermal bands were misregistered against themselves. Fig. 5(a) and (b) shows that the SV and normalized SV increases with progressive misregistration. As we can notice, the plots for all bands have an overall convex form and the SV and normalized SV plots for TM band 4 shows different pattern from those for any other nonthermal TM bands. The slopes of the SV and normalized SV increases for TM band 4 display the steepest rise within the first three pixels of misregistration than those of any of the other five nonthermal bands. The range for TM band 4 is less than 10 pixels, as comparing to any other bands that have almost the same length of range of about 15 pixels. Other TM test images shows the same results. From these experiments, we can conclude that TM band 4 is the most sensitive band to misregistration for the purpose of change detection among the six nonthermal TM bands. This result can be understood by knowing that TM band 4 is the most sensitive band to vegetation change. Therefore, misregistration has stronger effect on the changes detected by using band 4.

) by misregisB. Spatial Frequency Sensitivity and Temporal Sensitivity (23)

4) percentage of overall change increase ( ) caused by misregistration (24)

Fig. 6 shows the results obtained when all eight images are progressively misregistered against themselves using TM band 4. As anticipated, the plots for all images also have an overall convex forms. The SV plot and normalized SV plot show similar results in terms of the overall trends of SV increase and normalized SV increase with misregistration. All four pairs of plots show that temporal images of the same areas but of different times have the same sensitivities to misregistration


(a)

1573

(b)

Fig. 5. TM band sensitivity to misregistration. Six nonthermal TM bands are misregistered against themselves: (a) increases in SV’s with progressive misregistration and (b) increases in normalized SV’s with progressive misregistration.

(a)

(b)

Fig. 6. Spatial frequency sensitivity and temporal sensitivity to misregistration. Eight images are progressively misregistered against themselves: (a) increases in SV and (b) increases in normalized SV’s.

when the basic image structure (spatial frequency) remains the same. There are differences between the individual pairs of images. The pair of images of Bladen pocosin are most sensitive to misregistration, which have SV increases by 60% normalized SV within less than 2.3 pixels of misregistration. While the images of Wilmington and Wilson fields have moderate sensitivity to misregistration, the images of Carolina Bays have the lowest sensitivities to misregistration. As also noticed in previous work by [4], no simple trends can be observed in terms of the relationship between the increase in SV’s and normalized SV’s and obvious vegetation types and the moisture change patterns and spatial image structure (spatial frequency) are two important factors that affect the sensitivity to misregistration. The SV’s were normalized by dividing them by the V’s of the images, as shown in Fig. 6(b). From this plot, we can notice that there are basically three types of responses. The images of Bladen pocosin and Wilson fields reach this maximum in about 20 pixels of misregistration, while the images of Wilmington reaches the stabilized point of 0.8 within 10 pixels of misregistration. However, the images of Carolina Bays just

gradually reach the point of 0.8 in normalized SV increase with more than 28 pixels of misregistration, which suggests that either larger misregistration than 28 pixels of movement would be required to reach the maximum value of 1.0 or the images are not stationary. C. Percentage Increase of SV Changes Due to Misregistration The SV plots of the misregistration processes using multitemporal images of the four study areas are shown in Fig. 7(a). The actual changes between the two images are displayed as the intercepts with -axis. When the images are progressively misregistered, false changes are introduced and their SV’s increase from their bases due to the combined effects of actual changes in the images and those caused by misregistration. To evaluate the contribution of the error introduced by misregistration compared with actual changes, the actual percentage increases of SV’s are plotted in Fig. 7(b). It is notable that, even for less than three pixels of misregistration, the percentage increases on SV’s exceed 50% for all of the four areas. Furthermore, with less than two pixels of misregistration for Wilmington urban area, the percentage increase in SV

1574


(a)

(b)

Fig. 7. SV increases with misregistration when pairs of TM band 4 images of two different dates are misregistered: (a) absolute SV increases with misregistration and (b) percentage increases of SV’s due to misregistration.

exceeds 100%. This result further supports the conclusion that the higher the spatial frequency in the images, the greater the influence of misregistration on change detection accuracy. D. Analysis of False Changes Caused by Misregistration The simulation study on SV increases when misregistration is present shows only general trends and combined effects of real changes on the ground and changes caused by misregistration. When misregistration exists in the data, it will introduce both additional false-positive changes (commission errors) and false-negative changes (omission errors) as well in the detected changes. More importantly, misregistration reduces the ability of any change detector to detect the true changes and increase the probabilities to pick additional false changes because the wrong locations are compared in the presence of misregistration. This effect of misregistration has barely been understood in most remote-sensing applications. Besides, we need a more efficient presentation method than SV increase to express the actual errors induced by a certain degree of misregistration, which a change detection system is forced to carry. That is why we designed a new change detection technique to simulate the error propagation process within a change detection system with progressive misregistration. In the following experiments, we applied the ECD method introduced in the previous section to the four test areas. To simulate real applications, TM bands 2–4 were used in the change detection. Images were progressively misregistered by 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, and 10 pixels direction and 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, in 8, 9, 9, 10, and 10 pixels in direction, which correspond to 0, 1.0000, 1.4142, 2.2361, 2.8284, 3.6056, 4.2426, 5.0000, 5.6569, 6.4031, 7.0711, 7.8102, 8.4853, 9.2195, 9.8995, 10.6301, 11.3137, 12.0416, 12.7279, 13.4536, and 14.1421 pixels in total misregistration, respectively. The threshold in the Mahalanobis distance-based discriminant function was set to 2.5 for all images to assure the comparability among results. For each area and at every point of misregistration, the total changes detected, the true changes present, the false changes added by misregistration, and the true changes removed by misregistration were identified. Fig. 8 shows the experimental results of the false change analysis.

From the plots in Fig. 8, we notice that, for all four study areas, the percentages of true changes detected within onepixel misregistration dramatically drop to 62.5% for the images of Carolina Bays, 35.0% for the images of Bladen pocosin, 55.6% for the images of Wilson fields, and 40.9% for the images of Wilmington urban area. The main contribution to this drop is the percentage of true changes removed by misregistration (omission error), which is 32.5, 60.0, 42.0, and 53.4%, respectively. On the other hand, misregistration causes additional false changes at the same time. The percentages of false change added by misregistration (commission error) are slightly greater than the corresponding percentages of true changes removed for all areas, except the images of Wilson fields, which causes the overall changes detected for the entire areas remains slightly increasing over the range of misregistration. Since there is only a slight difference between the commission and omission errors, the overall changes detected remain almost constant, regardless of the amount of misregistration. It is desirable to further investigate the effect of onepixel misregistration on the spatial distributions of the false changes caused by misregistration since one-pixel registration accuracy is usually used as one of the registration objectives in most applications. Fig. 9 shows the spatial distributions of the changes detected for the images of Carolina Bays. The total changes detected are shown in Fig. 9(a). The actual true changes present in the images are shown in Fig. 9(b). The false changes added by misregistration are shown in Fig. 9(c), in which we can notice that the false changes caused by misregistration are mainly distributed spatially along the edges of the images. The true changes removed by misregistration, on the other hand, are spatially distributed away from the edges, as shown in Fig. 9(d). It is interesting to quantitatively evaluate the required registration accuracy to achieve a specific accuracy of change detection. The registration accuracy quoted here is measured by the root mean square error (RMSE) at the GCP’s, although other criterion may be used for more subjective and consistent measurement of the accuracy, i.e., [15]. From the curve in Fig. 8(a), we notice that less than 0.2667 pixel of registration accuracy is needed to assure an accuracy of 90% for change


1575

(a)

(b)

(c)

(d)

Fig. 8. Analysis of the false changes caused by misregistration using the ellipsoidal change detection: (a) false changes in Carolina Bays; (b) false changes in Bladen pocosin; (c) false changes in Wilson fields; and (d) false changes in Wilmington area.

detection in the images of Carolina Bays. By the same token, we obtain that registration accuracy of 0.1538, 0.1838, and 0.1692 pixel are needed for the images of Bladen pocosin, Wilson fields, and Wilmington area, respectively, to detect 90% of real changes. On average, less than 0.1934 pixel of registration accuracy should be achieved to detect 90% of the true changes, which means that a registration accuracy of less than one-fifth of a pixel is required to achieve a change detection error of less than 10%. V. DISCUSSION

AND

CONCLUSIONS

This study is a comprehensive simulation study of the error propagation process of misregistration in remotely sensed change detection. The simulation research concentrated on quantification of the error terms caused by misregistration when multitemporal images were used to detect land cover changes. The evidence from the simulations strongly suggests that misregistration has significant effects on the accuracy of remotely sensed change detection. Subpixel misregistration has a major impact on change analysis since marked errors tend to occur within one-pixel misregistration. The results also indicate the need to achieve higher values of registration accuracy to assure a reasonable accuracy for change detection from remotely sensed imagery. From these experiments, we conclude that TM band 4 is most sensitive to misregistration among the six nonthermal TM bands since the slopes of the SV and normalized SV increases display the steepest rise within the first three pixels

of misregistration among six nonthermal bands. The range for TM band 4 is less than 10 pixels, as comparing to any of the other bands, which have about 15 pixels. Therefore, misregistration has stronger effect on the changes detected by using band 4. All four pairs of plots show that temporal images of the same areas but of different times have the same sensitivity to misregistration if the basic image structure (spatial frequency) remains the same. No simple trends can be observed in terms of the relationship between the increase in SV’s and normalized SV’s and obvious vegetation types. However, the moisture change patterns and spatial image structure (spatial frequency) are two important factors that affect the sensitivity of misregistration. Even for less than three pixels of misregistration, the percentage increases on SV’s exceeds 50% for all four study areas. For Wilmington urban area, the percentage increase in SV even exceeds 100% with less than two pixels of misregistration, which further supports the conclusion that the finer the spatial frequency in the images, the greater the effects of misregistration on change detection accuracy. The change detection accuracy drops dramatically within the first pixel of misregistration. The true changes removed (omission error) by misregistration is the main reason why the true changes detected drop. Misregistration causes false changes at the same time. The percentage of false change added by misregistration are slightly greater or slightly less than the corresponding percentage of true changes removed. It is interesting to notice that the overall changes detected remain

1576


(a)

(b)

(c)

(d)

Fig. 9. Spatial distribution of the false changes added by misregistration and true changes removed by misregistration when multitemporal images are one-pixel misregistered: (a) the total changes detected by ECD; (b) true changes present between the images; (c) false changes added by misregistration; and (d) true changes removed by misregistration.

almost constant, regardless of the amount of misregistration. We have further observed that the false changes added by misregistration are mainly distributed spatially along the edges of the images. The true changes removed by misregistration, on the other hand, are spatially distributed away from edges. On average, less than 0.1934 pixel of registration accuracy should be achieved to detect 90% of the true changes, which means that a registration accuracy of less than one-fifth of a pixel is required to achieve a change detection error of less than 10%. This study has provided systematic quantification techniques for evaluating and investigating the ability of remotely sensed data in change detection in the presence of image misregistration. The scientific content of this work also lies in establishing a framework for exploring the effects of misregistration on multisource data analysis, such as data fusion. Without accurate registration of images, the resultant errors are likely to compromise other technical improvements in multisource data analysis. The results strongly suggest that an accurate image registration system be established in the design of any multisource remote-sensing systems. Our plans for future work include the development and implementation of an automated image registration system with higher accuracy in multisource data fusion systems. ACKNOWLEDGMENT Thanks are extended to Dr. W. Snyder and Dr. G. Bilbro of the Department of Electrical and Computer Engineering and

Dr. H. Devine and Dr. J. Roise of the Department of Forestry at North Carolina State University for the stimulating discussion and their helpful suggestions. The authors are also thankful to the anonymous referees for their constructive comments, which improved the paper.

REFERENCES [1] S. Khorram, G. S. Biging, N. R. Chrisman, D. R. Colby, R. G. Congalton, J. E. Dobson, R. L. Ferguson, M. F. Goodchild, J. R. Jensen, and T. H. Mace, “Accuracy assessment of land cover change detection,” in Mono., ASPRS, Photogramm. Eng. Remote Sensing, to be published. [2] P. R. Coppin and M. E. Bauer, “Digital change detection in forest ecosystems with remote sensing imagery,” Remote Sens. Rev., vol. 13, pp. 207–234, 1996. [3] A. Singh, “Digital change detection techniques using remotely-sensed data,” Int. J. Remote Sensing, vol. 10, pp. 989–1003, 1989. [4] J. R. G. Townshend, C. O. Justice, C. Gurney, and J. McManus, “The impact of misregistration on change detection,” IEEE Trans. Geosci. Remote Sensing, vol. 30, pp. 1054–1060, Sept. 1992. [5] P. H. Swain, V. C. Vanderbilt, and C. D. Jobusch, “A quantitative applications-oriented evaluation of Thematic Mapper design specifications,” IEEE Trans. Geosci. Remote Sensing, vol. GE-20, pp. 370–377, Mar. 1982. [6] P. E. Anuta, L. A. Bartlucci, M. E. Dean, D. F. Lozano, E. Malaret, C. D. McGillem, J. A. Valdes, and C. R. Valenzuela, “Landsat-4 MSS and Thematic Mapper data quality and information content analysis,” IEEE Trans. Geosci. Remote Sensing, vol. GE-22, pp. 222–236, May 1984. [7] E. Malaret, L. A. Bartolucci, D. F. Lozano, P. E. Anuta, and C. D. McGillem, “Landsat-4 and Landsat-5 Thematic Mapper data quality analysis,” Photogramm. Eng. Remote Sensing, vol. 51, pp. 1407–1416, Sept. 1985. [8] X. Dai and S. Khorram, “Quantification of the impact of misregistration on the accuracy of remotely sensed change detection,” in Proc. 1997


[9]

[10] [11] [12] [13] [14] [15]

IEEE Int. Geosci. Remote Sensing Symp. (IGARSS’97), Singapore, pp. 1763–1765. X. Dai, S. Khorram, and H. Cheshire, “Automated image registration for change detection from Thematic Mapper imagery,” in Proc. 1996 IEEE Int. Geosci. Remote Sensing Symp. (IGARSS’96), Lincoln, NE, pp. 1601–1603. F. C. Billingsley, “Modeling misregistration and related effects on multispectral classification,” Photogramm. Eng. Remote Sensing, vol. 48, pp. 421–430–1416, 1982. T. F. Knoll and E. J. Delp, “Adaptive gray scale mapping to reduce registration noise in difference images,” Comput. Vision, Graph., Image Processing, vol. 33, pp. 129–137, 1986. E. F. Lambin and D. Ehrlich, “The surface temperature-vegetation index space for land cover and land cover change analysis,” Int. J. Remote Sensing, vol. 17, pp. 463–487, 1996. R. M. Haralick and L. G. Shapiro, Computer and Robot Vision. Reading, MA: Addison-Wesley, 1993, vol. 1. J. R. Jenson, Introductory Digital Image Processing. Englewood Cliffs, NJ: Prentice-Hall, 1996. J. Y. Chiang and B. J. Sullivan, “Coincident bit counting—A new criterion for image registration,” IEEE Trans. Med. Imag., vol. 12, pp. 30–38, Mar. 1993.

Xiaolong Dai (S’95–M’97) received the B.S.E.E. degree in electrical engineering from the Huazhong University of Science and Technology, Wuhan, China, in 1986, the M.S. degree in ecology from the Chinese Academy of Forestry, Beijing, China, in 1989, and the Ph.D. degree, both in electrical engineering and forestry, from North Carolina State University (NCSU), Raleigh, in 1997. He was with the Chinese Academy of Sciences from 1989 to 1992. He has been affiliated with the Center for Earth Observation (CEO) (formerly the Computer Graphics Center) at NCSU since 1992. He is currently a Senior Research Associate at CEO. For the past 10 years, he has been involved in conducting research in digital image processing and computer vision as applied to remotely sensed data and medical imagery, as well as in geoinformatics. His research interests include computer vision, image processing, statistical and neural network-based pattern recognition, development of automated data processing systems in remote sensing, change analysis, and multisource data fusion. Dr. Dai is a member of the IEEE Geoscience and Remote Sensing Society, IEEE Pattern Analysis and Machine Intelligence Society, and American Society of Photogrammetry and Remote Sensing.

1577

Siamak Khorram (M’84) received the M.S. degree in engineering and the M.S. degree in ecology from the University of California (UC), Davis, and the Ph.D. degree in water science and engineering with emphasis on remote sensing and image processing in 1975 under a joint study program from UC, Berkeley, and UC, Davis. He has been involved in teaching and conducting research in remote sensing, geoinformatics, environmental science and engineering, development of computer-based information technologies, and systems integration since 1975. In 1982, he established the Computer Graphics Center at North Carolina State University, Raleigh, as a university-wide organized research center involved in conducting and facilitating research and training in remote sensing and geoinformatics. In 1995 and 1996, he served as the Dean and Vice President for Academic Programs at International Space University (ISU), Strasbourg, France. In 1997, he established the Center for Earth Observation and serves as the Professor and the Director. He has taught courses in air photo interpretation and photogrammetry, environmental remote sensing, digital image processing, and information systems. He has trained a number of Ph.D. and M.S. students in environmental sciences as well as in electrical engineering. He has authored more than 110 technical publications. Dr. Khorram serves on the Board of Trustees of ISU and is a founding member of the Peaceful Uses of Space for all Humanity in Switzerland. He is a member of several professional societies and has been recognized and awarded by a number of scientific societies.