rus
Issue #4/2023
Yu. I. Yakimenko, V. I. Bobkov, I. V. Yakimenko
Method of Detecting Artifacts on a Complex Background by an Optical-Eelectronic System
Method of Detecting Artifacts on a Complex Background by an Optical-Eelectronic System
DOI: 10.22184/1993-7296.FRos.2023.17.4.272.282
A procedure for developing a detection method using a passive optoelectronic system of an unmanned aircraft against a complex background formed by atmospheric radiation in the far infrared range (8–13 microns) is proposed. The atmospheric background on which the unmanned aircraft is detected is formed by the radiation of the cloudy atmosphere when observed from the Earth’s surface. Of particular interest is the complex background created by cumulus clouds of different scores or other classes of clouds with discontinuities. The following assumptions are accepted: a short-focus optoelectronic system has a wide field of view, video information about the artifact and background characteristics is presented in binary form. The processed video stream is a two-dimensional array, the elements of which contain information about the level of energy brightness of radiation in the selected direction. The emphasis is on the need to monitor changes in the structure of the emitting background and the absence of the need to process each frame of the video stream.
A procedure for developing a detection method using a passive optoelectronic system of an unmanned aircraft against a complex background formed by atmospheric radiation in the far infrared range (8–13 microns) is proposed. The atmospheric background on which the unmanned aircraft is detected is formed by the radiation of the cloudy atmosphere when observed from the Earth’s surface. Of particular interest is the complex background created by cumulus clouds of different scores or other classes of clouds with discontinuities. The following assumptions are accepted: a short-focus optoelectronic system has a wide field of view, video information about the artifact and background characteristics is presented in binary form. The processed video stream is a two-dimensional array, the elements of which contain information about the level of energy brightness of radiation in the selected direction. The emphasis is on the need to monitor changes in the structure of the emitting background and the absence of the need to process each frame of the video stream.
Теги: artifact atmospheric background background-target image field of view nfrared range passive optoelectronic system robotic system unmanned aircraft артефакт атмосферный фон беспилотное воздушное судно инфракрасный диапазон пассивная оптико-электронная система поле зрения роботизированная система фоно-целевое изображение
Method of Detecting Artifacts on a Complex Background by an Optoelectronic System
Yu. I. Yakimenko, V. I. Bobkov, I. V. Yakimenko
Branch of National Research University Moscow Power Engineering Institute, Smolensk, Russia
A procedure for developing a detection method using a passive optoelectronic system of an unmanned aircraft against a complex background formed by atmospheric radiation in the far infrared range (8–13 microns) is proposed. The atmospheric background on which the unmanned aircraft is detected is formed by the radiation of the cloudy atmosphere when observed from the Earth’s surface. Of particular interest is the complex background created by cumulus clouds of different scores or other classes of clouds with discontinuities. The following assumptions are accepted: a short-focus optoelectronic system has a wide field of view, video information about the artifact and background characteristics is presented in binary form. The processed video stream is a two-dimensional array, the elements of which contain information about the level of energy brightness of radiation in the selected direction. The emphasis is on the need to monitor changes in the structure of the emitting background and the absence of the need to process each frame of the video stream.
Keywords: Infrared range, field of view, passive optoelectronic system, robotic system, atmospheric background, background-target image, artifact, unmanned aircraft
The article received: May 16, 2023
The article accepted: June 10, 2023
1. Introduction
The aim of the study is to develop a method of optical and information support for the detection of artifacts by a robotic system against a complex background under the following assumptions:
the artifact to be detected is an unmanned aircraft (UAV);
the information channel of the robotic system is a passive optoelectronic system (POES) with a wide field of view, operating in the far infrared range (8–13 microns);
video information about the background and artifacts in the field of view after final processing is presented in binary form (Fig. 1).
The atmospheric background (AB) on which the UAV is detected is formed by the radiation of the cloudy atmosphere when observed from the Earth’s surface. Of particular interest is the complex background created by cumulus clouds of different scores or other classes of clouds with discontinuities. In the process of receiving and processing radiation from the atmospheric background the POES and the UAV form a video stream on the terminal device, each frame of which is a two-dimensional array UN, M, the elements of which contain information about the level of energy brightness of radiation in the selected direction.
2. Fundamentals of the method of optical and information support for artifact detection
The method of optical and information support for the detection of artifacts on a complex background by a robotic system is based on the background principle of information extraction [1–4]. Its essence lies in the fact that in the absence of a priori information about the presence of UAVs in the field of view of the POES, it is necessary to monitor local changes in one or more parameters of the spatio-temporal structure of AB radiation different from knowledge about natural patterns. Changes in these parameters occur due to the distortion of natural patterns of the spatial structure of the AB radiation by UAV radiation (Fig. 2) [1].
2.1. A Method for Obtaining the Spatial Component of Optical and Information Support
Experimental studies of the spatial structure of AB radiation consist in assessing the dependence of the spatial correlation coefficient R(n) of various forms of clouds in the horizontal directions between the rows and in the vertical directions between the columns of arrays of elements of the background-target image (BTI). A characteristic difference in the spatial structure of the radiation of different cloud classes is the size of the inhomogeneities, which were determined by the value of the spatial correlation coefficients between the rows and columns of the BTI arrays, taking a value above the level of 0.5. Hence, by the level of 0.5 of the spatial correlation coefficient R(n), knowing the step of the angular shift between the rows, it is possible to estimate the angular dimensions by the angle of the place (e), and between the columns the angular dimensions by the azimuth (b) of the AB inhomogeneities (Fig. 3) [5–7].
Thus, the obtained results of experimental studies made it possible to estimate in two directions the angular dimensions of the AB inhomogeneities, which became the basis of the spatial component of the optical and information support for the detection of the UAV image on the AB.
Analysis of the results of studies of the radiation of AB inhomogeneities (Fig. 4) made it possible to divide them into two groups depending on their angular size:
The first group includes those classes of clouds that contain small-scale inhomogeneities of magnitude 5–15° in the vertical and horizontal directions: cumulus (Cu), altocumulus (Ac), cirrocumulus (Cc) and cirrus (Ci) (Fig. 4a).
The second group includes cloud classes, which contain large-scale inhomogeneities with angular dimensions exceeding the obtained BTI, are 25–40°: stratus (St), strato-cumulus (Sc), cirro-stratus (Cs) cloud forms and clear skies (Fig. 4b) [1–4].
On the basis of the knowledge gained about the spatial spectra of inhomogeneities of AB and point images of UAVs, a spatial method for detecting UAVs on AB was developed (Fig. 2). The essence of the method consists in preliminary segmentation of the BTI before applying the threshold processing algorithm, which distinguishes it from the known methods (Fig. 2). The size of the segments is determined in accordance with the method of obtaining the spatial component of the optical and information support for detection (Fig. 5 a, b). This allows us to assume that within the angles limited by the size of the identified inhomogeneities, the random process of AB radiation can be considered stationary, since its spatial spectrum does not contain high-frequency components. In contrast, the random process of UAV radiation, the spatial spectrum always contains high-frequency components, which makes it possible to develop a decisive rule for the algorithm of threshold processing of BTI segments (Fig. 2).
Thus, the proposed spatial method for detecting UAVs on AB allows, by sequentially applying the segmentation algorithm and the algorithm for threshold processing of BTI segments, to obtain information about the presence of artifacts in the field of view of the POES, presented in binary form with the possibility of determining the coordinates of the UAV [5–7].
2.2. A Method for Obtaining
the Spatio-Temporal Component
of Optical and Information Support
The process of AB radiation is associated with thermodynamic and turbulent processes occurring in the atmosphere, and is random non-stationary both in space and time. It is known that all random processes under long-term consideration are always non-stationary by nature, but for each of them there is a limited time interval when a random process can be considered stationary. In order for the spatial optical information support within the detection method to contain stationary characteristics of the random process of AB radiation, it is necessary to periodically refine them.
Such periodicity can be estimated using a method for obtaining the spatio-temporal component of optical and information support for detecting artifacts on a complex background (Fig. 2). The basis of this method is the calculation of the coefficient of mutual correlation between the frames of the video stream arriving at regular intervals (Fig. 6a). The time interval corresponding to the level of 0.5 of the coefficient of mutual correlation between the frames of the video stream (Fig. 6 b) allows us to estimate the stationarity time – the “lifetime” of the characteristics of the spatial structure of the AB, i. e. the time interval with the periodicity of which the spatial optical and information support of detection should be updated [1–4].
2.3. Method of Obtaining the Time Component of Optical
and Information Support
In addition to the above, the question of the expediency of processing all frames of the video stream generated by the POES during processing by algorithms of the spatial method of detecting UAVs on AB remains relevant (Fig. 2). The hypothesis was put forward that the frequency of the fundamental harmonic of the spectral power density (SPD) fluctuations in the brightness of the radiation of AB inhomogeneities will be lower than the frame frequency of the POES. Therefore, it is possible to reduce the number of frames from the video stream being processed by algorithms within the spatial method of detecting UAVs on the AB.
To find the optimal frame processing frequency, a method for obtaining the time component of optical and information support was developed. The estimation of temporal variability was carried out by estimating the frequency of the main harmonic of the SPD fluctuations in the brightness of the radiation of AB inhomogeneities formed by different classes of clouds and matching with it the frame rate for processing from the video stream.
To obtain SPD estimates, fluctuations in the brightness of the radiation of AB inhomogeneities were measured in fixed directions along the angle of the place (e) in the near-horizon region with a sampling frequency a thousand times higher than the frame scan. At the same time, the azimuth value (b) remained constant for several minutes (Fig. 7a). In each direction studied, sequences of several thousand values of fluctuations in the brightness of the radiation of AB inhomogeneities for different classes and cloud scores were obtained [1–5].
The obtained arrays were subjected to statistical processing using the periodogram method of estimating the SPD. The results of statistical studies were an interval of 0.08–0.25 Hz for estimating the fundamental frequency of the harmonic of the SPD fluctuations in the brightness of the radiation of AB inhomogeneities formed by clouds of different classes and scores. Based on the results obtained, it became possible to choose the optimal frame rate from the video stream for subsequent processing by the spatial method of detecting UAVs on AB at frequencies significantly lower (up to 1 Hz) than the frame rate (50 Hz) of the video stream (Fig. 7 b).
Conclusions
Thus, the developed method of optical and information support for the detection of artifacts by a robotic system on a complex background based on the background principle of information extraction consists of three ways to obtain the necessary components of optical and information support: temporal; spatial-temporal; spatial.
The method of obtaining the time component of the optical information support makes it possible to optimize the selection of the frame rate from the video stream for subsequent processing by the spatial method of detecting the UAV on the AB.
The method of obtaining the spatio-temporal component of optical information support makes it possible to optimize the time interval with the frequency of which the segment size selection should be updated. This forms the basis of a method for obtaining the spatial component of optical and information support necessary for the implementation of a spatial method for detecting UAVs on the AB, consisting of the BTI segmentation algorithm and the threshold processing algorithm.
The use of the method of optical and information support for detection will allow the robotic system to provide information about the presence of artifacts in the field of view of the POES in the form of a binary BTI, which will further provide the ability to determine the coordinates of the UAV in space.
AUTHORS
Igor Vladimirovich Yakimenko, Doctor of Technical Sciences, Associate Professor, branch of the National Research University Moscow Power Engineering Institute in Smolensk, Smolensk, Russia.
ORCHID: 0000-0002-1003-8403
Vladimir Ivanovich Bobkov, Doctor of Technical Sciences, Associate Professor, branch of the National Research University Moscow Power Engineering Institute in Smolensk, Smolensk, Russia.
ORCHID: 0000-0002-5715-7450
Yuri Igorevich Yakimenko, post-graduate student, branch of the National Research University Moscow Power Engineering Institute in Smolensk, Smolensk, Russia.
ORCHID: 0009-0001-2631-5997
Yu. I. Yakimenko, V. I. Bobkov, I. V. Yakimenko
Branch of National Research University Moscow Power Engineering Institute, Smolensk, Russia
A procedure for developing a detection method using a passive optoelectronic system of an unmanned aircraft against a complex background formed by atmospheric radiation in the far infrared range (8–13 microns) is proposed. The atmospheric background on which the unmanned aircraft is detected is formed by the radiation of the cloudy atmosphere when observed from the Earth’s surface. Of particular interest is the complex background created by cumulus clouds of different scores or other classes of clouds with discontinuities. The following assumptions are accepted: a short-focus optoelectronic system has a wide field of view, video information about the artifact and background characteristics is presented in binary form. The processed video stream is a two-dimensional array, the elements of which contain information about the level of energy brightness of radiation in the selected direction. The emphasis is on the need to monitor changes in the structure of the emitting background and the absence of the need to process each frame of the video stream.
Keywords: Infrared range, field of view, passive optoelectronic system, robotic system, atmospheric background, background-target image, artifact, unmanned aircraft
The article received: May 16, 2023
The article accepted: June 10, 2023
1. Introduction
The aim of the study is to develop a method of optical and information support for the detection of artifacts by a robotic system against a complex background under the following assumptions:
the artifact to be detected is an unmanned aircraft (UAV);
the information channel of the robotic system is a passive optoelectronic system (POES) with a wide field of view, operating in the far infrared range (8–13 microns);
video information about the background and artifacts in the field of view after final processing is presented in binary form (Fig. 1).
The atmospheric background (AB) on which the UAV is detected is formed by the radiation of the cloudy atmosphere when observed from the Earth’s surface. Of particular interest is the complex background created by cumulus clouds of different scores or other classes of clouds with discontinuities. In the process of receiving and processing radiation from the atmospheric background the POES and the UAV form a video stream on the terminal device, each frame of which is a two-dimensional array UN, M, the elements of which contain information about the level of energy brightness of radiation in the selected direction.
2. Fundamentals of the method of optical and information support for artifact detection
The method of optical and information support for the detection of artifacts on a complex background by a robotic system is based on the background principle of information extraction [1–4]. Its essence lies in the fact that in the absence of a priori information about the presence of UAVs in the field of view of the POES, it is necessary to monitor local changes in one or more parameters of the spatio-temporal structure of AB radiation different from knowledge about natural patterns. Changes in these parameters occur due to the distortion of natural patterns of the spatial structure of the AB radiation by UAV radiation (Fig. 2) [1].
2.1. A Method for Obtaining the Spatial Component of Optical and Information Support
Experimental studies of the spatial structure of AB radiation consist in assessing the dependence of the spatial correlation coefficient R(n) of various forms of clouds in the horizontal directions between the rows and in the vertical directions between the columns of arrays of elements of the background-target image (BTI). A characteristic difference in the spatial structure of the radiation of different cloud classes is the size of the inhomogeneities, which were determined by the value of the spatial correlation coefficients between the rows and columns of the BTI arrays, taking a value above the level of 0.5. Hence, by the level of 0.5 of the spatial correlation coefficient R(n), knowing the step of the angular shift between the rows, it is possible to estimate the angular dimensions by the angle of the place (e), and between the columns the angular dimensions by the azimuth (b) of the AB inhomogeneities (Fig. 3) [5–7].
Thus, the obtained results of experimental studies made it possible to estimate in two directions the angular dimensions of the AB inhomogeneities, which became the basis of the spatial component of the optical and information support for the detection of the UAV image on the AB.
Analysis of the results of studies of the radiation of AB inhomogeneities (Fig. 4) made it possible to divide them into two groups depending on their angular size:
The first group includes those classes of clouds that contain small-scale inhomogeneities of magnitude 5–15° in the vertical and horizontal directions: cumulus (Cu), altocumulus (Ac), cirrocumulus (Cc) and cirrus (Ci) (Fig. 4a).
The second group includes cloud classes, which contain large-scale inhomogeneities with angular dimensions exceeding the obtained BTI, are 25–40°: stratus (St), strato-cumulus (Sc), cirro-stratus (Cs) cloud forms and clear skies (Fig. 4b) [1–4].
On the basis of the knowledge gained about the spatial spectra of inhomogeneities of AB and point images of UAVs, a spatial method for detecting UAVs on AB was developed (Fig. 2). The essence of the method consists in preliminary segmentation of the BTI before applying the threshold processing algorithm, which distinguishes it from the known methods (Fig. 2). The size of the segments is determined in accordance with the method of obtaining the spatial component of the optical and information support for detection (Fig. 5 a, b). This allows us to assume that within the angles limited by the size of the identified inhomogeneities, the random process of AB radiation can be considered stationary, since its spatial spectrum does not contain high-frequency components. In contrast, the random process of UAV radiation, the spatial spectrum always contains high-frequency components, which makes it possible to develop a decisive rule for the algorithm of threshold processing of BTI segments (Fig. 2).
Thus, the proposed spatial method for detecting UAVs on AB allows, by sequentially applying the segmentation algorithm and the algorithm for threshold processing of BTI segments, to obtain information about the presence of artifacts in the field of view of the POES, presented in binary form with the possibility of determining the coordinates of the UAV [5–7].
2.2. A Method for Obtaining
the Spatio-Temporal Component
of Optical and Information Support
The process of AB radiation is associated with thermodynamic and turbulent processes occurring in the atmosphere, and is random non-stationary both in space and time. It is known that all random processes under long-term consideration are always non-stationary by nature, but for each of them there is a limited time interval when a random process can be considered stationary. In order for the spatial optical information support within the detection method to contain stationary characteristics of the random process of AB radiation, it is necessary to periodically refine them.
Such periodicity can be estimated using a method for obtaining the spatio-temporal component of optical and information support for detecting artifacts on a complex background (Fig. 2). The basis of this method is the calculation of the coefficient of mutual correlation between the frames of the video stream arriving at regular intervals (Fig. 6a). The time interval corresponding to the level of 0.5 of the coefficient of mutual correlation between the frames of the video stream (Fig. 6 b) allows us to estimate the stationarity time – the “lifetime” of the characteristics of the spatial structure of the AB, i. e. the time interval with the periodicity of which the spatial optical and information support of detection should be updated [1–4].
2.3. Method of Obtaining the Time Component of Optical
and Information Support
In addition to the above, the question of the expediency of processing all frames of the video stream generated by the POES during processing by algorithms of the spatial method of detecting UAVs on AB remains relevant (Fig. 2). The hypothesis was put forward that the frequency of the fundamental harmonic of the spectral power density (SPD) fluctuations in the brightness of the radiation of AB inhomogeneities will be lower than the frame frequency of the POES. Therefore, it is possible to reduce the number of frames from the video stream being processed by algorithms within the spatial method of detecting UAVs on the AB.
To find the optimal frame processing frequency, a method for obtaining the time component of optical and information support was developed. The estimation of temporal variability was carried out by estimating the frequency of the main harmonic of the SPD fluctuations in the brightness of the radiation of AB inhomogeneities formed by different classes of clouds and matching with it the frame rate for processing from the video stream.
To obtain SPD estimates, fluctuations in the brightness of the radiation of AB inhomogeneities were measured in fixed directions along the angle of the place (e) in the near-horizon region with a sampling frequency a thousand times higher than the frame scan. At the same time, the azimuth value (b) remained constant for several minutes (Fig. 7a). In each direction studied, sequences of several thousand values of fluctuations in the brightness of the radiation of AB inhomogeneities for different classes and cloud scores were obtained [1–5].
The obtained arrays were subjected to statistical processing using the periodogram method of estimating the SPD. The results of statistical studies were an interval of 0.08–0.25 Hz for estimating the fundamental frequency of the harmonic of the SPD fluctuations in the brightness of the radiation of AB inhomogeneities formed by clouds of different classes and scores. Based on the results obtained, it became possible to choose the optimal frame rate from the video stream for subsequent processing by the spatial method of detecting UAVs on AB at frequencies significantly lower (up to 1 Hz) than the frame rate (50 Hz) of the video stream (Fig. 7 b).
Conclusions
Thus, the developed method of optical and information support for the detection of artifacts by a robotic system on a complex background based on the background principle of information extraction consists of three ways to obtain the necessary components of optical and information support: temporal; spatial-temporal; spatial.
The method of obtaining the time component of the optical information support makes it possible to optimize the selection of the frame rate from the video stream for subsequent processing by the spatial method of detecting the UAV on the AB.
The method of obtaining the spatio-temporal component of optical information support makes it possible to optimize the time interval with the frequency of which the segment size selection should be updated. This forms the basis of a method for obtaining the spatial component of optical and information support necessary for the implementation of a spatial method for detecting UAVs on the AB, consisting of the BTI segmentation algorithm and the threshold processing algorithm.
The use of the method of optical and information support for detection will allow the robotic system to provide information about the presence of artifacts in the field of view of the POES in the form of a binary BTI, which will further provide the ability to determine the coordinates of the UAV in space.
AUTHORS
Igor Vladimirovich Yakimenko, Doctor of Technical Sciences, Associate Professor, branch of the National Research University Moscow Power Engineering Institute in Smolensk, Smolensk, Russia.
ORCHID: 0000-0002-1003-8403
Vladimir Ivanovich Bobkov, Doctor of Technical Sciences, Associate Professor, branch of the National Research University Moscow Power Engineering Institute in Smolensk, Smolensk, Russia.
ORCHID: 0000-0002-5715-7450
Yuri Igorevich Yakimenko, post-graduate student, branch of the National Research University Moscow Power Engineering Institute in Smolensk, Smolensk, Russia.
ORCHID: 0009-0001-2631-5997
Readers feedback
