rus
Issue #6/2014
M. Anikeeva, A. Sofronov, S. Dremov¬, A. Ter-Martirosyan
Digital Imaging System For Photodynamic Therapy
Digital Imaging System For Photodynamic Therapy
A system for visualization of spatial distribution of fluorescence during photodynamic therapy procedures is presented. The system successfully passed out clinical trials.
Теги: object-oriented architecture photodynamic therapy visulisation визуализация объектно-ориентированная архитектура фотодинамическая терапия
P
hotodynamic therapy (PDT) represents the method of diagnostics and treatment of malignant cancer tumors based on the capability of some preparations (photosensitizers) to accumulate selectively in the diseased tissues and interact simultaneously with the electromagnetic radiation of visible or near infrared band. The local light activation of the photosensitizer which was accumulated in the tumor from the band of its absorption in the presence of tissue oxygen causes the development of photochemical reaction which destroys the tumor cells and at its expense the therapeutic effect can be achieved. The process is accompanied by fluorescence of the preparation and oxygen as well in longer wavelength infrared region of spectrum. The image of spatial distribution of fluorescence gives information about the tumor size and preparation concentration and its variation in time can give the quantitative information about the required radiation dose. This effect can be used not only for the treatment but diagnostics of oncologic diseases with various localizations at early stages, as well [1].
The recent achievements in the area of semiconductor optoelectronics make the PDT method available for widespread use in the medical institutions due to the development of inexpensive compact high-power sources of laser radiation on the basis of semiconductor laser diodes [2]. However, in order to obtain and visualize the image of spatial distribution of photosensitizer fluorescence the spectral-selective system of registration of optical radiation, which permits the light to go through the whole fluorescence band and is not transparent for the radiation exciting fluorescence, is needed. Currently, in the existing PDT devices glasses or analog video-cameras with the optical interference filters serve in the capacity of such systems. Such approach does not only make it impossible to estimate the tumor parameters quantitatively but also it does not provide the potential usability of PDT systems.
All these aspects speak about the necessity to develop the digital system of photosensitizers fluorescence imaging which will allow using all potential capabilities of photodynamic therapy method in full and perform imaging in the most convenient form and automatic quantitative analysis of fluorescence image on real-time basis.
The software, which controls the system photosensitive element and processes the incoming data, is the basis of such system.
Analysis of Requirements for the Software of Photodynamic Therapy Systems
One of the key aspects concerning the development of software for the systems of photodynamic therapy includes the form of visualization of received information which will be the most convenient for end user. Analysis of requests of practicing physicians showed that the "classic” conception of the image form in tints of grey color received directly from the system camera through the spectral-selective optics is insufficiently convenient for the visual perception and not sufficient for the analysis of visual information. The best imaging form turned out to be the pseudo-threedimensional histogram of brightness distribution in two coordinates which is displayed near the actual image received from camera (Fig. 1). And the actual image itself can be displayed in the initial form in tints of grey color and in pseudo-colors according to the determined adopted color chart.
For the additional visualization of results it is also required to define the contours of the constant value of brightness on the obtained images on the basis of determined set level which actually represent the boundaries of tumor.
Besides the conception of fluorescence intensity distribution on real-time basis, there should be also ability to freeze the images at certain points of time for the further comparison and analysis.
The following aspect which requires the special attention is the graphic interface of software user. Taking into account the specificity of software end users, the interface should be simple, convenient and provide minimum user actions and allow easy change of form and displaying of visualized information.
Hardware Platform
The system hardware, in which the software functioning is located, consists of the standard personal computer with the connected digital camera on the basis of any sufficiently-fast interface (USB2.0, IEEE1394, Ethernet). The spectral-selective optical filter, which permits only radiation in the fluorescence band of used photosensitizer to pass through, is installed on the lens of digital camera. Thus, the video frames which arrive from the camera are the initial data for imaging.
The hardware block diagram is shown in Fig. 2. Analog video signal from photosensitive array is digitized by the digital processing unit which is also responsible for the information transmission to computer. These two units are integrated in the digital camera. Also, the emitter which excites the fluorescence, power-generating unit and some auxiliary components are included into the device structure.
Software Object Model
Following [3], the construction of object-oriented architecture of developed software consists in the building abstractions for the system individual elements, determination of responsibility of every abstraction and relationship between them. The system finished object model is shown in Fig. 2 in the form of combined diagram of classes and components.
The class frameInput is the program abstraction for the source of initial frames. Besides the obvious ability to return the next frame of video stream, the class duties also include furnishing of information on the frame parameters (dimensions, bit count etc.). The class hides in itself the specific details of hardware interface of camera connection to the computer and the specific implementation of the class is performed with the help of proprietary SDK of camera or through DirectShow. In the last case the open-ended library videoInput [4], which is the object-oriented wrapper above DirectShow, can be used.
The classes VideoOutput and ChartOutput refer to the abstractions for the information output devices in the form of bitmap and pseudo-threedimensional histogram respectively. Duties of the classes include the display of graphic information in the form shown in Fig. 1. The specific implementation of classes is accomplished with the use of the library OpenGL, which has the hardware implementation at the majority of video cards, and use of OpenGL considerably accelerates the graphics output in comparison with the means of Windows GDI. Both classes are successors of the abstract class GLWidget intended for the encapsulation of functions connected with design and maintenance of the window of user graphic interface into which the output by the means of OpenGL is performed.
The class Analyzer encapsulates the function connected with the analysis of image frame, search of the contours of constant value and calculation of data for the construction of surface chart. The class has methods for the transfer of frame analysis parameters (thresholds of contours etc.) and main method of processing which in arguments receives the pointer in frame input unit frameInput and pointers in output units VideoOutput and ChartOutput. The main function of the class is to determine the boundaries of tissue diseased site, calculate its area and prepare data for the display in output units and update data. The specific implementation of class is based on the use of vision algorithms of the library OpenCV. In particular, the topological structural analysis of frame is performed for the search of exhaustive set of contours of constant value [5].
The Class mainWindow encapsulates the functions connected with the creation of user graphic interface and plays the role of "sampler” activating the main processing method of Analyzer class by timer. The specific implementation is based on cross-platform tools Qt (versions licensed according to GNU LGPL). This is the most convenient and well-documented method of quick creation of graphic interface. In comparison with the set of classes MFC from Microsoft, Qt offers completely correct object-oriented approach to the programming with the active use of class hierarchy which simplifies and accelerates development providing higher level of abstraction.
Complete model contains a number of additional classes. In particular, classes Contours for the storage of contours of constant value and ImgStack averaging some quantity of sequenced frames for the reduction of noise level, which were constructed according to the principle of linked list, and class DimDialog responsible for the output of elements of graphic user interface which are necessary for the calibration of actual dimensions of objects. All program dependences on external libraries are shown in Fig. 3.
Clinical Testing
On the basis of the developed model the specific implementation of the prototype of fluorescence imaging system – device "Fluovizor” was created. The digital monochromic camera with the resolution 752×582 pixels using the interface USB2.0 is applied in the device. The laptop of the lowest price segment with the dual core processor of series Intel Celeron B800 was used in the capacity of personal computer. Application of the described approaches in the software development made it possible for the system to operate in the mode of "real time” (practically, under the conditions of images update at the screen with the frequency of 30 fps) even at such hardware platform.
The prototype underwent the clinical tests in the Research Institute of Oncology named after N. N. Petrov where it was used for the imaging during diagnostics and treatment of basal-cell skin cancer by PTD method upon the irradiation of tumor by semiconductor laser with the radiation wavelength λ = 665 nm. Fotoditazin was used in the capacity of photosensitizer.
The actual image of fluorescence distribution is shown in Fig. 4. The image of spatial distribution of fluorescence in pseudo-color is shown in Fig. 5. And the tumor is highlighted in red color providing the clear visualization of tumor boundaries and their variation with the time in the process of PDT session.
Visualization of the area of full frame with the diseased site in the form of pseudo-threedimensional distribution of fluorescence intensity in coordinates is given in Fig. 6. Also the detected boundaries of tumor highlighted in red line at initial image and white line at threedimensional picture is shown.
Given examples illustrate the capabilities concerning the interpretation of actual images which is provided to user by the developed software.
In particular, such form of data representation visually demonstrates the distribution of photosensitizer concentration and its dynamic variation in time, allows ensuring the tissue irradiation by laser with more accurate dose and direction for the optimal accomplishment of the treatment procedure using PDT methods.
Conclusion
As a result of the works execution, the software for imaging system of fluorescence spatial distribution during diagnostics and treatment using the methods of photodynamic therapy has been created. Object-oriented architecture of the imaging system has been developed; selection of the optimal variant of specific implementation with the active use of third-party libraries with open source code has been performed. The prototype of imaging system including hardware and software which successfully underwent clinical testing has been fabricated.
By now, in LLC "Atkus" the line of medical laser apparatuses of "Latus" series has been developed and successfully implemented in batch production. Apparatuses are characterized by the unique scheme of combining of semiconductor lasers radiation, up-to-date elemental base, and use of high-performance high-power laser diodes. Besides, LLC "Atkus" offers the system of photosensitizer fluorescence imaging with the commercial name "Fluovizor”.
The work was executed within the framework of the project on development of cooperation of the Russian higher educational institutions and industrial enterprises (Agreement No. 13.G25.31.0055). According to the work results, the application for the patent on the device for photosensitizer fluorescence imaging during diagnostics and treatment using PDT method is submitted. Authors express their gratitude to M. L. Gelfond for the assistance in the system clinical testing.
hotodynamic therapy (PDT) represents the method of diagnostics and treatment of malignant cancer tumors based on the capability of some preparations (photosensitizers) to accumulate selectively in the diseased tissues and interact simultaneously with the electromagnetic radiation of visible or near infrared band. The local light activation of the photosensitizer which was accumulated in the tumor from the band of its absorption in the presence of tissue oxygen causes the development of photochemical reaction which destroys the tumor cells and at its expense the therapeutic effect can be achieved. The process is accompanied by fluorescence of the preparation and oxygen as well in longer wavelength infrared region of spectrum. The image of spatial distribution of fluorescence gives information about the tumor size and preparation concentration and its variation in time can give the quantitative information about the required radiation dose. This effect can be used not only for the treatment but diagnostics of oncologic diseases with various localizations at early stages, as well [1].
The recent achievements in the area of semiconductor optoelectronics make the PDT method available for widespread use in the medical institutions due to the development of inexpensive compact high-power sources of laser radiation on the basis of semiconductor laser diodes [2]. However, in order to obtain and visualize the image of spatial distribution of photosensitizer fluorescence the spectral-selective system of registration of optical radiation, which permits the light to go through the whole fluorescence band and is not transparent for the radiation exciting fluorescence, is needed. Currently, in the existing PDT devices glasses or analog video-cameras with the optical interference filters serve in the capacity of such systems. Such approach does not only make it impossible to estimate the tumor parameters quantitatively but also it does not provide the potential usability of PDT systems.
All these aspects speak about the necessity to develop the digital system of photosensitizers fluorescence imaging which will allow using all potential capabilities of photodynamic therapy method in full and perform imaging in the most convenient form and automatic quantitative analysis of fluorescence image on real-time basis.
The software, which controls the system photosensitive element and processes the incoming data, is the basis of such system.
Analysis of Requirements for the Software of Photodynamic Therapy Systems
One of the key aspects concerning the development of software for the systems of photodynamic therapy includes the form of visualization of received information which will be the most convenient for end user. Analysis of requests of practicing physicians showed that the "classic” conception of the image form in tints of grey color received directly from the system camera through the spectral-selective optics is insufficiently convenient for the visual perception and not sufficient for the analysis of visual information. The best imaging form turned out to be the pseudo-threedimensional histogram of brightness distribution in two coordinates which is displayed near the actual image received from camera (Fig. 1). And the actual image itself can be displayed in the initial form in tints of grey color and in pseudo-colors according to the determined adopted color chart.
For the additional visualization of results it is also required to define the contours of the constant value of brightness on the obtained images on the basis of determined set level which actually represent the boundaries of tumor.
Besides the conception of fluorescence intensity distribution on real-time basis, there should be also ability to freeze the images at certain points of time for the further comparison and analysis.
The following aspect which requires the special attention is the graphic interface of software user. Taking into account the specificity of software end users, the interface should be simple, convenient and provide minimum user actions and allow easy change of form and displaying of visualized information.
Hardware Platform
The system hardware, in which the software functioning is located, consists of the standard personal computer with the connected digital camera on the basis of any sufficiently-fast interface (USB2.0, IEEE1394, Ethernet). The spectral-selective optical filter, which permits only radiation in the fluorescence band of used photosensitizer to pass through, is installed on the lens of digital camera. Thus, the video frames which arrive from the camera are the initial data for imaging.
The hardware block diagram is shown in Fig. 2. Analog video signal from photosensitive array is digitized by the digital processing unit which is also responsible for the information transmission to computer. These two units are integrated in the digital camera. Also, the emitter which excites the fluorescence, power-generating unit and some auxiliary components are included into the device structure.
Software Object Model
Following [3], the construction of object-oriented architecture of developed software consists in the building abstractions for the system individual elements, determination of responsibility of every abstraction and relationship between them. The system finished object model is shown in Fig. 2 in the form of combined diagram of classes and components.
The class frameInput is the program abstraction for the source of initial frames. Besides the obvious ability to return the next frame of video stream, the class duties also include furnishing of information on the frame parameters (dimensions, bit count etc.). The class hides in itself the specific details of hardware interface of camera connection to the computer and the specific implementation of the class is performed with the help of proprietary SDK of camera or through DirectShow. In the last case the open-ended library videoInput [4], which is the object-oriented wrapper above DirectShow, can be used.
The classes VideoOutput and ChartOutput refer to the abstractions for the information output devices in the form of bitmap and pseudo-threedimensional histogram respectively. Duties of the classes include the display of graphic information in the form shown in Fig. 1. The specific implementation of classes is accomplished with the use of the library OpenGL, which has the hardware implementation at the majority of video cards, and use of OpenGL considerably accelerates the graphics output in comparison with the means of Windows GDI. Both classes are successors of the abstract class GLWidget intended for the encapsulation of functions connected with design and maintenance of the window of user graphic interface into which the output by the means of OpenGL is performed.
The class Analyzer encapsulates the function connected with the analysis of image frame, search of the contours of constant value and calculation of data for the construction of surface chart. The class has methods for the transfer of frame analysis parameters (thresholds of contours etc.) and main method of processing which in arguments receives the pointer in frame input unit frameInput and pointers in output units VideoOutput and ChartOutput. The main function of the class is to determine the boundaries of tissue diseased site, calculate its area and prepare data for the display in output units and update data. The specific implementation of class is based on the use of vision algorithms of the library OpenCV. In particular, the topological structural analysis of frame is performed for the search of exhaustive set of contours of constant value [5].
The Class mainWindow encapsulates the functions connected with the creation of user graphic interface and plays the role of "sampler” activating the main processing method of Analyzer class by timer. The specific implementation is based on cross-platform tools Qt (versions licensed according to GNU LGPL). This is the most convenient and well-documented method of quick creation of graphic interface. In comparison with the set of classes MFC from Microsoft, Qt offers completely correct object-oriented approach to the programming with the active use of class hierarchy which simplifies and accelerates development providing higher level of abstraction.
Complete model contains a number of additional classes. In particular, classes Contours for the storage of contours of constant value and ImgStack averaging some quantity of sequenced frames for the reduction of noise level, which were constructed according to the principle of linked list, and class DimDialog responsible for the output of elements of graphic user interface which are necessary for the calibration of actual dimensions of objects. All program dependences on external libraries are shown in Fig. 3.
Clinical Testing
On the basis of the developed model the specific implementation of the prototype of fluorescence imaging system – device "Fluovizor” was created. The digital monochromic camera with the resolution 752×582 pixels using the interface USB2.0 is applied in the device. The laptop of the lowest price segment with the dual core processor of series Intel Celeron B800 was used in the capacity of personal computer. Application of the described approaches in the software development made it possible for the system to operate in the mode of "real time” (practically, under the conditions of images update at the screen with the frequency of 30 fps) even at such hardware platform.
The prototype underwent the clinical tests in the Research Institute of Oncology named after N. N. Petrov where it was used for the imaging during diagnostics and treatment of basal-cell skin cancer by PTD method upon the irradiation of tumor by semiconductor laser with the radiation wavelength λ = 665 nm. Fotoditazin was used in the capacity of photosensitizer.
The actual image of fluorescence distribution is shown in Fig. 4. The image of spatial distribution of fluorescence in pseudo-color is shown in Fig. 5. And the tumor is highlighted in red color providing the clear visualization of tumor boundaries and their variation with the time in the process of PDT session.
Visualization of the area of full frame with the diseased site in the form of pseudo-threedimensional distribution of fluorescence intensity in coordinates is given in Fig. 6. Also the detected boundaries of tumor highlighted in red line at initial image and white line at threedimensional picture is shown.
Given examples illustrate the capabilities concerning the interpretation of actual images which is provided to user by the developed software.
In particular, such form of data representation visually demonstrates the distribution of photosensitizer concentration and its dynamic variation in time, allows ensuring the tissue irradiation by laser with more accurate dose and direction for the optimal accomplishment of the treatment procedure using PDT methods.
Conclusion
As a result of the works execution, the software for imaging system of fluorescence spatial distribution during diagnostics and treatment using the methods of photodynamic therapy has been created. Object-oriented architecture of the imaging system has been developed; selection of the optimal variant of specific implementation with the active use of third-party libraries with open source code has been performed. The prototype of imaging system including hardware and software which successfully underwent clinical testing has been fabricated.
By now, in LLC "Atkus" the line of medical laser apparatuses of "Latus" series has been developed and successfully implemented in batch production. Apparatuses are characterized by the unique scheme of combining of semiconductor lasers radiation, up-to-date elemental base, and use of high-performance high-power laser diodes. Besides, LLC "Atkus" offers the system of photosensitizer fluorescence imaging with the commercial name "Fluovizor”.
The work was executed within the framework of the project on development of cooperation of the Russian higher educational institutions and industrial enterprises (Agreement No. 13.G25.31.0055). According to the work results, the application for the patent on the device for photosensitizer fluorescence imaging during diagnostics and treatment using PDT method is submitted. Authors express their gratitude to M. L. Gelfond for the assistance in the system clinical testing.
Readers feedback
