\documentstyle[twocolumn]{article} \pagestyle{empty} \setlength{\textwidth}{6.9in} \setlength{\textheight}{8.75in} \setlength{\columnsep}{2.5pc} \setlength{\topmargin}{-0.3in} \setlength{\oddsidemargin}{-.3in} \setlength{\parindent}{1pc} \makeatletter \def\@normalsize{\@setsize\normalsize{12pt}\xpt\@xpt \abovedisplayskip 11pt plus2pt minus5pt\belowdisplayskip \abovedisplayskip \abovedisplayshortskip \z@ plus3pt\belowdisplayshortskip 6pt plus3pt minus3pt\let\@listi\@listI} %the following line was changed \def\subsize{\@setsize\subsize{12pt}\xipt\@xipt} \def\section{\@startsection{section}{1}{\z@}{24pt plus 2 pt minus 2 pt} {12pt plus 2pt minus 2pt}{\large\bf}} \def\subsection{\@startsection {subsection}{2}{\z@}{12pt plus 2pt minus 2pt}{12pt plus 2pt minus 2pt}{\subsize\bf}} \makeatother \input{psfig} \begin{document} \bibliographystyle{plain} \date{} \title{\Large\bf Automated Digital Image Analysis of Video Ice Crystal Data\thanks{This work was partially supported by a NASA Ames under contract NAG 2-1104} } %\title{\Large\bf Sample Article for ISCA Format} %\author{I. M. Author \\ %My Department\\ %My Institute \\ %My City, ST, Zip} %for two authors (type the following) %\author{\begin{tabular}[t]{c@{\extracolsep{8em}}c} %r & U. R. Coauthor \\ %My Department & Coauthor Department\\ %My Institute & Coauthor Institute \\ %City, St ~~Zip & City, ST~~Zip %\end{tabular}} \author{Jane Niehues-Brooks\\ Frederick C. Harris, Jr.\\ Department of Computer Science\\ University of Nevada, \\ Reno, NV 89557\\ fredh@cs.unr.edu } \maketitle \thispagestyle{empty} \subsection*{\centering Abstract} \vspace*{-3mm} This paper presents a procedure for automating digital image analysis of cloud particle images recorded on video tape using the Cloudscope.\footnote{This instrument is operated by the Atmospheric Science Center at Desert Research Institute.} The Cloudscope is mounted on the wing of an aircraft and records cloud crystals as they impact the external lens and sublimate. The procedure presented breaks the analysis into several parts: frequency domain filtering, frame to frame movement correction, and finally new particle detection, counting, and sizing. These procedures were automated using LabVIEW and Concept V.i. \parskip=12pt \parindent=0pt {\bf keywords:} image analysis, Cloudscope, fft, ift, frequency mask, %erosion, threshold, replicator \section{Introduction} This paper discusses an analytic tool for Cloudscope ice measurement. We have implemented a technique for automating the digital image analysis of cloud particle data recorded using the Cloudscope instrument. The Cloudscope is a specialized data collection device operated for atmospheric research by the Atmospheric Science Center at Desert Research Institute. This instrument mounts on the wing of an aircraft and is essentially a microscope with an attached video camera. Ice crystals and cloud droplets impact on the small window of the Cloudscope and are recorded by the video system (Hi8 video tape). A line drawing of this instrument is shown in Figure~\ref{fig:cloudscope}~\cite{Arnott:010}. The goal of this project was to automate the counting and sizing of the cloud crystals recorded and in future work add the ability to compute their the sublimation rates. \begin {figure*} %\centerline{\psfig{figure=psfigs/Cloudscope3b.ps,width=6.0in}} \centerline{\psfig{figure=psfigs/Cloudscope3b.ps,height=1.5in}} \caption{Cloudscope Line Drawing} \label{fig:cloudscope} \end{figure*} The automated processing system is made up of a programmable Sony-Visca Hi-8 VCR and an Apple Macintosh Centris 650 with a Scion LG-3 Frame Grabber Card. The frame grabber is loaded with 64 MBytes of RAM and is capable of grabbing up to 128 frames of video, which is equivalent to 4 seconds of flight time at a 30 frames per second capture rate. We created a virtual instrument to perform the automation and image analysis using National Instrument's LabVIEW instrument control software and Graftek's Concept V.i image analysis software. A similar program for the Replicator instrument~\cite{Turner:011} was used as a guide in implementing the LabVIEW and Concept V.i tools. %Virtual instruments will be referred to as Vi's %throughout this paper. Many of the Vi's available in LabView %and Concept V.i are used as functional building blocks toward %the creation of our CloudScope Autoanalysis Vi. The CloudScope Autoanalysis Vi is also made up of several of our own more complex image analysis modules. A detailed description of these can be found in \cite{jane:mspaper}. The rest of this paper is structured as follows: Section 2 discusses the actual techniques used to perform the image processing. The Image Processing is broken down into three areas: noise removal, image alignment between frames, and new particle detection, sizing and counting. Results follow in Section 3. \section{Image Processing Algorithm} To succeed in analyzing the Cloudscope video data there are three basic tasks that have to be accomplished. These tasks include: filtering noise, correcting for image jitter, and finally detecting, counting, and sizing new particles. Once captured, each image goes through a sequence of steps to clean it up. After the image is cleaned up, a threshold is used to change it to a binary image of white particles on a black background. A comparison with the prior image is performed and new particles present are identified and sized. In principle this is a very simple process. Unfortunately, the Cloudscope experiences both electrical interference and physical instability. These issues result in flight video recordings containing excessive movements from frame to frame, color variations due to inconsistent lighting, considerable high frequency signal interference in the images, along with false particles caused by tiny scratches and foreign matter on both the CCD lens and the impact lens. The ability to successfully threshold an image into particles and background is the key to automating the counting and sizing of the crystals. Unless preprocessing is performed on each image, any of the above problems will contribute to unsuccessfully thresholding the image, thereby making it impossible to automate the particle count. For our processing purposes we use a 256 by 256 image. When captured, each image is 640 by 480, but we choose to transfer in every other line in both the X and Y direction for two reasons. The most obvious being that a smaller image will process faster, and the second, to handle a problem caused by the interlacing of the image. If a crystal impacts on the lens after the odd rows have been recorded, it will have a striped appearance due to just the even rows being recorded and would not be detected as a single particle. Using every other line of the image alleviates this problem. An adjustment to the image size must also be made. Since our Fast Fourier Transform (FFT) and Inverse Fourier Transform (IFT) routines require dimensions in powers of two, we extract a 256 by 240 portion of the image and expand the image in the Y direction from 240 to 256. This expansion causes an aspect ratio change, which we can correct for later. %\subsection{Noise Removal Method} The noise on the videos along with the wide color variations result in images that are impossible to threshold properly. Figure~\ref{fig:noisyim0}\footnote{All images are from the NASA SUCCESS Project, May 3, 1996 Flight} is an example of a noisy low contrast image. The noise in this image is more apparent when using an enhancement technique called contrast stretching. This is a method of increasing the image dynamic range by spreading the images narrow range of gray levels over the available range of 256 colors. This technique is used here for better viewing. \begin {figure}[h] \centerline{\psfig{figure=psfigs/noisyim0.ps,height=2.0in}} \caption{Unprocessed noisy image with poor contrast.} \label{fig:noisyim0} \end{figure} To investigate the noise present, we looked at the Fourier spectrum of the images in the frequency domain. The vertical lines on either side of the center line in the spectrum showed the high frequency components corrupting the image. From this it was apparent that some of the preprocessing must be performed in the frequency domain. We used an FFT to transform the image from the spatial domain to the frequency domain, and chose to use a frequency filter mask to remove the undesirable frequency components from the image. The lowest frequency color variation is due in a large part to the shape of the impact lens (a property known as vignetting~\cite{Russ:013},~page 11). Other contributors to the light variation are our artificial light source, daylight, flying into the sun, and night flights. This slowly varying color can be corrected by blocking out the lowest frequency, also known as the central--order component, located at the center of our spectrum. This technique is confirmed by discussions in ~\cite{Russ:013} pages 10 to 12 and ~\cite{Goodman:012} pages 144 to 145. By placing a horizontal center line in the mask we can remove the vertical wave-like ridges that traverse horizontally from left to right across the image. This filtering method was adapted from the optical grid example in~\cite{Goodman:012} pages 142--144, where a vertical slit passes only the vertical components of the grid and a horizontal slit passes only the horizontal components. The resulting high contrast image with the corrupting noise removed is given in Figure~\ref{fig:processed0}. \begin {figure}[h] \centerline{\psfig{figure=psfigs/processed0.ps,height=2.0in}} %\caption{High contrast image after processing (image 0).} \caption{High contrast image after processing.} \label{fig:processed0} \end{figure} % \subsection{Image Alignment Technique} Another hurdle to overcome before any reliable data analysis can be performed is to correct for the image movement from frame to frame. Plane vibration from turbulence and electro-mechanical problems are more than likely the cause of this jitter. Image movement in this section of our video tape has been measured in excess of +/- 14 pixels. This is considerable since it lies within the size range of the crystals we hope to measure. In order to successfully detect a new particle, the two consecutive images must have the same reference point. Several techniques for aligning the images were tried with various levels of success, such as: using the first image as a reference and aligning all following images relative to a fixed particle on that first image; selecting a target particle that is present on two consecutive images and aligning them based on the difference in the center of mass coordinates of the two matched particles; and performing a cross-correlation between the two images (the discrete version of this computation is presented in~\cite{Gonzalez:009}). In aligning the two images, the prior image is held fixed and the current image is realigned by extracting all or a portion of the image based on the offset values, and then pasting it back onto a white background. A white background is used so that any particle lying on, or touching the boarder, will still be considered a boarder particle when the image is aligned. Figure~\ref{fig:im2aligned} shows the result of aligning image 2 with image 1. A correction of +3 in the X--direction and +12 in the Y--direction is made. For most particles in the image, this is a fairly accurate correction, but this method is not without exceptions. There are always a few specks or scratches present on the CCD lens that show up in the images. Because of their physical location, they do not jump from frame to frame as do the particles on the impact lens. This introduces an error back into the analysis because these particles have just been misaligned causing them to be counted as new particles. It is possible that as a second stage to this procedure, we could reverse the alignment and subtract out the CCD lens noise and hope that no valid data gets removed. An alternate plan would be to leave it to some post processing number crunching where erroneous data can be identified and deleted from the data set. \begin {figure} \centerline{\psfig{figure=psfigs/borderim2align2.ps,height=2.0in}} \caption{Binary version of image 2 after alignment.} \label{fig:im2aligned} \end{figure} % \subsection{New Particle Detection and Counting} Given two clean aligned consecutive images, we are then able to count and size the particles. When counting, we are only concerned with the new particles in an image. A new particle is one that was not present on the previous frame. The size of a particle is the area it occupies within the image. %Figure~\ref{fig:nomedian1} is an example of a filtered image %after thresholding. After thresholding this image is still not as clean as we would like so we add a median filtering step. This is a spatial convolution process in which we keep the median value of all neighboring pixels. For a particle to be counted it must be surrounded by black background. If a particle touches or overlaps the image border, it is considered outside the field of view. Since its actual size is not known, it is removed from the image. Notice that both of the particles located at the middle of the right border of Figure~\ref{fig:im2aligned} will be removed because they are touching each other and the border. In this case it is debatable whether all border particles should be rejected. Future advancements in processing techniques may minimize the loss of valid data such as this. Finally, the {\em new} particles can be counted and sized. The size of a particle is based on the number of pixels that make up its contiguous white area. The parameters selected for our analysis include; area, max intercept length, mean perpendicular intercept, perimeter, center of mass X and Y coordinates, and the particles bounding box coordinates. Once the measurement is complete, the data is written to a file after the set of frames have been analyzed. This data will then be used to compute ice crystal concentrations The actual dimensions of the 640 by 480 pixel image is 420 $\mu$m by 314 $\mu$m. Correcting for expansion of the original image in the Y dimension and the extraction of only 256 pixels of the 320 in the X dimension our actual per pixel dimension in the X dimension is 1.313 $\mu$m and 1.227 $\mu$m in the Y dimension. Our smallest particle size of three by three pixels square (3.94$\mu$m x 3.68$\mu$m) is 14.5 $\mu$$m^{2}$. Realistically, only those particles with an area of more than fifty pixels (~80 $\mu$$m^{2}$) should be counted as real with any accuracy. For our measurements we have chosen to count all particles remaining after erosion and post filter the data based on size once it's been stored to a file. \section{Results} The time to process the ten frames of data is approximately 6.5 minutes, and to process a maximum of 128 frames (4 seconds of flight) is 87 minutes which includes the time to initialize and and grab the frames. This time will increase in future runs when we use the whole 320 by 240 image expanded to 512 by 256 or a 640 by 480 contracted and expanded to 512 by 512. Ideally, we would have preferred a faster process time, but due to the compute time necessary for the FFT and IFT, this was not possible. However, considering the alternative of a manual count, which for a dense particle concentration would be extremely tedious and time consuming, time is not the main issue. There are some physical properties of the CloudScope data that we may never be able to handle. One example of this would be: a particle landing on its edge in one frame then falling over in the next frame. Even to the human eye this appears to be two particles and as expected our program will count it as two particles. Crystals overlapping either partially or completely is another uncontrollable occurrence. With our current program version this will be counted as one large particle if they both landed at the same time. If on the other hand, as discussed in the previous section, they touched down on two different frames the second particle may be detected as a much smaller particle, as two separate smaller particles, or as a particle with a whole or cutout in it. Being able to detect and size very small ice crystals is of great interest to the atmospheric scientists who want to use this kind of analysis tool. We foresee this as an on going project of analysis enhancements and improved capabilities. %One of the areas of future work is to compute the ice crystal %sublimation rate. %The sublimation rate is of interest since it can be used in %subsequent studies as a means of computing the particle's %mass. To compute a sublimation rate, a crystal must be %identified and its data parameters recorded for each frame. A %reduction in particle size is expected from frame to frame. %Once the particle has been tracked for a specified period of %time, or once it has lost a certain amount of area, the %difference in area from the first frame to the last frame will %be recorded along with the number of elapsed frames. With this %information we can compute a reasonable sublimation rate. \baselineskip=12pt \addcontentsline{toc}{section}{References} \bibliography{jane} \end{document}