Methods of manual cell localization and outlining are so onerous that automated tracking methods would seem mandatory for handling huge image sequences, nevertheless manual tracking is, astonishingly, still widely practiced in areas such as cell biology which are outside the influence of most image processing research. The goal of our research is to address this gap by developing automated methods of cell tracking, localization, and segmentation. Since even an optimal frame-to-frame association method cannot compensate and recover from poor detection, it is clear that the quality of cell tracking depends on the quality of cell detection within each frame.
Cell detection performs poorly where the background is not uniform and includes temporal illumination variations, spatial non-uniformities, and stationary objects such as well boundaries (which confine the cells under study). To improve cell detection, the signal to noise ratio of the input image can be increased via accurate background estimation. In this paper we investigate background estimation, for the purpose of cell detection. We propose a cell model and a method for background estimation, driven by the proposed cell model, such that well structure can be identified, and explicitly rejected, when estimating the background.
The resulting background-removed images have fewer artifacts and allow cells to be localized and detected more reliably. The experimental results generated by applying the proposed method to different Hematopoietic Stem Cell (HSC) image sequences are quite promising.
The understanding of cell behavior relies on precise information about the temporal dynamics and spatial distribution of cells. Such information may play a key role in disease research and regenerative medicine, so automated methods for observation and measurement of cells from microscopic images are in high demand. The proposed method in this paper is capable of localizing single cells in microwells and can be adapted for the other cell types that may not have circular shape. This method can be potentially used for single cell analysis to study the temporal dynamics of cells.
The automated acquisition of huge numbers of digital images has been made possible due to advances in and the low cost of digital imaging. In many video analysis applications, the goal is the tracking of one or more moving objects over time such as human tracking, traffic control, medical and biological imaging, living cell tracking, forensic imaging, and security [1–7].
The possibility of image acquisition and storage has opened new research directions in cell biology, tracking cell behaviour, growth, and stem cell differentiation. The key impediment on the data processing side is that manual methods are, astonishingly, still widely practiced in areas such as cell biology which are outside the influence of most image processing research. The goal of our research, in general, is to address this gap by developing automated methods of cell tracking.
Although most televised video involves frequent scene cuts and camera motion, a great deal of imaging, such as medical and biological imaging, is based on a fixed camera which yields a static background and a dynamic foreground. Moreover, in most tracking problems it is the dynamic foreground that is of interest, hence an accurate estimation of the background is desired which, once removed, ideally leaves us with the foreground on a plain background. The estimated background may be composed of one or more of random noise, temporal illumination variations, spatial distortions caused by CCD camera pixel non-uniformities, and stationary or quasi-stationary background structures.
We are interested in the localization, tracking, and segmentation of Hematopoietic Stem Cells (HSCs) in culture to analyze stem-cell behavior and infer cell features. In our previous work we addressed cell detection/localization [8, 9] and the association of detected cells . In this paper cell detection and background estimation will be studied, with an interest in their mutual inter-relationship, so that by improving the performance of the background estimation we can improve the performance of the cell detection. The proposed approach contains a cell model and a point-wise background estimation algorithm for cell detection. We show that point-wise background estimation can improve cell detection.
There are different methods for background modelling, each of which employs a different method to estimate the background based on the application at hand, specifies relevant constraints to the problem, and makes different assumptions about the image features at each pixel, processing pixel values spatially, temporally, or spatio-temporally [11–23].
There is a broad range of biomedical applications of background estimation, each of which introducing a different method to estimate the background based on some specific assumptions relevant to the problem [12–14, 24]. Close and Whiting  introduced a technique for motion compensation in coronary angiogram images to distinguish the arteries and background contributions to the intensity. They modelled the image in a region of interest as the sum of two independently moving layers, one consisting of the background structure and the other consisting of the arteries. The density of each layer varies only by rigid translation from frame to frame and the sum of two densities is equal to the image density.
Boutenko et. al
 assumed that the structures of interest are darker than the surrounding immobile background and used a velocity based segmentation to discriminate vessels and background in X-ray cardio-angiography images, considering the faster vessel motion in comparison with the background motion.
Chen et. al
 modelled the background of a given region of interest using the temporal dynamics of its pixels in quantitative fluorescence imaging of bulk stained tissue. They modelled the intensity dynamics of individual pixels of a region of interest and derived a statistical algorithm to minimize background and noise to decompose the fluorescent intensity of each pixel to background and the stained tissue contributions.
A simulation and analysis framework to study membrane trafficking in fluorescence video microscopy was proposed by Boulanger et. al
. They designed time-varying background models in fluorescence images and proposed statistical methods for estimating the model parameters. This method decides whether any image point belongs to the image background or a moving object.
Several segmentation and tracking methods are proposed for a broad range of biomedical applications, each of which introducing a different method to segment and/or track specific biological materials based on some specific assumptions relevant to the problem [25–28].
Cheng et. al used shape markers to separate clustered nuclei from fluorescence microscopy cellular images in a watershed-like algorithm . Shape markers were extracted using H-minima transform. A marking function was introduced to separate clustered nuclei while geometric active contour was used for initial segmentation.
Gudla et. al proposed a region growing method for segmentation of clustered and isolated nuclei in fluorescence images . They used a wavelet-based approach and a multi-scale entropy-based thresholding for contrast enhancement. They first oversegmented nuclei and then merged the neighboring regions into single nuclei or clustered nuclei based on area followed by automatic multistage classification.
A semi-automatic mean-shift-based method for tracking of migrating cell trajectories in vitro phase-contrast video microscopy was proposed by Debeir et. al
. They used mean-shift principles and adaptive combinations of linked kernels in the proposed method. They used this method for detection of different gray-level configurations. This method required manual initialization of the cell centroids on the first frame, it did not use temporal filtering or time-dependent feature, and it did not provide precise information on the cell boundaries and shapes.
Most tracking problems have an implicit, nonparametric model of the background to avoid making assumptions regarding the foreground. By developing a model for the background it is possible to find a classifier that labels each image pixel as background/not background; i.e., the foreground is identified as that which is not background. In contrast, the more focussed context of our cell tracking problem admits an explicit model of the foreground. Because of the low SNR of our problem, where illumination is limited to minimize cell phototoxicity, it is desired to remove all deterministic non-cell variations in the image (i.e., the background) before localizing the cells.
Some of the earlier works have integrated foreground detection and background estimation in a mutual framework, however most of the previous methods classify each pixel to either foreground or background, where their goal is the general segmentation of dynamic objects with no assumptions regarding the foreground. In contrast our goal is the localization of foreground objects, given specific assumptions that are integrated in the form of a foreground model.
In our proposed method, in place of classifying each pixel to either foreground or background, we estimate a single global background and do detection of foreground objects (but not pixel by pixel). Our proposed method addresses foreground detection and background estimation as inter-related processes, and take advantage of this inter-relation to improve the performance of cell detection. In the proposed algorithm, the background elements are removed from the scene frame by frame using a spatio-temporal background estimator while a probabilistic cell model is applied to the image sequence to localize cell centers. The spatio-temporal estimator has been applied to estimate the background in phase contrast image sequences taken from living Hematopoietic (blood) Stem Cells in culture, and leads to substantial improvements in cell localization and cell outline detection.
To produce the data for this study, HSC samples are first extracted from mouse bone marrow, then cultured in custom arrays of microwells. The cells were imaged using manual focusing through a 5× phase contrast objective using a digital camera (Sony XCD-900) and acquired by an IEEE 1394 standard (FireWire) connector. Images were sampled every three minutes over the course of several days. During imaging, chambers were maintained at 37°C, in a 5% CO2 humidified air environment.
Two original frames taken from a cropped well is depicted in Fig. 1(a-i) and 1(a-ii). Well cropping is often approximate and the well boundaries may be partially or completely visible in the cropped image sequence, as can be seen in Fig. 1(b-i). Modelling cells on a uniform, zero-mean background requires that any existing background be estimated and subtracted.
A probabilistic cell model is proposed as the product of two probabilistic terms associated with the cell brightness and the surrounding background. We also propose a background estimation method which is driven by the proposed cell model to identify and reject the well structures. In this section the background estimation method is first presented followed by the proposed probabilistic cell model.
Point-wise Background Estimation for Cell Detection
To estimate the background, let image Ikbe defined on a fixed lattice L:
thus for each frame Ikof an image sequence we can write
where Fkis the dynamic foreground, B is the fixed background, nkmodels the temporal variations in global lighting, and Vkis spatio-temporal random additive noise. The temporal noise nkis estimated over all pixels in each frame k
For temporal correction, the estimated temporal noise is subtracted from the original frame Ik:
The true background B = [Bij] is composed of stationary distortions and illumination variations at each pixel location. The point-wise estimation of the background can be estimated over K frames of temporal corrected sequence g = (g1, g2,..., gK):
and is subtracted out from g:
An imperfectly cropped well, the corresponding estimated background, and the corrected image are depicted in Figs. 1(b-i), (b-ii), and 1(b-iii) respectively.
Justification of mode as statistical measure
Empirically, the motion of blood stem cells is essentially random, especially when observed minutes apart. Since cell motion is rarely zero, the spatial variations in cell brightness, mean that the variability of an image pixel, located within a cell, is considerably higher than the variability of an image pixel lieing in the background, whose variability is due only to random noise. Therefore, excepting cases of unusually small cell motion, the distribution of brightness values at a pixel should be most sharply peaked at the background, which is therefore recovered by the mode of the sample histogram.
Cell Detection in a Uniform Background
Let I = (I1, I2,..., IK) be a set of K images which we will assume to consist of cells on a plain, uniform background. A typical microscopic multi-well image sequence I in our experiments consists of 32 separated wells, in each of which two to four HSCs are injected. Single-well image sequences are cropped from the original multi-well image sequence and are processed individually. Frames 1 and 50 of a typical cropped well are depicted in Fig. 1(a).
In our previously designed  cell detection method, stem-cell center locations were inferred from an image Ikby
where represents a cell with radius located at coordinate , and where Pcb, Pinand Pcdfcharacterize the bright cell boundary, the dark cell interior, and the boundary uniformity, respectively.
As we can see in Fig. 2(a), all HSCs cannot, in fact, be characterized by model (8) as a well recognizable dark interior and a bright boundary and the previous method in  performs poorly to detect them.
Different HSC phenotypes can be characterized as approximately circular objects with high intensity variations against the background, as can be observed in Fig. 2(a). Assuming a uniform background, we designed a general cell detection model in our previous work . This method performed well to detect different HSC phenotypes, however it suffered from discontinuity and spurious detection. Moreover the gray level image was converted to a binary image using Otsu's thresholding at the expense of losing gray level features that could be used to improve cell detection.
Therefore to model the HSCs we propose an improved model, characterized by the following criteria:
The cell is round, with some radius.
The intensities of cell pixels deviate (both brighter and darker) from those of background pixels.
Most of the pixels around the cell are near the background mean.
We propose a revised probabilistic cell model as the product of two probabilistic terms Pcelland Pbackwhich are associated with the cell brightness and the surrounding background respectively:
This revised model (8) is similar in sprit to , but generalized to different HSC phenotypes investigated in this work. The description and computation of each probabilistic term follows.
Cell Probability as a Circular Anomaly
A cell is modelled as a circular anomaly, darker or brighter than the uniform background. Let return a circular set of pixels
The cell model assumes the background to have been subtracted, therefore the background mean is zero, and deviations from zero suggest the presence of a cell. Thus we extract the mean-square intensities
The cell probability Pcellis proposed, based on observations, to be an exponential
so that Pcellhas a strong response for cell pixels, and weak (close to zero) for background pixels.
Penalizing False Candidates
Except for cells which are very tightly packed, most cells will be surrounded by background. Therefore to distinguish a cell center from a point between two adjacent cells, we can test for the presence of background pixels in a ring around the cell, between radii of r and 2r:
Not all of the pixels surrounding a cell are necessarily background, but we do expect at least half of them as background, and penalize the mean-square deviation from zero of this fraction. Let be the half subset of E with intensities closest to the background mean of zero:
We then calculate the mean square
We assume that we have an image sequence with a zero-mean background plus additive noise which, if Gaussian, leads to be χ2 when contains mostly background, so the cell/background hypothesis test can be approximated by a simple exponential:
Locating The Cell Centers
To locate the cell centers, we compute the probability map by applying the cell model in (8) to image frame Ik, and then find the cell centers as the thresholded local maxima in .
The threshold is computed analogously to , in which the threshold is varied and selected to minimize the sum of missed detections and false alarms. For the proposed cell metric (8), a threshold of 0.25 was found to be very effective, giving a detection rate over 95%, and a false alarm rate of approximately 6%. Fig. 2(b) shows the application of the cell model to a cropped well interior with no boundaries (Fig. 2(b-i)) and a coarsely cropped well before and after background correction is depicted in Figs. 3(a-i) and 3(a-iii) respectively. As it can be observed, the cell model performs very poorly (Fig. 3(a-ii)) where the cropped well contains visible well boundaries, however the cell model performs perfectly where it is applied to a cropped well interior with no boundaries (Fig. 2(b-ii)) or a background corrected coarsely cropped well (Fig. 3(a-iv)).
We have applied the proposed cell model and background estimation method to different sequences of phase contrast HSC images. The entire proposed method, consisting of point-wise background estimation for cell detection, and cell detection based on a probabilistic model is applied to a whole, imperfectly cropped well as depicted in Fig. 3(b). Point-wise background removal is illustrated in Fig. 3(b-ii) while cell-center detection is shown in Fig. 3(b-iii). Fig. 4 shows the estimated background and the corrected image frames for different HSC phenotypes under consideration applying the point-wise background estimation/subtraction.
The results obtained by the proposed point-wise background estimation method is compared with other methods as follows. First, the proposed method by Heikkila and Pietikainenin
, a recent background modelling method with very promising results, is implemented. To train the background model using LBP (Local Binary Pattern) texture operator we use the same number of frames as we used to estimate the background in our proposed method. Fig. 5 compares the estimated background images of our proposed method with that of Heikkila and Pietikainenin. As can be observed, where cells have slow dynamics, cell boundary pixels are apparent in the estimated background images using LBP, leading to significant degradation of the bright cell boundaries, in contrast with our proposed method that precisely estimates background and generates a smooth, uniform background. As can be observed in the normalized histograms, most of the pixels have background of zero, however there is significant number of pixels with intensities higher than background in the top histogram in comparison with the bottom one confirming that some cell boundary pixels are interpreted as background in the estimated background image by  which leads to the loss of cell-background contrast and in turn increases the misdetection (missed cells and false alarms) rate.
Two additional approaches, a frame-difference segmentation method and a morphological averaging background estimation method in , are implemented. Depicted in Fig. 6(a) and 6(b), we can observe that neither frame-difference segmentation nor morphological averaging background estimation provides a satisfactory result. Segmented cells lose their form and become scattered pixels.
Background correction is essential for imperfect cropped wells as well as poor contrast image sequences. However cell model can be directly applied to the image sequences of different pheno-types in which well interior is perfectly cropped and no well boundaries is visible. Fig. 7 shows the cell detection results obtained by applying the proposed cell model to the image sequences of cropped well interior with no well boundaries for different phenotypes before and after point-wise background correction. The detection rate for different cell phenotypes is depicted in Fig. 7(a). As it can be observed, even for the worst case (very poor contrast images), the detection rate was over 90% while it reached 98% for phenotypes 1 and 2. The proposed cell model works well with all cell phenotypes while the cell detection improved significantly (about 10%) for poor contrast images using point-wise background correction. The detection rate applying the proposed cell model to phenotypes 1 and 2 is very high (close to 98%) and there is no need to use the point-wise background correction if well structure is not visible in the cropped image sequence. As we can see the detection rate is almost the same before and after point-wise correction for phenotypes 1 and 2.
Figs. 7(b), (c), and 7(d) show the detection rate as a function of the chosen threshold for phenotypes 1, 2, and 3 respectively. As it can be seen, before background correction, the misdetection rate which includes both missed cells and false alarms, is consistently very high for τ ≤ 0.5 where τ is the threshold by which the detected local maxima are filtered. However, the performance of the proposed cell model on background corrected (point-wise) sequences is more robust such that the misdetection rate drops fast and stables below 10% for 0.3 ≤τ≤ 0.6 after background correction.
Information about the temporal dynamics and spatial distribution of cells is beneficial to study the cell behavior. The traditional method of manual observation and measurement of cells from microscopic images is tedious, expensive, and time consuming. Thus, automated methods for the observation and measurement of cells from microscopic images are in high demand.
In this paper, a mutual algorithm for cell detection and background estimation is presented. The proposed probabilistic cell detection approach, assuming a uniform background, models a cell by the product of the cell probability based on cell interior and the penalizing probability based on outer cell ring. The proposed background estimation method estimates the background over the 3-D sequence to remove the well structure. The proposed method was applied to different HSC image sequences and generated promising results. The purpose of developing this method was analysis of single cells, dividing cells, and low packed cells such that by further outlining the boundary of cells, the cell shape, size, and other features can be extracted over time to study the spatiotemporal cell behavior. Although the proposed method can be modified to potentially detect single cells in tightly packed cell cultures (roughly based on the average cell diameter), the boundary of individual cells can not be discriminated. This might be useful for counting the number of cells, however cannot be used for single cell analysis which is the purpose of this paper.
The proposed method is capable of localizing the specific cell types that have been used in our experiments, however to adapt the method for the other cell types that may not have circular shape, this method must be modified to model the specific cell shape.
Further, the proposed method can potentially be used for detection of different stem cell types; however it has not been tested for detecting cells in stem cell aggregates, in particular for circumstances in which neighboring cells significantly overlap with each other, and where cell morphology may considerably change.
The proposed algorithm was applied to bright field cell images, so in situations that cells are fluorescent (for example by adding calceinAM) or where the user needs to distinguish features across cells such as antibody stained cells, the proposed method must be modified to properly model the cell for the fluorescence range of interest. In such cases, if the color stain would spill outside the cell boundary or partially stain the cell area, the cell detection might drop in accuracy.
Aghili F, Namvar M: "Adaptive control of manipulators using uncalibrated joint-torque sensing,".IEEE Transactions on Robotics 2006, 22(4):854–860. 10.1109/TRO.2006.875489
Santibaez V, Kelly R, Llama M: "Global asymptotic stability of a tracking sectorial fuzzy controller for robot manipulators,".IEEE Tran on SMC Part B: Cybernetics 2004, 34(1):710–718. 10.1109/TSMCB.2003.811764
Ray N, Acton S: "Motion gradient vector flow: An external force for tracking rolling leukocytes with shape and size constrained active contours,".IEEE Transactions on Medical Imaging 2004, 23(12):1466–1478. 10.1109/TMI.2004.835603
Chen X, Zhou X, Wong S: "Automated segmentation, classification, and tracking of cancer cell nuclei in time-lapse microscopy,".IEEE Transactions on Biomedical Engineering 2006, 53(4):762–766. 10.1109/TBME.2006.870201
Close RA, Whiting JS: "Motion-compensated signal and background estimation from coronary angiograms,".Proc Society of Photo-Optical Instrumentation Engineers (SPIE) Conference, Medical Imaging 1995, 2434: 185–194.
Heikkila M, Pietikainen M: "A texture-based method for modeling the background and detecting moving objects,".IEEE Tran on Pattern Analysis and Machine Intelligence 2006, 28(4):657–662. 10.1109/TPAMI.2006.68
Boulanger J, Kervrann C, Bouthemy P: "A simulation and estimation framework for intracellular dynamics and trafficking in video-microscopy and fluorescence imagery,".Medical Image Analysis 2009, 13: 132–142. 10.1016/j.media.2008.06.017
Yang X, Li H, Zhou X: "Nuclei segmentation using marker-controlled watershed, tracking using mean-shift, and kalman filter in time-lapse microscopy,".IEEE Tran on Circuits and Systems I: Regular Papers 2006, 53(11):2405–2414. 10.1109/TCSI.2006.884469
Debeir O, Ham PV, Kiss R, Decaestecker C: "Tracking of migrating cells under phase-contrast video microscopy with combined mean-shift processes,".IEEE Tran on Medical Imaging 2005, 24(6):697–711. 10.1109/TMI.2005.846851
This research has been funded by Natural Science and Engineering Research Council of Canada (NSERC). This research has been also performed in close cooperation with Chemical Engineering Department of University of Waterloo, Canada and Terry Fox Laboratory, Vancouver, Canada. We would like to thank Prof. Connie Eaves and Dr. Brad Dykstra from Terry Fox Lab for providing HSCs and Mr. John Ramunas (at Stanford University) for providing microscopic image sequences for this research and valuable inputs on biological properties of HSCs.
Authors and Affiliations
Department of Systems Design Engineering, University of Waterloo, Waterloo, Canada
Nezamoddin N Kachouie & Paul Fieguth
Department of Chemical Engineering, University of Waterloo, Waterloo, Canada
Harvard-MIT Health Sciences and Technology Harvard Medical School, Cambridge, USA
The authors declare that they have no competing interests.
NNK designed and implemented the algorithm and drafted the manuscript. NNK and PF discussed the methods and modified them. PF read, commented on, and edited the manuscript. EJ provided the image sequences and commented on the methods. All authors read and approved the final manuscript.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Kachouie, N.N., Fieguth, P. & Jervis, E. A probabilistic cell model in background corrected image sequences for single cell analysis.
BioMed Eng OnLine9, 57 (2010). https://doi.org/10.1186/1475-925X-9-57