Image resizing using saliency strength map and seam carving for white blood cell analysis
© Ko et al; licensee BioMed Central Ltd. 2010
Received: 11 May 2010
Accepted: 20 September 2010
Published: 20 September 2010
A new image-resizing method using seam carving and a Saliency Strength Map (SSM) is proposed to preserve important contents, such as white blood cells included in blood cell images.
To apply seam carving to cell images, a SSM is initially generated using a visual attention model and the structural properties of white blood cells are then used to create an energy map for seam carving. As a result, the energy map maximizes the energies of the white blood cells, while minimizing the energies of the red blood cells and background. Thus, the use of a SSM allows the proposed method to reduce the image size efficiently, while preserving the important white blood cells.
Experimental results using the PSNR (Peak Signal-to-Noise Ratio) and ROD (Ratio of Distortion) of blood cell images confirm that the proposed method is able to produce better resizing results than conventional methods, as the seam carving is performed based on an SSM and energy map.
For further improvement, a faster medical image resizing method is currently being investigated to reduce the computation time, while maintaining the same image quality.
Peripheral blood cell differential counting provides valuable information for accurate patient diagnoses, yet the microscopic review is labor intensive and requires a highly trained expert. Current automated cell counters are based on laser-light scatter and flow-cytochemical principles, nonetheless, 21% of all processed blood samples still require microscopic review by experts . Therefore, various efforts [1–5] have already been made to develop automatic cell analysis systems using image processing. Blood cell images consist of both white and red blood cells scattered across the entire image, however, it is the white blood cells (WBCs) that provide the important information for patient diagnoses, such as leukemia or cancer . Thus, in most research, WBC segmentation is the important procedure, where the ultimate goal is to extract all the WBCs from a complicated background and then segment the WBCs into their morphological components, such as the nucleus and cytoplasm.
Representative WBC analysis systems, such as Cellarvision Diffmaster Octavia  and Cellarvision DM96 , scan the whole slide at a low magnification first to identify potential WBCs using the specific characteristics of WBCs, such as their color, size, and shape, and then take digital images at a high magnification. Thereafter, pre-classification is performed using only the cropped digital images. While this method is more efficient than scanning WBCs from a high-resolution image of the whole slide, additional time is required for the WBC search, especially when the image contains several WBCs. Furthermore, additional storage is needed to save the individual potential WBCs and extra time required to classify the WBCs, as the system has to check all potential WBC images to analyze just one slide.
Meanwhile, other methods [2, 3] use only an original high-resolution image for the WBC analysis. However, analyzing WBCs from the whole image is time consuming, since the size of blood cell images is normally at least 800 × 600. Therefore, an image-resizing method is needed that retains all the WBCs without morphological distortion in order to reduce the post-segmentation classification time. Furthermore, since resized high-quality images require less storage, the post-image segmentation and classification can be more accurate than with conventional image compression, such as JPEG.
Related work can be divided into two parts; image compression and image resizing.
First, various lossless compression techniques already exist that can preserve the characteristics of an image, yet with a low compression rate. For example, several researchers [6–8] have proposed transform coding schemes, such as a Principal Component Analysis (PCA) and Discrete Cosine Transform (DCT), while Karras et al.  used a discrete wavelet transformation (DWT) and fuzzy c-means clustering technique. Plus, to achieve higher compression rates without detracting from the quality, region of interest (ROI) methods with a DCT have also been investigated [6, 10]. In particular, Gokturk et al.  proposed a hybrid model, using lossless compression in regions of interest and high-rate motion-compensated lossy compression in other regions in the case of a sequence of CT images. Nonetheless, even though lossless compression produces a higher compression rate without distorting ROIs, the exact preservation of a ROI is still difficult when the compression rate is above a specific limitation. Therefore, a new algorithm is needed that can efficiently preserve ROIs, regardless of the compression rate.
Accordingly, this paper presents a new method for resizing blood cell images while preserving the size and shape of WBCs. In peripheral blood, WBCs are divided into five classes according to their maturation stage, making it essential to preserve the size and shape of the nucleus. Thus, to provide an efficient image-resizing method that treats WBCs as ROIs, a Saliency Strength Map (SSM) is proposed using a visual attention model and the structural properties of WBCs to generate a new energy map. As such, this map maximizes the energies of the WBCs, while minimizing the energies of the red blood cells and background. Therefore, in contrast to previous algorithms, the main contribution of this study is to improve the resizing performance with a lower file size, while preserving the WBCs using the proposed SSM with an energy map.
The remainder of this paper is organized as follows. Methods describes the algorithms used to create the saliency strength map, an Ellipse Attention Window (EAW) that removes useless regions from the image, and the seam removal using an energy map based on the saliency strength. Results and Discussion evaluates the accuracy and applicability of the proposed resizing method based on experiments, and some final conclusions and areas for future work are presented in Conclusions.
This paper proposes a new image-resizing method with a lower file size that can efficiently preserve WBCs using a visual saliency map based on the following two assumptions:
▪ the nuclei of WBCs are nearly round.
▪ the nuclei of WBCs are highlighted in purple on a white background with mono-chromatic red blood cells.
Using these characteristics, a Saliency Strength Map (SSM) is proposed using a visual attention model, while the structural properties of WBCs are used to generate an energy map.
Saliency map generation
In contrast to nature images, microscopic images, especially blood cell images, have different characteristics with distinct diagnostic meanings, such as a varying color and saturation according to fluorescence staining. For example, in the case of blood cell images, the salient parts, the WBCs, tend to be highly saturated and purple in color, while the remaining parts, the red blood cells, have a more monotonous appearance. Thus, for semantic seam carving, knowledge of the exact positions of the relevant WBCs is crucial. Therefore, to obtain the position of WBCs, a modified visual attention model is used, as proposed in our previous research .
Saliency strength map generation
The six steps for extracting the AW and estimating the EAW S are as follows:
Step2: Morphological opening is performed to fill any holes in the cell nuclei.
Step3: After region labeling, small regions are removed if the size of a region is below a predefined minimum threshold (3% of all image pixels). This predefined minimum threshold was determined by analyzing the minimum cell region from whole training cell regions.
Step4: The initial position of the AW in each region is estimated using an X-Y projection.
Step5: The elliptical AWs (EAW) are re-estimated using the centroid and radius of the initial AW.
Step6: A distance transform is performed and the strength of the EAW(EAW S ) estimated.
where g represents a Gaussian smoothing operator to reduce minor noise.
Finally, the resized image based on the SSM and its energy map is shown in Fig. 5-(j).
Seam removal using energy map based on saliency strength map
Seam carving uses two types of energy removal strategy: backward and forward. Backward energy strategies are based on evaluating the energy, yet they introduce visual artifacts due to their seam removal strategy. The seams containing the lowest energy are removed one after another, however, the energy inserted into the new edges created by previously non-adjacent pixels that become new neighbors is ignored after a seam is removed. Thus, to reduce these visual artifacts, forward energy strategies  substitute an energy evaluation that calculates three possible seam step costs and defines the minimal amount of energy inserted by the removal of a seam.
where SSM(i, j-1) is the new pixel that is replaced after removing SSM(i, j), SSM(i, j+1) and SSM(i-1, j) are the new right and upper neighbors, respectively, and C L , C U , and CR represent the costs of the three possible vertical seams.
where P(i, j)is the gradient value obtained from SSM(i, j), M(i-1, j-1) is the left upper neighbor, and C L is its cost. The cost of the corresponding upper M(i-1, j) neighbors C U and right upper M(i-1, j+1) neighbors C R are computed in the same manner to determine the minimum energy of the new saliency strength after removing SSM(i, j).
Once the cost matrix is constructed, M(i, j) in a random position represents a pixel (i, j) in a path crossing the image from top to bottom, and is connected to other adjacent pixels containing the minimal energy according to the saliency strength. Consequently, the resizing is performed by iteratively creating a cost matrix after blending the gaps arising from seam removal.
Results and discussion
The experimental tests used color peripheral blood images collected at the Severance Hospital, Yonsei University. The 8 test images were based on a slide of a peripheral blood smear and taken using a microscope, charge-coupled device (CCD) camera, and 24-bit digitizer with an 800 × 600 image size.
As there is no specific method for evaluating the performance of image resizing, the Peak Signal-to-Noise Ratio (PSNR) was used first to evaluate the content preservation of the proposed method. Plus, the Ratio of Distortion (ROD) was applied to evaluate the geometric distortion of the resized images. Note that, the size of the source images was 800 × 600 and the target size was automatically determined according to the size of the EAWs to include all the nuclei without distortion.
PSNR (Peak Signal-to-Noise Ratio) comparison
An experimental comparison of seam carving is generally very difficult as there are no standard criteria for performance tests. Thus, to validate the effectiveness of the proposed approach, this study used the Peak Signal-to-Noise Ratio (PSNR).
Where m and n represent width and height of the image, respectively.
For the performance test, gradient-based seam carving and the proposed method were applied to the source images to create target images. The WBC nuclei were then cropped manually from each image with a graphic tool and used for the PSNR comparison.
Thus, although JPEG compression can produce the perception of identical results with the original WBCs, there is a loss of image quality, as shown in Figs. 8-(b) and 9, meaning the characteristics needed for WBC segmentation and classification are not always preserved. In contrast, since the proposed method is able to preserve the original image quality, better segmentation and classification results can be expected in the post-processing steps. Therefore, the proposed image resizing algorithm can be very useful for reducing the cell analysis and memory storage in the case of medical images, especially blood cell images.
ROD (Ratio of Distortion) comparison
To evaluate the ratio of distortion (ROD) of the WBC nuclei after image resizing, a new evaluation method was used based on the geometric properties of the objects. First, three different users were asked to crop the WBC nuclei from the resized images shown in Figs. 8(c) and 8(d) using a graphic tool, and only those WBCs where at least two users were in agreement were then used for the comparison.
ROD comparison of seam carving and proposed method
The seam carving results produced distortions of 0.5~0.8, whereas no distortion occurred with the proposed method. As such, the seam carving was unable to preserve the WBCs when the image was resized harshly. Since an energy map using the original gradient magnitude of the WBCs is not distinctive in blood images, this makes it hard to apply a seam carving operator to blood cell images, and the WBC contents are inevitably affected during the pixel removal. In contrast, the proposed method was able to preserve the WBCs exactly, as it used the SSM to maximize the energies of the WBCs and minimize the energies of the red blood cells and background.
This paper proposed a new image compression method that uses a Saliency Strength Map (SSM) and seam carving to preserve important contents, such as WBCs included in blood cell images, with a lower file size. The SSM is constructed using a visual attention model and the structural properties of WBCs to generate a new energy map. Thus, the purpose of the map is to maximize the energies of the WBCs, while minimizing the energies of the red blood cells and background.
In experiments, the proposed method was shown to improve the file compression performance when compared to JPEG. Nonetheless, despite the improved performance of seam carving based on an SSM, additional computation time is required depending on the image resolution. Therefore, a faster cell-image resizing method is currently being investigated to reduce the computation time, while maintaining the same image quality.
This work was supported by grant No. RTI04-01-01 from the Regional Technology Innovation Program of the Korean Ministry of Knowledge Economy (MKE).
- Ceelie H, Dinkelaar RB, Gelder W: Examination of peripheral blood films using automated microscopy; evaluation of Diffmaster Octavia and Cellavision DM 96. J Clinical Pathology 2006, 60: 72–79. 10.1136/jcp.2005.035402View ArticleGoogle Scholar
- Nipon TU, Sompong D: Morphological Granulometric Features of Nucleus in Automatic Bone Marrow White Blood Cell Classification. IEEE Trans. Information Technology in Biomedicine 2007, 3: 353–359.Google Scholar
- Sadeghian F, Seman Z, Ramli AR, Kahar BH, M I: A Framework for White Blood Cell Segmentation in Microscopic Blood Images Using Digital Image Processing. Biological Procedures Online 2009, 1: 196–206. 10.1007/s12575-009-9011-2View ArticleGoogle Scholar
- Swolin B, Simonsson P, Backman S: Differential counting of blood leukocytes using automated microscopy and a decision support system based on artificial neural networks-evaluation of DiffMaster Octavia. Clinical and laboratory haematology 2003, 25: 139–147. 10.1046/j.1365-2257.2003.00516.xView ArticleGoogle Scholar
- Kratz A, Bengtsson H, Casey JE: Performance evaluation of the Cellavision DM96 system. American J of clinical pathology 2005, 124: 770–780. 10.1309/XMB9K0J41LHLATAYView ArticleGoogle Scholar
- Vlaciu A, Lungu S, Crisan N, Persa S: New compression techniques for storage and transmission of 2-D and 3-D medical images. Proceedings of the SPIE Advanced Image and Video Communications and Storage Technologies 1995, 2451: 370–377.View ArticleGoogle Scholar
- Taur JS, Tao CW: Medical Image Compression using Principal Component Analysis. Proceedings of the Conference on Image Processing 1996, 1: 903–906. full_textView ArticleGoogle Scholar
- Yoshioka M, Omatu S: Image Compression by nonlinear principal component analysis. Proceedings of the IEEE Conf. Emerging Technologies and Factory Automation 1996, 2: 704–706.Google Scholar
- Karras DA, Karkanis SA, Maroulis DE: Efficient Image Compression of Medical Images Using the Wavelet Transform and Fuzzy c-means clustering on Regions of Interest. Proceedings of the IEEE Conf. EUROMICRO Medical Informatics 2000, 2: 469–473.Google Scholar
- Gokturk SB, Tomasi C, Girod B, Beaulieu C: Medical Image Compression Based on Region of Interest, with Application to Colon CT Images. Proceedings of the IEEE Conf. Medicine and Biomedical Engineering 2001, 3: 2453–2456.Google Scholar
- Suh B, Ling H, Bederson BB, Jacobs DW: Automatic thumbnail cropping and its effectiveness. Proceedings of the 16th annual ACM symposium Conf. User interfaces software and technology 2003, 1: 95–104. full_textView ArticleGoogle Scholar
- Chen L, Xie X, Fan X, Ma W, Zhang H, Zhou H: A visual attention model for adapting images on small displays. ACM J Multimedia Systems 2003, 9: 353–364. 10.1007/s00530-003-0105-4View ArticleGoogle Scholar
- Avidan S, Shamir A: Seam carving for content-aware image resizing. ACM Trans. SIGGRAPH 2007, 3: 1–9.Google Scholar
- Ko BC, Nam JY: Object-of-Interest Image Segmentation using Human Attention and Semantic Region Clustering. J of Optical Society of America A: Optics, Image Science, and Vision 2006, 10: 2462–2470. 10.1364/JOSAA.23.002462View ArticleGoogle Scholar
- Itti L, Koch C, Niebur E: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Analysis and Machine Intelligence 1998, 11: 1254–1259. 10.1109/34.730558View ArticleGoogle Scholar
- Otsu N: A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 1979, 9: 62–66. 10.1109/TSMC.1979.4310076View ArticleGoogle Scholar
- Rubinstein M, Shamir A, Avidan S: Improved seam carving for video retargeting. Proceedings of the ACM Conf. SIGGRAPH 2008, 3: 1–9. 10.1145/1360612.1360615View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.