Image dataset
In this study, a video (length—3 s, 25 frames/second) of the ATFL region is acquired using linear probe (5–13 MHz) of iU22 Philips colour ultrasound machine from 25 subjects (12 healthy, 8 patients with tear injury and 5 with thickened ligament injury) with the age ranges from 18–60 (mean 34) years. Institutional medical ethics approval was obtained prior to the study. Subjects were informed about the study protocol and consent form is obtained from all the subjects. A total of 1250 slices are extracted in the video acquired from 25 subjects, each video contains 75 slices representing ultrasound images of ATFL region. The involved three experts have the experience of 18 years, 9 years and 5 years in ultrasound imaging are asked to visually inspect 75 slices of each dataset and assign a grade (lowest 1, low 2, moderate 3, high 4, and highest 5) to each slices based on the quality according to the method proposed earlier [44]. In most datasets, around 36–37 slices are graded as score 5, around 13–14 slices are graded as score 4, around 9–10 slices are graded as score 3, around 5–6 slices are graded as score 2 and remaining are graded as score 1. In this study, slices those were graded as 4 and 5 are included for the further processing and post processing is performed on 50 slices of each subject.
ATFL segmentation framework
As discussed in introduction section of this paper, this research is mainly focuses on three major challenges such as homogeneous intensity, homogeneous texture and low contrast regions that create the difficulties in ATFL interpretation and extraction from ultrasound images. To overcome these problems, a framework is developed to segment the ATFL region from ultrasound images as illustrated in Fig. 2.
As presented in Fig. 2, there are mainly five steps involved in the segmentation of ATFL from 2D ultrasound images. At first, process of segmentation starts with the initialisation of ROI in ultrasound image frame consisting of ATFL region, which is followed by the AHE method to increase the contrast of the ROI. The contrast enhanced image is further processed by the PSO algorithm for optimization to produce more accurate results. In the following steps, the Chan–Vese method is applied on the optimized image to extract the desired ATFL region. The extracted images are smoothed by the morphological operation for better visualization and accurate interpretation. It should be noted that this research has developed a novel segmentation framework due to the unique hybridization of advanced image processing and optimization algorithms for ATFL segmentation for the first time. The details of the developed framework are described in the following sections:
Initialization of ROI
In this research, ROI is initialized prior to post-processing due to the followings reasons: (1) simplifying the input image, (2) reducing the occurrences of errors during ATFL extraction and, (3) better computational performance. For the selection of ROI, variability of ATFL location in ultrasound images is considered and expert radiologist inspected ATFL region in several slices. In this framework, ROI is selected by selecting a defined region of interest that includes presence of ATFL within this region and an automated cropping of selected region is made. An example of ROI selection where ROI initialized region is indicated by green colour rectangle on input image is shown in Fig. 3b which is extracted from original image as shown in Fig. 3a. Once the ROI selection is performed, developed framework utilises selected ROI to perform further operations as discussed in the next following sections.
Contrast enhancement using AHE
Contrast enhancement methods such as histogram equalization are widely used in image processing to improve the interpretation capability of the resultant image. However, traditional histogram equalization does not performing well for variable contrast images [30]. In such cases, adaptive histogram equalisation (AHE) can be used that shows better performance to adjust the local contrast and deals better with variable contrast [31]. Thus, AHE is used in this research that may provide better outcomes than traditional histogram.
Mathematically, AHE can be described in Eqs. 1 and 2 [30, 31]:
$$\hat h\left( {m,n,g} \right) = \delta {\left( {g,x\left( {m,n} \right)} \right)^{m,n}}\cdot{f_w}\left( {m,n} \right)$$
(1)
$${f_w}(m,n) = \left\{ {\begin{array}{*{20}{c}} {{w^{ - 2}},\begin{array}{*{20}{c}} {\left| m \right| \le (w - 1)/2,}&{\left| n \right| \le (w - 1)/2} \end{array}}\\ {0,\begin{array}{*{20}{c}} {}&{}&{}&{\begin{array}{*{20}{c}} {\begin{array}{*{20}{c}} {}&{} \end{array}}&{}&\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,{otherwise}&{} \end{array}} \end{array}} \end{array}} \right.$$
(2)
where, x is an input image, \(\hat{h}\) presents the histogram and δ is the Kronecker delta function. g indicates an image with grey levels. The parameter f
w
indicates the rectangular kernel with m rows and n columns, and w represents the width of the window. In this research, size of the kernel is considered as equal to the size of input image to cover all the objects. The ROI initialized image shown in Fig. 3b is further improved by the AHE method that helps in the betterment of contrast of input image as shown in Fig. 3c.
Optimization using PSO
PSO algorithm optimizes a problem iteratively to find out the candidate solution by moving particles around in search space. Each particle has a specific position and velocity. The corresponding position x
i
(t) and velocity v
i
(t) are determined and update based on the Eqs. 3 and 4 [32, 45–47]:
$$v_{i} \left( {t + 1} \right) = wv_{t} \left( t \right) + c_{1} r_{1} \left( t \right)\left( {y_{i} \left( t \right) - x_{i} } \right) + c_{2} r_{2} \left( t \right)\left( {\hat{y}\left( t \right) - x_{i} \left( t \right)} \right)$$
(3)
$$x_{i}^{{}} (t + 1) = x_{i}^{{}} (t) + v_{i}^{{}} (t + 1)$$
(4)
where, w represents the inertia weight that provides the storage to resultant velocities, \(y_{i} \left( t \right)-x_{i}\) works based on each particle personal experience with respect to the best solution and \(\hat{y}_{i} (t) - x_{i}\) indicating the confidence of whole swarm for best solution. The c
1 and c
2 are acceleration constants that are used to speed-up the image particles, \(\left\{ {r_{1} \left( t \right),r_{2} \left( t \right)} \right\} \sim u\left( {0,1} \right)\), in which u(0,1) represents a random number range from 0 to 1. The t indicates the time unit. The x
i
(t) and v
i
(t) represents the current position and velocity of a particle i at time t. The term y
i
(t) indicates the personal best position and \(\hat{y}_{i}\) is the global best position of a particle i.
PSO is a problem-independent algorithm, which means that little specific knowledge relevant to given problem is required. What we have known is just the fitness evaluation for each solution. This advantage makes PSO more robust than many other search algorithms. The main benefits of the PSO algorithm as compared to existing genetic algorithm and other heuristic algorithms in image segmentation are as follows: (1) The PSO algorithm is easy to implement and only few parameters have to be adjusted, (2) unlike the genetic algorithm, the PSO algorithm has no evolution operators such as crossover and mutation, which can be the main cause of computational complexity in some cases, (3) in PSO, only global best particle gives out information to the other image particles rather than whole population that makes it more robust, (4) unlike other heuristic algorithms, PSO has the flexibility to control the balance between global and local exploration of the search space.
Basically, the PSO algorithm is a multilevel thresholding approach that optimizes the energy level of the input image and helps in efficient curve evolution to extract the desired ATFL ligament region. In image segmentation, the use of PSO method is quite simple and effective, which improves the real-time performance of the image segmentation to a large extent. This research performs the segmentation on ATFL ligament ultrasound images based on the PSO algorithm to optimise the energy level of the image as illustrated in Fig. 3d. The experiments of segmentation indicates that the developed framework can get ideal segmentation results with less computation cost due to the efficient use of PSO algorithm. Therefore, the optimized image would help in efficient extraction of the ATFL boundaries by the curve evolution, which is performed by the Chan–Vese method as discussed in the next section of this paper.
ATFL extraction using Chan–Vese method
In this research, curve evolution is performed by the Chan–Vese method. The Chan–Vese method is the association of the Mumford–Shah mathematical model, level set method and curve evolution for energy minimization, accurate initialization and curve evolution during segmentation progression, respectively. The entire processing of the Chan–Vese method is described in Eq. 5 [33, 48–50]:
$$E^{CV} (c_{1} + c_{2} ,\varphi ) = E^{FT} (c_{1} + c_{2} ,\varphi ) + \mu L(\varphi )$$
$$= \left( {\int_{\varOmega } {|\text{Im} - c_{1} |^{2} H(\varphi )dxdy + } \int_{\varOmega } {|\text{Im} - c_{2} |^{2} (1 - H(\varphi ))dxdy} } \right) + \mu \int_{\varOmega } {\delta (\varphi )|\nabla \varphi |dxdy}$$
(5)
where, c
1
, c
2
are constants and φ is an undefined curve. E
FTrepresents the external energy function of curve C. H(φ) is heaviside function and δ(φ) is the Dirac one dimensional function. L is the image gradient and μ is a fixed parameter range less than 1. The image Im indicates the input image and the Im(x, y) depicts the particular image coordinate. Here ∇ stands for the gradient operator. In Eq. 5, function E
CV(c
1
, c
2, φ) depicted the Chan–Vese function.
Basically, the Chan–Vese method is the extended version of the traditional active contour method [51], which used in a variety of image processing tasks such as image segmentation and object boundary tracking. The traditional active contour method initially specifies a contour, which evolves under smoothness control (internal energy) and image driven forces (external energy) embedded with energy minimization capability to detect the boundary of the desired object. To minimize the energy, the traditional active contour method used Euler–Lagrange equation [52]. Although, this method is performing well, but it has some major limitations such as long runtime, need to initialize snake near to the object boundary and it is unable to merge two contours into one or split one contour into two contours. In order to overcome these issues, Chan and Vese proposed a segmentation method, which is applied in this research.
The Chan–Vese method does not depend on the edge function for the termination of shrinking or expanding curve for a preferred object boundary. The Chan–Vese method detects object boundaries much clearer; in case of undefined boundary gradients and noisy image. Thus, the Chan–Vese method is applied on the optimized images (e.g. PSO outcomes) to provide more accurate segmentation results as described in the developed framework. For example, boundary extraction of ATFL is performed by the Chan–Vese method [33] with 800 iterations as illustrated in Fig. 3e by three processing stages (1–2–3). Here, Optimized image is given as input for further processing using Chan–Vese method that shows number of iterations at different scales as shown in Fig. 3e and iterations are discontinued after its optimal position that leads to the extraction of ATFL region as shown by marked region in Fig. 3e, which is further extracted as binary image shown in left image in Fig. 3f.
Hole filling using morphological operation
Morphological operations are used to restore and reconstruct the damaged parts of the extracted image. Out of numerous morphological operations, this research applied the close operation to produce smoother results (see Fig. 3), which is described in Eq. 6 [34–36]:
$$C\left( {A,B} \right) = A \cdot B = E\left( {D\left( {A, - B} \right), - B} \right)$$
(6)
where, C is the close operator. A and B are the object sets of a binary image. E and D indicates the erosion and dilation, respectively. The close operation used two inputs such as, an image that need to be smoothed and a structuring element. However, this research used the disk structuring element to preserve the arbitrary shape of the object, which specifies a radius of 10 pixels so that the largest gap gets filled. For instance, during curve evolution the obtained segmented outcomes are not so accurate due to the asymmetrical boundaries and uncertain shape that leads to interpretation problem of the damaged tissues of ATFL as shown in left image of Fig. 3f. To overcome this problem, the developed framework uses morphological close operation on the extracted images for smoothing and clear boundaries as depicted in right image of Fig. 3f.
Figure 3 presents the entire process flow of ATFL segmentation performed by the developed framework. Figure 3a has shown the input ultrasound image of ATFL ligament, which presented the ROI initialization in green colour. Thereafter, the selected ROI is illustrated in Fig. 3b, which is further enhanced by the adaptive histogram equalization method to improve the contrast for better visualization of ATFL region (see Fig. 3c). The contrast enhanced image has shown the boundaries of ATFL ligament more clearly as compared to the input image. To make the enhanced image more optimized for further processing, the developed framework applied the PSO algorithm as depicted in Fig. 3d. The optimized image is further used in contour evolution, which is performed by the Chan–Vese method as illustrated in Fig. 3e by the three processing stages (1–2–3). In addition, Fig. 3e presented the process flow of the contour evolution to extract the ATFL region, which is indicated by the green colour. As mentioned earlier, the developed framework is capable to extract the ATFL region by the use of 800 iterations only, which are very less as compared to the existing methods. The extracted ATFL region is shown in Fig. 3f by two stages (1–2) that has the lacking of regular boundaries. To recover this issue, this framework used the morphological close operation to fill the gaps of boundaries for accurate interpretation of ATFL ligament injuries by the clinicians.
Performance evaluation
The developed framework was implemented on MATLAB [53, 54] running on a CPU (configuration: 64-bit operating system, 8.00 GB RAM, Intel (R) core (TM), i7-2600, 3.40 GHz). Once the developed framework is tested on the sample and image datasets, performance of the developed framework is further evaluated by measuring few performance metrics such as computation time, sensitivity, specificity, accuracy, Hausdorff distance, Jaccard index and segmented area which are further elaborated in the next sub-sections. After segmentation of ATFL region using developed method, performance metrics such as computation time and segmented area are directly measured from segmented images. However, for the measurement of sensitivity, specificity, accuracy, Hausdorff distance and Jaccard Index, the obtained segmentation results are compared with ground truths that were manually segmented from 2D ultrasound images by three expert radiologists having experience of 18 years, 9 years and 5 years. An example showing manual segmentation of ATFL region in 2D ultrasound image performed by the expert is illustrated in Fig. 4.
The reliability of above ground truths generation by different experts is estimated by the intra-observer variability among the experts. In order to measure the reliability of expert’s segmentation, measurement of true positive rate (TPR) among different experts is performed using Eq. 7 given in the next section.
Performance metrics
In order to analyse the performance of the developed framework, computational time, sensitivity, specificity, accuracy, Hausdorff index, Jaccard index and segmented area metrics are explained in detail in the following sections [35]:
Computational time
Computational time is the amount of time for which a central processing unit (CPU) was used for processing instructions of a computer program that can be measured in seconds [55].
Sensitivity
Sensitivity is the proportion of true positives that are correctly identified by a diagnostic test. It shows how good the test is at detecting a disease [55].
$$Sensitivity = \frac{TP}{TP + FN}$$
(7)
where, true positive (TP) is the number of pixels correctly labelled as ATFL region, false negative (FN) is the number of pixels incorrectly labelled as non-ATFL region.
Specificity
Specificity is the proportion of the true negatives correctly identified by a diagnostic test. It suggests how good the test is at identifying normal condition [55].
$$Specificity = \frac{TN}{TN + FP}$$
(8)
where, true negative (TN) is the number of pixels correctly labelled as non-ATFL region, false positive (FP) is the number of pixels incorrectly labelled as ATFL region.
Accuracy
Accuracy is the proportion of true results, either true positive or true negative, in a population. It measures the degree of accuracy of a diagnostic test on a condition [55].
$$Accuracy = \frac{TP + TN}{TP + FP + FN + TN}$$
(9)
Hausdorff distance
Hausdorff distance is a validation metrics used in medical image segmentation for shape matching. Hausdorff index determined the degree of similarity between two superimposed sets, which is defined in Eq. 10 [56]:
$$H(A,B) = \max_{a \in A} \cdot \min_{b \in B} \left\| {a - b} \right\|$$
(10)
where, H presents the Hausdorff index. a and b are the points defined in sets A and B. ‖a − b‖ is indicating the underlying distance in tests.
Jaccard index
Jaccard index is a similarity measure that lies 0 to 1 as presented in Eq. 11 [56]:
$$J(A,B) = \frac{A \cap B}{A \cup B}$$
(11)
where, J depicts the Jaccard index, A indicates the region segmented by the developed framework and B is the region segmented by the experts.
Segmented area
The extracted desired and meaningful region from an input image is referred to as segmented area, which is determined by the calculation of number of pixels from the extracted region [55]. In this research, segmented area is calculated from normal and patients to demonstrates the clinical significant of the developed method.
