Sleeping posture recognition using fuzzy c-means algorithm

Background Pressure sensors have been used for sleeping posture detection, which meet privacy requirements. Most of the existing techniques for sleeping posture recognition used force-sensitive resistor (FSR) sensors. However, lower limbs cannot be recognized accurately unless thousands of sensors are deployed on the bedsheet. Method We designed a sleeping posture recognition scheme in which FSR sensors were deployed on the upper part of the bedsheet to record the pressure distribution of the upper body. In addition, an infrared array sensor was deployed to collect data for the lower body. Posture recognition was performed using a fuzzy c-means clustering algorithm. Six types of sleeping body posture were recognized from the combination of the upper and lower body postures. Results The experimental results showed that the proposed method achieved an accuracy of above 88%. Moreover, the proposed scheme is cost-efficient and easy to deploy. Conclusions The proposed sleeping posture recognition system can be used for pressure ulcer prevention and sleep quality assessment. Compared to wearable sensors and cameras, FSR sensors and infrared array sensors are unobstructed and meet privacy requirements. Moreover, the proposed method provides a cost-effective solution for the recognition of sleeping posture.

Sleeping posture recognition is also an important issue in medical care for ambulatory patients or outpatients, as deterioration or amelioration of certain diseases are related to sleeping posture. For example, researchers have shown that sleeping in the decubitus position increases the risk of developing subacromial impingement syndrome by 3.7× as compared with those who sleep in the supine position [2]. Sleeping in a lateral position can effectively reduce symptoms in patients with mild or moderate sleep apnea [3]. Symptoms of sleep paralysis usually occur in the supine position [4]. Sleeping on the right side has a higher risk than sleeping on the left side of development of transient lower esophageal sphincter relaxation, which is a main factor in nocturnal gastroesophageal reflux [5,6].
Researchers have also found that sleep quality is related to sleeping position and frequent sleep postural changes. For example, snoring or extensive body movement may result in a shorter sleep duration [7]. Subjects who report a history of poor sleep quality spend more time in the supine position with the head straight [8], and subjects who sleep on their left side have a significantly higher rate of nightmares (40.9%) than those who sleep on their right side (14.6%). Also, recent research in rats indicated that a lateral sleep posture is more effective for waste removal from the brain during sleep [9], while waste in the brain may contribute to some neurological diseases such as Alzheimer's and Parkinson's.
For bed-bound and limited-mobility patients, the sensor system employed is usually simple and inexpensive, but may achieve a high posture recognition accuracy when the patient is placed by the caregiver in the central axis of the bed. However, the accuracy drops acutely with deviation of the patient's body from the central axis [1,10]. For ambulatory patients or healthy elderly people, the design becomes much more complicated and relatively more expensive, as human beings exhibit a variety of postures during sleep.
In early studies, researchers obtained sleeping posture information by interviewing subjects. In recent years, however, multiple approaches have been developed based on different sensing modalities to determine sleeping postures [11,12]. Among the various widely-used techniques, pressure-sensing and camera-based visual data are most common. Different camera systems are used to acquire visual data. A common digital camera, described in [13], is used mainly as a fall-detection system, but can also identify a lying posture. In [14], the authors modeled the human body in terms of its constituent body parts; then, for each part, a multi-view Eigen model that combined image views from numerous calibrated cameras was built. Using a deformable triangulation method, a body part segmentation algorithm was presented in [15] based on body postures. However, these methods require high-resolution camera images or a sequence of high-resolution video recordings, and concerns have been raised regarding video monitoring systems in terms of invasion of privacy. In [16][17][18], the researchers used pressure sensors to determine sleeping posture based on the pressure distribution. In [19], Wai et al. proposed a 56-sensor layout for patients with higher mobility. This scheme consisted of a 7 × 7 round-sensor array for the upper body and a 7 × 1 elongated sensor array for the lower body. The sleeping posture classification accuracy of this system was reported to be 93%.
As most of the pressure distribution is contributed by the hips and chest, limbs are rarely recognized unless thousands of sensors are deployed; in addition, the positions and postures of the feet are more variable than those of the chest and hips. These are the reasons for which the accuracy of FSR-based posture recognition is significantly low. To increase the accuracy and lower the cost, in this work we deployed a new sensing system over the lower body instead of a pressure sensor array. This paper presents a sleeping posture recognition scheme based on force-sensing resistor (FSR) and infrared array sensors. FSR sensors were deployed to obtain the pressure distribution of the upper body, while the lower body position was detected by a single infrared array sensor. In summary, the contribution of this paper is the novel design of an accurate, cost-effective technique for the recognition of sleeping posture.

Methods
The proposed sleeping posture recognition method uses two kinds of sensor: FSR sensors for the upper body and an infrared array sensor (Grid-EYE) for the lower body. FSR sensors are thin-film sensors made of a piezoelectric polymer, and the resistance decreases in proportion to the applied force on the active surface [20]. Each FSR sensor can measure force up to a value of 10 kg/cm 2 . The Grid-EYE system is an infrared thermal imaging array sensor consisting of an 8 × 8 matrix of sensors [21] that detects and renders thermal images of the temperature distribution of a movable or non-movable object. The FSR sensor array used in our experiment is shown in Fig. 1a; the infrared array sensor is shown in Fig. 1b. The sensing coverage of each pixel changed when the distance between the Grid-EYE array and the subject varied, as shown in Fig. 2. Figure 3 shows the different levels of data-processing required for sleeping posture recognition. The sensor data from the pressure and infrared sensor arrays were processed separately to extract the features from each sensor type. The features extracted from both types of sensor were then combined into the final system in order to recognize and distinguish six kinds of sleeping posture, as shown in Fig. 4.
As human beings exhibit a variety of sleeping postures, the postures are usually grouped into several categories based on the body trunk and limb positions. Idzikowski [22] [18] further classified the fetus (sleeping on one's side with the body curled up) and the log (sleeping on one's side with the back and legs straight) positions into left fetus, right fetus, left log, and right log positions, as shown in Fig. 4. This classification method was adopted in our experiments.

Pre-processing
This section presents the pre-processing method used for both the pressure and infrared sensor data. The purpose of pre-processing of data was to remove noise from the collected data. The moving average filter (MAF) method is an effective technique by which to reduce random noise of time domain-encoded signals [23]. The effect of noise arising from mattress pressure can be minimized by calculating the MAF as follows: where x n is the nth time sensing data and N is the number of points (N = 10). After using the MAF, a thresholding filter was adopted to highlight essential sensing data. If the pressure/temperature value was greater than the defined threshold, the sensing value was assigned as 1; otherwise, the sensing value was assigned as 0. Data that exceeded the thresholds were termed "sensed points".

Middle point/axis determination using fuzzy c-means clustering
After pre-processing of the raw sensor data, the middle point and the middle axis needed to be calculated using a fuzzy c-means (FCM) clustering algorithm. FCM [24][25][26] is a widely-used method for soft image clustering, and was employed in this study to eliminate the effects of position and orientation differences of different subjects on sleeping posture determination. Given a horizontal/vertical axis and a dependent feature vector x, FCM aims to minimize the following objective function: where A is the objective function, n is the number of horizontal/vertical axes, c is the number of clusters, u ik is the degree of membership of x i in cluster k, x i is the ith value of d-dimensional measured data, c k is the d-dimension center of the kth cluster. The Euclidean distance between x i and c k , �x i − c k � , can be calculated as: where x iP and c kP are positioning indexes on the horizontal/vertical axis of sensing points after histogram projection processing, and x iN and c kN are the numbers of sensed points. The degree membership u ik can be calculated as: The FCM clustering algorithm has five steps: 1. Set the number c of clusters and the stopping condition. 2. Calculate the cluster centroid. 3. For each sensor point, compute the membership value for each cluster. 4. Compute the objective function, shown in formula (2). If the value of A between consecutive iterations < ε, then stop. Otherwise, go to step 2. 5. Assign each positioning index to a cluster after defuzzification.
After building up the FCM clustering, it can be applied to find the middle point/axis. For determination of the middle point of the trunk, the proposed method consisted of five steps, as follows:

Histogram projection
The data received by each pressure sensor is defined as a sensing point and the 2-dimension space of the bedsheet is defined as a coordinate. After pre-processing, the data of each sensing point are projected to horizontal and vertical directions, as shown in Fig. 5.

Clustering
Using the FCM clustering algorithm, the Euclidean distance between x i and c k is calculated by considering axis positioning indexes and the number of projections, shown in formula (3). In other words, if the positioning indexes are closer to one another, they will be more likely to cluster into the same cluster; and if the number of projections on an axis positioning index is near to another, the axis positioning index will be more likely to cluster into the same cluster. The algorithm allows pre-definition of the number of clusters. In this study, the pressure sensors were clustered into three clusters in the horizontal and vertical directions, respectively (yellow, blue and green curves represent the three different clusters in Fig. 6). Figure 6 shows the pressure sensors that detect the upper body. On the horizontal projection, the left-side cluster and the right-side cluster are both background clusters. The middle cluster is a foreground cluster mapped to the trunk, and the maximum membership value of the positioning index of the middle cluster is chosen as the middle axis. The same can be applied to the lower body to segment legs mapped into two sub-regions to find the middle axis of the leg map.

Finding the middle axis of the vertical projection
On the vertical projection of Fig. 6, the middle cluster is mapped to the waist; the upperside cluster is mapped to the chest, the lower-side cluster is mapped to the hips, and the maximum membership value of the positioning index of the middle cluster is chosen as the middle axis.

Cross point of the two middle axes
Finally, by combining the two middle axes, the cross point can be determined. The cross point offers the key to segmenting the trunk map into four sub-regions.

Feature extraction
The work in [18] proposed a feature extraction method for posture classification that is based on the geometry of the pressure images. The features are described as either spatial features or body-part features. Our feature extraction was based on the features of symmetry and balance. The middle point of the trunk and the middle axis of the legs were applied to segment the body-part map, as shown in Fig. 7. Four sub-regions were segmented by the middle point. In Figs

Sleeping posture classification using a decision tree
The decision tree method is one of the most common predictive modeling approaches used in statistics, data-mining and machine learning. We used a distance-weighted k-nearest neighbor algorithm (k-NN) [27] to classify the sleeping postures of a subject. In our decision tree structure, as shown in Fig. 10 feature. In each feature space, we employed the k-NN classification technique to recognize the sleeping posture.

Feature spaces of sleeping postures
This study applied two kinds of feature space to recognize sleeping postures: the trunk symmetry feature space (Fig. 11) and the leg symmetry feature space (Fig. 12). Four major upper-body postures, consisting of supine, prone, left lateral and right lateral positions, were described by the trunk symmetry feature space. As described in previous sections, T D-RL and T D-CH were used to compose the trunk symmetry feature space. When the posture was supine, both T D-RL and T D-CH were close to zero (close to the origin of the space). When the posture was left lateral, T D-RL was negative and T D-CH close to zero. When the posture was right lateral, T D-RL was positive and T D-CH close to zero. When the posture was prone, T D-RL was close to zero and T D-CH was negative (close to  the bottom of the space). The two major lower-body postures, the fetus and the log positions, are described by the leg symmetry feature space. As described above, L D-RL and L MAC were used to compose the leg symmetry feature space. When the posture was of the fetus position, L D-RL was large and L MAC was small (close to the right-hand bottom of the space). When the posture was of the log position, L D-RL was small and L MAC large (close to the left-hand upper limit of the space).

Measurement of the sensing gap for sensor deployment on the bedsheet
Deploying fewer sensors in a fixed sensing coverage will lead to sensing gaps between sensors. We deployed a 16 × 10 pressure sensor array for the upper body to avoid creation of a sensing gap, as shown in Fig. 13. Sensing gaps occur when sensors are not adjacent to each other in sensor deployment. In such cases, there exist areas between sensors in which the system does not work when a subject's body part is smaller than the area and falls within the area. In sensing coverage in which the size of the sensors is fixed, deployment of fewer sensors will create larger sensing gaps. Figure 14 shows that many sensing gaps exist between sensors. An area that contains a sensor and a gap is defined as a sensing grid. When a subject sleeps on the bedsheet, the major pressure distribution is contributed by the chest and hips; in other words, pressure sensors have a low utilization in the leg area, which is a waste of resources. To address the cost issue, previous studies usually deployed fewer sensors. A typical example is shown in Fig. 15a, which depicts a pressure sensor array deployed on a bedsheet. However, fewer sensors in the sensing area enlarge the gap between sensors. To resolve this problem, we adopted a Panasonic AMG8852 Grid-EYE [21] infrared array  Figure 15 presents a comparison of a typical pressure sensor deployment (Fig. 15a) and an infrared array sensor deployment (Fig. 15b). The sensor deployment area is crisscrossed with dotted lines that form sensing grids. To simplify the calculation process, the pressure sensors shown in Fig. 15a were attached to the upper and left dotted lines of each sensing grid, while in practice the force applied to the pressure sensors diffuses radially outward owing to the effect of the mattress covering the sensors.

Spatial sensing resolution
The sensing gaps created during pressure sensor and infrared array sensor deployment were compared. In order to make a fair comparison, a similar spatial resolution was considered for both the FSR and the Grid-EYE sensors. In [19], spatial sensing resolution was proposed as a tool for comparison of three different pressure sensor deployments. For the pressure sensor deployment shown in Fig. 15a, the spatial sensing resolution is shown in Eq. (9): However, the spatial sensing resolution for the deployment of an infrared array sensor of the same coverage (Fig. 15b) is given by: The sensing gap of the pressure sensor array deployment was calculated for the array shown in Fig. 15a as 0.1 m. Sensing grids of every two adjacent or diagonal pixels of the Grid-EYE system adjoin each other, sharing a common border or endpoint. The sensing coverage of each pixel will change if the distance between the Grid-EYE array and the subject varies, meaning that there exists no obvious sensing gap in the Grid-EYE pixel array.

Signal detection threshold
The signal detection threshold is the largest body part of a subject that is undetectable by the deployed sensor(s) in a sensing system. In this study, the signal detection threshold was applied to evaluate the capability of forming a normal-size human leg contour on the sensing bedsheet. In common usage, a threshold is the minimal amount of a subject's feature necessary to be detectable by the sensory receptor or system [28]. When value of the feature (e.g., brightness, weight, temperature) reaches or exceeds a certain threshold, the system will detect the existence of the feature. In our evaluation of a sensing bedsheet system, the maximal undetectable size of the subject within the sensors was considered "the worst case" in the sensing grid. The signal detection threshold in a sensing grid is described by Eq. where A u is an undetectable area of a subject's body part.
The quality of sensor deployment can be examined using the value of the signal detection threshold. When the undetectable area reaches the maximal value, which is its signal detection threshold, the ratio of the size of the maximal value to the sensing grid reflects the quality of sensor deployment. In the equation, A u is the undetectable area of the subject's body part on the sensing bedsheet. The area will keep increasing until it hits a sensor and is detected by that sensor. Therefore, the largest size of body part that is not detected by any of the sensors, which is the max(A u ) , is the signal detection threshold. The signal detection threshold is better when the value is smaller.

Pressure sensor array deployment
In a pressure sensor array deployment, the sensing grid is the minimum value of a quadrilateral area that includes the center of every four sensors of an in-bed pressure sensor deployment. "The worst case" position means that a body part of the subject falling within the area of four pressure sensors has the largest undetectable feature area. The subject touches the four sensors but does not overlap any of them, rendering it undetectable. A sensing unit containing four quarter circles and the largest-but-not-overlapping subject's body part is shown in Fig. 16; this figure is another demonstration of the max(A u ) , in which the sensing gap and the subject's body part are placed in the middle of the sensing grid. According to the experimental results, the signal detection threshold (i.e., the maximum undetectable area of a body part of the subject) was approximately 180.83cm 2 .

Infrared array sensor deployment
In an infrared array sensor deployment, the signal detection threshold is the smallest detectable area of a body part of the subject within the sensing grid. Our Grid-EYE experiments showed that a shank was undetectable when it fell in one-fifth (20 cm 2 ) of the sensing grid, but detectable in one-quarter (25 cm 2 ) of the sensing grid. Similarly, when two fingers (18 cm 2 ) fell in the Grid-EYE sensing grid, they were undetectable; however, three fingers (27 cm 2 ) were detectable in the sensing grid. According to the experimental results, the max(A u ) for the Grid-EYE system was approximately 25 cm 2 . Table 1 presents a comparison of the pressure sensor array (FSR sensors) and infrared array sensor deployment in the leg area. The calculations showed that under the same spatial resolution, the signal detection threshold of the Grid-EYE system was much better than that of the pressure sensor array (FSR sensors). A single infrared sensor covers a larger sensing area than 49 FSR sensors. The pressure distribution of the legs is very small, and this decreases the accuracy of the FSR-based method. Hence, combining FSR sensors and infrared array sensors renders posture recognition effective and cost-efficient.

Results and discussion
We performed experiments with different subjects based on their body weight: light (around 40 kg), medium (around 60 kg), and heavy (around 80 kg). The sleeping posture recognition accuracy of the proposed method was as shown in Table 2; the average sleeping posture recognition accuracy was 88.05%.
We compared the number of deployed sensors and the cost of the method with systems described in previous studies, as shown in Table 3. In [17,18], 8192 FSR sensors were deployed, achieving an accuracy of 90.78% and 83%, respectively. The cost of 8192 FSR sensors was approximately 8850 USD. Our proposed scheme employs 160 FSR sensors for the upper body and one infrared array sensor for the lower body, the total cost of which was approximately 950 USD. We achieved an accuracy of 88.05% using very few pressure sensors. In the experiment, an infrared array sensor system was chosen instead  of a pressure sensor system for detection of the position of the lower part of the body. When sensor deployment only is taken into consideration, the cost of an infrared array sensor system is about one-tenth that of a pressure sensor system. Additionally, in comparison to an infrared array sensor, pressure sensor systems have a much higher maintenance cost, as patients do not come into physical contact with infrared sensors. A hybrid deployment consisting of an infrared array sensor for the lower body effectively reduces the deployment cost. The proposed sensor deployment was found to be much more cost-effective as compared with the number of pressure sensors deployed in [17] and [18]. The experimental results indicated that our proposed system achieved highly accurate posture recognition at a much lower cost.

Conclusions
In this study, we designed and tested a sleeping posture recognition scheme consisting of a pressure sensor array and an infrared array sensor. We deployed 16 × 10 FSR sensors for the upper body and a Grid-EYE infrared array sensor for the lower body to create a bed sensing system. We applied the symmetry of feature space to distinguish six types of sleeping body posture. The proposed method overcame lying position variations by determining the middle point and middle axis of the body. The experimental results showed the accuracy of posture recognition to be 88.05%.
As we deployed FSR sensors only on the upper bedsheet, and employed a single infrared array sensor for the lower body, the novel scheme was cost-effective. Moreover, the proposed scheme avoided sensing gaps between sensors, which is a common issue but is usually ignored in low-cost pressure sensor systems. The Grid-EYE infrared array sensor deployment balanced the trade-off between recognition accuracy and sensor cost. In addition, the Grid-EYE sensor deployment negated privacy concerns.