The year 2009 saw the start of three dimensional (3D) consumer televisions (TVs) and the launch of Avatar in December 2009 that completely changed the mindset regarding 3D content, whether movies, documentaries, or games. Despite criticism and pessimistic reviews from various sectors that suggested that the interest in 3D would soon die, it has been reported [1] by the Information and Analytics Provider Survey (IHS) in December 2012 that “the global 3D consumer market is growing across major platforms including cinema, home video, pay-TV, and video-on-demand (VOD)”. In addition, the success of Avatar has resulted in a multitude of 3D movies since 2010. Currently, there are 37 TV channels dedicated to full 3D content in the world. All this is resulting in an increase in 3D content being available on Blue-ray discs [2].
However, there are consistent reports of discomfort while viewing movies and video content in 3D mode [3]. The types of discomfort reported include dizziness, headaches, nausea etc. In addition, there had been reports where the viewers have reported that they are not able to see in 3D continuously, i.e. they can see in 3D for a while followed by 2D viewing and then again in 3D and so on so forth. According to a report in CNBC [3] “viewers of 3D may experience nausea (nausea, increased salivation, sweating) and disorientation (dizziness, vertigo, fullness of head).” Theoretically, viewing in 3D mode should have more content as compared to viewing in 2D mode and hence be closer to our natural way of viewing with two eyes [4]. That should result in viewer satisfaction rather than dissatisfaction. So is it the current 3D consumer technology that is the main cause of this dissatisfaction? Hence, the motivation of this research is to investigate the various 3D consumer technologies that may cause such discomforts leading to consumer dissatisfaction.
In this manuscript, we only focus on the two current commonly available consumer 3DTV technologies which are based on stereo method. Hence, we limit our discussion to 3D viewing due to stereo technology. One is based on active shutter glasses technology and is referred as 3DA in the manuscript. The other is based on passive polarized glasses and is referred as 3DP in this manuscript. However, there are various other passive 3D technologies that are not included in this research, e.g. passive 3D technology based on color to separate the images for each eye. We have selected the passive polarized glasses based on stereovision that is currently the most widely available in consumer 3D TVs [4]. The traditional TV is referred as 2D in the manuscript. However, 2D does not imply that there is no 3D information in traditional TVs. 3D effect had been generated through methods like vanishing lines on traditional TVs.
As mentioned above, with respect to current consumer 3D TVs, there is two 3D viewing technologies that are offered by various consumer electronics consortiums. Both of these technologies are based on stereoscopic vision, i.e. based on two images being viewed by each of our eye [5]. We see the world in 3D because of binocular vision. The images captured by each of the eye are slightly shifted from each other due to the distance between the two eyes and eventually this disparity in the two images result in the 3D vision [6]. Two types of glasses with divergent technologies have found their way to the consumer market: active shutter glasses and passive polarized glasses.
The passive polarized glasses are similar to the ones that are being used in cinema [6]. However, there is slight difference, i.e. the two images (required for 3D perception) are projected simultaneously on the 3D TV while they are projected alternatively in cinemas [6]. The polarization technique that is being used in 3D passive system may be linear or circular. For linearly polarized glasses, two images with different polarization (one horizontal and one vertical) are projected on the screen simultaneously which may result in lower resolution. There are many ways to project the images simultaneously, i.e. side by side, up and down, interlaced lines etc. [7]. Each of our eyes sees only one image because of the different polarization for each side of the glasses. This will produce 3D effect while viewers kept their head straight. The circular polarization also works under the same principle except that its viewing angles are more flexible than the former technique. Both techniques are called passive because no synchronization is required with the 3D screen and hence the passive glasses are also lightweight because no sensors and batteries are required [8,9].
The active shutter glasses, on the other hand, are synchronized with the 3D screen through a sensor and need circuitry and corresponding batteries which increase the weight of the glasses [10]. The two images for each of the eyes are projected on the screen one after the other and corresponding shutter is opened in the glasses through synchronization with the sensor on 3D screen. This opening and closing of shutter in the glasses is so rapid (generally 240 times per second) that our eyes see it as continuous video stream [8].
The main goal was to compare the two 3D technologies (passive polarized―3DP and active shutter glasses―3DA). The study was conducted with the use of electroencephalogram (EEG) and electrocardiogram (ECG) as well as viewers’ feedback to know the causes of discomfort subjectively and validate by the objective measurements i.e. EEG signal analysis. Both these methods (EEG and ECG) are capable of providing information on cerebral cortex and cardiovascular activities due to their high temporal resolution. The EEG and ECG data were recorded and analyzed in order to understand our responses to the 3D viewing by utilizing 3D movie clips. The literature review mostly mentioned subjective studies for the complaints of discomfort during 3D content viewing. This is the first study that relates eye blinks to the repeated loss of synchronization that causes discomfort and also shows the corresponding brain activity to prove this point. Experiment was done on both 3DA and 3DP technologies. The synchronization of the alternate images to both eyes is disturbed due to the eyes blinks. This desynchronization of alternate images due to eyes blinks causes the high discomfort in 3DA technology, which is not reported in the literature and the novelty of this study.
Our study is hypothesis-generating study and the EEG activation across various brain regions need to be discussed independently. However, due to availability of depth information in 3D viewing as compared to 2D, we expect that there will be higher engagement of working memory and visual attention in the case of 3D viewing than in 2D. From the EEG perspective, higher theta and alpha activity in the frontal region indicates that brain is actively involved in working memory processes (global processing) [11,12]. As for the ECG results, we assumed that the subjects will tend to have lower arousal state when viewing in 3D mode compared to 2D since the brain is highly engaged to processing extra information found in 3D contents.
In this research, we investigate the two 3DTV stereoscopic consumer technologies with following research questions: (i) What are the main causes of discomfort while viewing in 3D?, (ii) which technology produces better 3D visualization effects in our brain while causing minimum discomfort?, and (iii) Does viewing in 3D results in better visualization of the scenes in comparison to 2D? The better visualization refers to feeling of smooth movement in the video with better interpretation of the scene (since the scene is in 3D) which may result in high activation at the occipital, frontal and temporal lobes. Occipital is related to vision, temporal to memory processes and frontal to scene and objects interpretation. We attempt to answer these questions through a comprehensive study undertaken over a 1-year period by using EEG and ECG methods.
Related work
3D movies are not new and existed since early 1900’s [6]. However, it didn’t take off because of the high cost in producing and displaying 3D content, the low quality and lack of 3D standards. The low quality was because of the red/green anaglyph format with anaglyph glasses that were being used in early days of 3D [7]. The mid 1980’s saw the start of 3D content on IMAX (Image MAXimum) theatres since they had the advantage of large screens. The 3D content including documentaries and movies became successful on IMAX screens since 1990’s. By 2004, more than 50% of IMAX theatres were showing 3D content but relatively the overall number of IMAX theatres remained small [13]. With the transformation from analog to digital technology in late 1990’s and early 2000, 3D technology became affordable with quality both in cinemas as well as on personal 3D TVs. However, consumers as well as investors’ interest was still not there because of the lack of 3D content as well as confidence in the 3D technology. Something was required to boost the consumer’s confidence in the 3D technology and the investor’s confidence in producing the 3D content. That came in the shape of the 3D movie “Avatar” in December 2009 [14].
After this brief introduction, the following paragraphs summarize research related to various aspects of 3D processing in the brain, video content, memory and attention. First, the various frequency bands of the EEG signal are introduced. The EEG frequency bands include Delta (0-4 hertz or Hz), Theta (4-8 Hz), Alpha (8-12 Hz), Beta (12-30 Hz) and Gamma (>30 Hz) [14]. However, these ranges are not absolute and there is difference of opinion among the researchers. In addition, Alpha, Beta and Gamma had been further subdivided [11]. These frequency bands had been associated with various mental conditions [11,12,14]. For example, higher delta power in the frontal lobe indicates drowsiness, dizziness etc. [15]; while higher alpha power in the occipital lobe is associated with relaxation [12]. In general higher theta power is related to memory and attention processes while higher beta and gamma powers are associated with localized processes [12].
The researchers have attempted to understand how the brain perceives depth perception in 3D [16] especially with regards to our stereoscopic vision. They found that there are neurons residing in the visual cortex that decode the disparity from the two images viewed by each eye in order to get depth information in the scene [16,17].
Attention and working memory are two important concepts related to viewing and understanding of video streams [18]. Two different ways (i.e. bottom-up and top-down processing) have been described by researchers that our brains use to attend to surrounding items [19] and both of them exist for watching video content especially the 3D content. The brief storage of information and the ability to manipulate available information is the function for the working memory [20]. Few studies found frontal theta and alpha activity in a working memory task [12,21].
There are concerns about the effects of 3D stereoscopic technology on humans. Among the most common complaints received from viewers include the visual motion sickness, visual fatigue and eye strain [22]. Previous experimental evidences showed that any sort of distortion in stereoscopic images can cause visual fatigue [23]. In 3D passive, it is possible that viewers may experience cross-talk, where the right eye sees dim images intended for the left eye or vice versa due to the imperfection of polarizing filter that accidentally allows light to go through. As for in 3D active, flickering problem may occur if the switching frequency between images is slow, and as a result, the production of 3D perception is not smooth. In addition to that, the active shutter glasses require batteries to run, which adds to their weight that will somehow cause general discomfort if worn for too long [24]. In addition, as mentioned earlier in Introduction section, the types of discomfort reported include dizziness, headaches, nausea (nausea, increased salivation, sweating) and disorientation (dizziness, vertigo, fullness of head). In order to minimize these effects, our exclusion criteria for recruitment included those with motion sickness and the experiment viewing time was reduced to 20 minutes to reduce visual fatigue. In addition, all the equipment was repeatedly checked to avoid the effects of flickering as well as changes in the quality of polarizing filters.
Proposed theory for discomforts while 3D viewing
The answer to the source of discomforts because of viewing in 3D mode lies in the working principles of the 3D technology. As mentioned earlier, for passive polarized glasses, the two images are simultaneously projected with different polarization (horizontal and vertical) on the 3D TV. Each of our eyes views one image because of the different polarization for each side of the passive polarized glasses. Although the resolution may decrease, the phenomenon of watching the two images simultaneously is similar to how our eyes naturally perceive images, i.e. our eyes see two images simultaneously with a slight shift because of the distance between the two eyes [8,24,25].
On the other hand, the active shutter glasses work on the principle of alternatively opening the shutter of the glasses [8,10,24]. At any one moment, only one eye can see while the other eye cannot because the shutter of the glasses is closed at that moment. However, the alternate opening and closing of the shutters is done at a very high speed, i.e. approximately 240 times per second, so the effect is that of a continuous video stream [6,24]. This requires synchronization with the 3D screen that is achieved at the cost of additional sensors. This is also similar to how the images are projected in the cinema although passive polarized glasses are used in cinema. Hence, the two images (required for 3D perception) are projected alternatively on the cinema screen with different polarization corresponding to the polarization of each eyepiece of the passive polarized glasses [6,8]. Hence, at any one moment, only one eye can see. This is what makes it different from the consumer 3D TV technology based on passive polarized glasses where the images are projected simultaneously on 3D TV utilizing passive polarized glasses [8,24].
Based on above discussion, we categorize the 3D technology based on how the two images (required for 3D perception) are projected on the screen. The first category involves projecting the two images (required for 3D perception) simultaneously while the second category involves projecting the two images (required for 3D perception) alternatively. Consumer 3D TV technology utilizing passive polarized glasses fall under first category while the active shutter glasses based 3D TV technology and the 3D cinema technology based on passive polarized glasses fall under the second category. However, the following three problems arise because of the second category:
-
1)
Generally, it takes on average about 100 milliseconds (msec) for the images to be transferred from retina to visual cortex through the optical nerve [26]. An active shutter glasses open and close shutter at 240 times per second (refresh rate for 3DA TV is 240 hz in 3D mode and 60 hz in 2D mode), i.e. one image is viewed for approximately 4 msec. To see in 3D, two images are required to find disparity that gives the depth information. If the first image reaches at retina at time ‘t’ then its corresponding second image will reach retina at time ‘t + 4’ msec. Hence, the mechanism of decoding the 3D information is deviated slightly, i.e. the two images are not processed simultaneously by the brain rather there is a finite time difference between the processing of the two images. This finite time difference is minimum of 1 msec and can go up to 4 msec. In general, this 4 msec does not have perceptual impact as time duration is very short and the video stream appears to be continuous. However, no one has reported any related research with respect to the effect of displaying two stereo images, one after another, for each of the eye separately with this time interval of 4 msec. Hence it may be investigated in future research whether this time difference, though very small, introduce effects that may not result in true 3D perception and may be a cause of discomfort.
-
2)
The active shutter glasses assume that there are no eye blinks. However, we know that eye blinks are natural phenomenon and these eye blinks result in the leaking problem, i.e. de-synchronization between eyes and the two images that are required for viewing in 3D. On average [27], we blink eyes 10.3 ± 3.1 times per minute, inter-eye blink interval is 6.4 ± 2.4 seconds and the blink duration is about 0.1 second or 100 msec. Let us suppose that we watch one image at time ‘t’. Assume that when the second corresponding image (that is required for 3D vision) appears at ‘t + 4’ msec, our eye blinks and the blink during is on average 100 msec. As a result, after the eye blink, there are following four possibilities;
-
a.
Same eye (which saw the image before the eye blink) will see the next image. The next image can be either;
-
i.
the new image (the first of the two images that are required for 3D vision), or
-
ii.
the second image (of the two images that is required for 3D vision).
-
b.
Other eye (for which the shutter was closed before the eye blink) will see the next image. The next image can be either;
-
i.
the new image (the first of the two images that are required for 3D vision), or
-
ii.
the second image (of the two images that is required for 3D vision).
For any of the above mentioned four possibilities, there will be a break in the 3D visualization. This break will have more profound effect and will last longer for conditions a(ii) and b(i) because it will take another image for resynchronization. Overall, the effect will be breaks in 3D visualization, i.e. user may see in 3D for some time and then will see in 2D and then again in 3D and so on. Figure 1 shows one of the above mentioned scenarios. The scenarios are animated and can be seen in the ‘3D Visualization scenario 1’ as Additional file 1, and ‘3D Visualization scenario 2’ as Additional file 2.
Based on above discussion, it is quite clear that synchronization of the two images (required for 3D perception), with respect to each of our eye, is very important for 3D visualization. However, this synchronization is lost due to an eye blink. This de-synchronization will result in loss of 3D visualization. Again brain has to synchronize after an eye blink. But eye blinking is a continuous phenomenon. Hence, there will be synchronization followed by de-synchronization and then again synchronization and so on so forth. This process will go on until the very end of viewing the 3D contents. As a result, there will be eyes fatigue and stress on various brain lobes specifically on occipital lobe. All this can lead to various types of discomfort like dizziness, headache, nausea etc. The simplest result will be the breaks in 3D viewing, i.e. the viewer can see in 3D for a moment followed by viewing in 2D and then again followed by viewing in 3D and so on so forth.
-
3)
Weight of the 3D active shutter glasses is increased because of the extra circuitry and the corresponding batteries. This addition in weight may cause eye fatigue for the viewer that may result in increase in eye blinks. Increase in eye blinks mean more breaks in 3D visualization. It was reported in the subjective feedback that the subjects feel fatigues because of the heavier glasses in 3DA as compared to lighter glasses in 3DP.
Hence, it can be concluded that first category that involves passive polarized 3D TV technology results in better 3D visualization with minimum discomfort as compared to the second category involving active shutter glasses technology and the cinema technology. In fact, it is the second category that is responsible for the causes of discomfort due to viewing the two images (required for 3D perception) alternatively rather than simultaneously. This hypothesis is validated by the following sections that discuss the brain responses to 3D movie clips. In the next sections, 3DA refers to the active shutter technology from second category (alternative images projection) while 3DP refers to passive polarized technology from first category (simultaneous images projection).