A new method to measure higher visual functions in an immersive environment
© Zito et al.; licensee BioMed Central Ltd. 2014
Received: 10 April 2014
Accepted: 22 July 2014
Published: 28 July 2014
Higher visual functions can be defined as cognitive processes responsible for object recognition, color and shape perception, and motion detection. People with impaired higher visual functions after unilateral brain lesion are often tested with paper pencil tests, but such tests do not assess the degree of interaction between the healthy brain hemisphere and the impaired one. Hence, visual functions are not tested separately in the contralesional and ipsilesional visual hemifields.
A new measurement setup, that involves real-time comparisons of shape and size of objects, orientation of lines, speed and direction of moving patterns, in the right or left visual hemifield, has been developed. The setup was implemented in an immersive environment like a hemisphere to take into account the effects of peripheral and central vision, and eventual visual field losses. Due to the non-flat screen of the hemisphere, a distortion algorithm was needed to adapt the projected images to the surface. Several approaches were studied and, based on a comparison between projected images and original ones, the best one was used for the implementation of the test. Fifty-seven healthy volunteers were then tested in a pilot study. A Satisfaction Questionnaire was used to assess the usability of the new measurement setup.
The results of the distortion algorithm showed a structural similarity between the warped images and the original ones higher than 97%. The results of the pilot study showed an accuracy in comparing images in the two visual hemifields of 0.18 visual degrees and 0.19 visual degrees for size and shape discrimination, respectively, 2.56° for line orientation, 0.33 visual degrees/s for speed perception and 7.41° for recognition of motion direction. The outcome of the Satisfaction Questionnaire showed a high acceptance of the battery by the participants.
A new method to measure higher visual functions in an immersive environment was presented. The study focused on the usability of the developed battery rather than the performance at the visual tasks. A battery of five subtasks to study the perception of size, shape, orientation, speed and motion direction was developed. The test setup is now ready to be tested in neurological patients.
Visual perception is a complex process. In the brain, the primary visual cortex (V1) receives input arriving in an orderly manner from the retina. The properties of objects, their shape, color, motion and location in the world, are then represented beyond V1 in specialized extrastriate areas [1, 2]. It is here that the brain assesses the crucial defining qualities of objects and collects, processes and collates visual information to form a coherent whole. This high-level visual perception embraces the complex network of higher visual functions (HVF).
It is known that an impairment of HVF after unilateral brain injury may influence the perception of the visual world , for example the capacity to recognize visually presented objects (visual agnosia)  or the capacity to perceive visual motion (akinetopsia) . Impaired HVF are usually tested with paper pencil tests. Examples of such tests are the “Visual Objects and Space Perception battery” (VOSP) , where the test persons are asked to name visually presented objects, distinguish them from nonsense pictures and recognize the spatial location of figures, or the “Birmingham Object Recognition Battery” (BORB) , where images of animals together with meaningless figures are presented to the participants and they have to identify the meaningful ones. Another common test used for spatial perception is the “Benton judgment for line orientation” , in which the subjects compare straight lines oriented in different directions with a pull of reference lines and they have to find the line in the pull that matches the orientation of the line under test. Motion blindness is often tested with Random Dot Cinematograms , patterns of dots moving in a direction that has to be recognized by the subjects. In all these tests, the results are usually the number of items correctly addressed at the different tasks and the diagnosis is given based on this number and on a comparison with normative data.
However, there are no computer-based tests that return quantitative data as indicators for the magnitude of unilateral impairment. Furthermore, standard neurological tests do not discriminate objects perceived in the right visual hemifield and in the left one and, thus, the degree of dynamic interaction between the two brain hemispheres. Therefore a new measurement paradigm (HVF test battery) involving real-time comparison of images in the two visual hemifields was developed and piloted in healthy volunteers.
The HVF test battery was implemented in an immersive environment such as a hemispherical screen, whose peculiar characteristic is a field of view of ± 90° in both horizontal and vertical axes. Since HVF disorders are often related to visual field defects , the hemispherical screen presents the advantage of testing HVF in the region of the visual field that is still intact. Furthermore it allows for consideration of the effects of central vision and peripheral vision while accomplishing the tasks.
In order to implement the HVF test battery, interactive images need to be projected onto a non-flat surface. Several methods were analysed , then a mathematical model and a fully measured approach were implemented to pre-distort images to look undistorted once they are projected onto the hemispherical screen. The two methods were compared and the best solution was selected for the final implementation.
The pilot study on the HVF test battery was at last conducted. The aims were to assess the performance level at the proposed tasks in case of intact HVF, as well as to collect feedback from the participants regarding the usability of the battery itself.
The laser projector represents an advantage because it projects images in focus in all their pixels onto the cupola, even if the distance projector-screen varies point by point. The spherical mirror also plays a crucial role because it allows for coverage of the entire projection surface and the whole structure to be globally compact.
In order to achieve projected images which look undistorted onto the cupola, a warp algorithm was needed . Two approaches were implemented and compared.
A mathematical model was previously investigated to warp images : given the geometry of the system, the model computes, per each point on the cupola, where the corresponding point is reflected onto the spherical mirror and, therefore, where it is located on the projector. Then it distorts the images accordingly to the new computed locations of the points.
The projected images were captured by the external camera and the two outputs were correlated, respectively, to the original image. In particular the structure similarity index (SSIM) [15, 16] was used to measure the fidelity of the distortion. Since the distortion only affects the structure of the objects in the scene and not the brightness, the SSIM resulted as a good choice to explore the structural information in the images separating it from the influence of the luminosity of the light source (projector).
Participants and ethical approval
57 healthy volunteers (32 men and 25 women aged 22 to 73) were recruited to participate in this study (age M = 36.1; SD = 12.9). All subjects had a Master Diploma at least, a visual acuity > 0.8, corrected with lenses if needed, and a MoCA score > 26. Moreover they were all used to handling computers.
The usability testing was carried out in accordance with the latest version of the Declaration of Helsinki. Ethical approval was given by the Ethics Committee of the Canton of Bern, Switzerland.
All the five subtasks are performed under central fixation , which is controlled using the infrared camera and the infrared LEDs integrated into the perimeter device. If the test person moves his gaze out of an allowed region of ± 1° centered on the fixation marker, the images under balance disappear and they appear again when the marker is fixating again.
The presentation of the images and the recorded answers were processed on a Intel® Core™ i5 (3.10 GHz) with Windows 7 operating system (Microsoft Inc.) and within a Matlab script using the Psychophysics toolbox . The monitor of the computer was a 24’ screen with resolution of 1920x1080 pixels.
Prior to the examination all participants gave written informed consent. After performing several training sessions to become familiar with the tasks, the subjects started the real measurement (about 20 min), in which five repetitions per subtask were made. The experiments were carried out in a dark, noise-controlled room.
Mean ± SD
1. How mentally demanding were the tasks?
3.3 ± 1.5
2. How physically demanding were the tasks?
2.6 ± 1.9
3. How hurried or rushed was the pace of the tasks?
2.6 ± 1.3
4. How successful were you in accomplishing what you were asked to do?
4.7 ± 1.5
5. How hard did you have to work to accomplish your level of performance?
4.3 ± 1.7
6. How insecure, discouraged, irritated, stressed and annoyed were you?
1.8 ± 1.2
7. Overall, how satisfied are you with the ease of completing the tasks?
5.5 ± 1.3
8. How satisfied are you with the amount of time it took to complete the tasks?
5.6 ± 1.5
9. How satisfied are you with the support information when completing the tasks?
6.1 ± 1.4
10. How confident are you that the tests measure skills you need in everyday life activities?
5.0 ± 1.6
The outcome measures of the HVF test battery, per each subject, were the mean values, out of the five repetitions, of the absolute differences between the reference images and the images under manipulation at the very time when the confirmation button was pressed. In particular, for the line orientation subtask, it was the angle between the two oriented lines, for the shape perception subtask the difference in eccentricity of the ellipses, for the size perception subtask the difference between the radii of the circles, for the speed perception subtask the difference between the speeds of the patterns, and for the perception of motion direction subtask the angle between the two directions of motion of the patterns.
The outcome of the SQ was the level of agreement of the participants to the questions listed in Table 1.
Quality of the distortion
Fully measured approach
Performance at the HVF test battery
Results of the pilot study divided into the five subtasks
Final difference (mean)
Final difference (SD)
Orientation (N = 57)
Shape (N = 57)
Size (N = 57)
Speed (N = 57)
Direction (N = 57)
Table 1 summarizes the results of the SQ.
The ASQ score, calculated as the arithmetic mean of the three questions 7–9 , was 5.7 ± 1.4 which means that the participants liked the HVF test battery and they were particularly happy with the support information given by the instructors.
The score to the questions 1–6 showed that the mental, physical and temporal demands were well tolerated by the participants (questions 1–3), the self-estimation of the performance level (question 4) was positive, the effort needed to complete the tasks (questions 5) was also positive, but close to the middle point, the level of frustration (question 6) was low. Question 10 also had high positive feedback from the test persons, who agreed that the HVF test battery measures skills needed to correctly perceive the visual world.
The aims of this study were, first, to compare the image quality of two different projection approaches and, second, to conduct a pilot study to investigate the usability of the novel computer-based test to measure HVF in a hemispherical environment.
Quality of distortion
The analysis of the two projection approaches showed that the fully measured method returns better images, in terms of quality of distortion, compared with the mathematical one.
The mathematical model presents the main advantage of being an exact solution for the distortion problem. The distortion is described by a system of two equations of fourth degree, where the intrinsic parameters can be calculated given the geometry of the system, for instance parameters like the radius and the center of the mirror, the radius and the center of the cupola and the position of the projector . Among the four solutions of the equations, one and only one is acceptable for the physical problem. Once the distortion is computed, it can be easily applied to the projected images with a computational time of about 1 minute. However the mathematical model has several disadvantages, it is limited to only simple geometries and if the shapes of the cupola and the mirror are described by a higher degree polynomial, the required computational power becomes heavy and the solution would probably be approximated. Furthermore, if the mirror has a shape that cannot be described by a mathematical model or the projector is placed in an unknown position, it is impossible to compute the correct distortion. The model is also very sensitive to its own parameters, if one of those is not correctly set, the distorted images do not fit the hemispherical screen and a manual stretch is needed. In addition, the projected images have to be aligned in order to match the right position onto the cupola. But the strongest simplification of the mathematical model is that the projector is considered a point source of light, even if it is a physical device with a source screen of about 1 cm2. This introduces some irregularities which are more pronounced in the corner of the cupola. In order to correct them, a compensation can be applied. Under the assumption that the irregularities are small enough to be corrected with a linear compensation, a picture of the scene can be taken by a camera placed in front of the cupola and a direct comparison between this picture and the original image can be done. According to this, a map of the projection distortion can be created and used for the correction .
In contrast, in the fully measured approach, no mathematical hypothesis is made and the only assumption is that the observing camera is placed in the exact location where the viewer is. This approach presents the advantage of being independent from the location of the mirror and the projector, they can be ideally placed wherever it is more convenient without affecting the goodness of the distortion. Moreover the images result already aligned with the mirror and the hemisphere because they are computed directly on them. The weak point of this approach is that the positioning of the camera in front of the cupola is crucial. The camera, in fact, represents the view point of the observer and, if it is not perfectly directed towards the center of the hemisphere, the computed distortion is wrong. In this study, an infinite point of view for the observer was chosen and, as a consequence, the projected images looked like they were projected on a flat screen and not spread on a hemisphere. This was a good choice because, in the cupola, the test person has no reference point for 3D space and he performs the tasks in the same way he would on a flat screen.
Another weak point of the fully measured approach is that it requires a long calibration because the match between the two reference systems is done point by point and, in order to achieve the maximal resolution, the number of points must be equal to the resolution of the projector itself.
Processing time can be reduced by decreasing the number of projected points and interpolating the missing information, but for fine details it is not recommended. However, once the calibration is done, the images can be distorted in about two minutes.
Despite the high fidelity of the distortion shown for both methods in Table 2, the mathematical model resulted to be slightly worse than the fully measured approach because of the shortcomings mentioned above. The SSIM could be further improved for both methods considering that in the original images the shadows of the projector and the mirror are not present, and this could result in a mismatch that turns the SSIM down.
Performance at the HVF test battery
The pilot study on the HVF test battery measured the level of accuracy of healthy subjects in comparing objects presented in the two visual hemifields, as well as the usability of the battery itself. Such a value of accuracy has never been measured before.
The single subtests for line orientation, size and shapes of objects, motion direction and speed were selected according to the differentiation of the visual analysis in the various brain areas: orientation of objects is analyzed in the Brodmann Area 17, size and shapes of objects in the “what” stream, in the Brodmann Areas 19 (anteromedial), 37 and 20; motion is analyzed in the “where” stream, in the lateral part of the Brodmann Area 19 . It is clear that, according to the location of an eventual lesion, different properties of objects could result as impaired. However, in healthy subjects the HVF are intact and the only limiting factor could be the resolution of the eye, about 1 arcminute . In relation to this datum, the accuracy of the test persons was slightly lower. This can be explained by taking into account that the maximum resolution of the human eye is in the fovea and it drastically decreases out of it , but in the HVF test battery the fovea is pointing at the marker point and not at the images under comparison. Since the images are presented at an eccentricity of 20°, our results are in line with Hunziker, who showed a decrease of the perceived sharpness of an image of 90% in correspondence of such eccentricities . More in depth, the results from the Shape and Size subtasks were very similar because they investigate two aspects of the same skill, recognition of objects. A study on the redundancy of these subtasks could clarify the necessity of two separate tests but, given the purpose of this study, the integration in a clinical setting, redundancy can be helpful to generate more consistent outcomes. Similar considerations could be done for the Speed and Motion direction subtasks but, due to the different types of measure, a direct comparison is not possible. It is worth noting that the results of the Orientation and Motion direction subtasks were very different from each other although they both measure orientation. This can be explained because, according to the brain anatomy, judgment of line orientation is an easier task accomplished in the very first stages of visual perception, while motion analysis requires more specialized neurons in the posterior parietal cortex. This could be the reason why performance of the Motion direction subtask was worse than the one at the Orientation subtask.
The responses to the SQ (Table 1) showed that the HVF test battery was very well accepted by the test persons, the over-all feedback was very positive, and there were no drop-outs. In particular, question 10 measured the face validity of the subjects with regards to the battery. This is an important requirement for the usability of the test. The questionnaire measured the individual impressions of the participants, such as the mental and physical demand or the self-estimation of the performance, which are subjective but, since the test is designed to be integrated into a clinical setting, the information about the motivation of the test persons is relevant because it directly correlates with their attention to performing the tasks and, therefore, with the performance itself .
It is expected that patients with unilateral brain lesion in the region of the visual pathway would have worse performance compared with the age-matched healthy controls and that the performance itself negatively correlates with the degree of the lesion, the worse the performance, the higher the impairment. This can be a limitation in case of bilateral impairment, where both hemifields are affected and a comparison between affected vs. non-affected performance cannot be done. In this case, age-matched normative data could be used to judge the subject’s performance and screen for impairment. Luckily, the majority of clinical cases we see involves unilateral brain lesions (i.e. after ischemic stroke).
This paper presented a usability study of a new method to measure HVF (HVF test battery). The purpose of the study was to investigate healthy volunteers’ acceptance of the developed battery rather than their performance at the visual tasks. The HVF test battery consists of five subtasks where shape, size, orientation, speed and direction of motion of objects presented in the right visual hemifield and in the left one are dynamically compared. The HVF test battery was implemented in a hemispherical environment, whose characteristic is a ±90° field of view in both horizontal and vertical directions. A hemispherical screen allows for testing of HVF in relation to peripheral and central vision, as well as in a region of the visual field still intact after eventual brain lesions. The results of the pilot study on healthy subjects are in line with literature, the usability study showed a high acceptance of the tasks by the participants. The novel HVF test battery is now ready to be tested in patients.
The work described in this paper was partially financed by Haag-Streit Foundation. It was then extended and continued in order to be able to present the system described in this paper. We would like to acknowledge the ARTORG Center for Biomedical Engineering Research (University of Bern, Switzerland), the Department of Neurology of the Inselspital (Bern, Switzerland) and the University Hospital of Old Age Psychiatry and Psychotherapy (University of Bern, Switzerland). We would also like to express our thankfulness to all the volunteers who participated in our experiment, to Ioannis Tarnanas and Vanessa Vallejo, Theresa Senti, Loreen Sheffler and Denise De Jong for the inputs regarding the topic of the study and the help in the recruitment process.
- Lee TS, Mumford D, Romero R, Lamme VAF: The role of the primary visual cortex in higher level vision. Vis Res 1998, 38: 2429–2454. 10.1016/S0042-6989(97)00464-1View ArticleGoogle Scholar
- Rovati L, Salvatori G, Bulf L, Fonda S: Optical and electrical recording of neural activity evoked by graded contrast visual stimulus. Biomed Eng Online 2007, 6: 28. 10.1186/1475-925X-6-28View ArticleGoogle Scholar
- Holmes G: Disturbances of vision by cerebral lesions. Br J Ophthalmol 1918, 2: 353. 10.1136/bjo.2.7.353View ArticleGoogle Scholar
- Farah MJ: Visual agnosia. MA: MIT press Cambridge; 1990.Google Scholar
- Zeki S: Cerebral akinetopsia (visual motion blindness) a review. Brain 1991, 114: 811–824. 10.1093/brain/114.2.811View ArticleGoogle Scholar
- Warrington EK, James M: The visual object and space perception battery. Bury St Edmunds: Thames Valley Test Company; 1991.Google Scholar
- Riddoch MJ, Humphreys GW: BORB: Birmingham Object Recognition Battery. LEA; 2005.Google Scholar
- Benton AL: Judgment of Line Orientation. Oxford: Oxford University Press; 1983.Google Scholar
- Grossman ED, Blake R: Perception of coherent motion, biological motion and form-from-motion under dim-light conditions. Vis Res 1999, 39: 3721–3727. 10.1016/S0042-6989(99)00084-XView ArticleGoogle Scholar
- Ramrattan RS, Wolfs RC, Panda-Jonas S, Jonas JB, Bakker D, Pols HA, Hofman A, de Jong PT: Prevalence and causes of visual field loss in the elderly and associations with impairment in daily functioning: the Rotterdam Study. Arch Ophthalmol 2001, 119: 1788. 10.1001/archopht.119.12.1788View ArticleGoogle Scholar
- Raskar R, Brown MS, Yang R, Chen W-C, Welch G, Towles H, Scales B, Fuchs H: Multi-projector displays using camera-based registration. In Visualization’99, Proceedings. San Francisco, CA, USA: IEEE; 1999:161–522.Google Scholar
- Bourke P: Spherical mirror: a new approach to hemispherical dome projection. In Proceedings of the 3rd international conference on Computer graphics and interactive techniques in Australasia and South East Asia. New York, NY, USA: ACM; 2005:281–284.Google Scholar
- Nef T, Gruber N, Zito G, Nyffeler T, Müri R, Mosimann U: Development and evaluation of a new instrument to measure visual exploration behavior. Med Eng Phys 2014, 36: 490–495. 10.1016/j.medengphy.2013.09.011View ArticleGoogle Scholar
- Zhang Z: Flexible camera calibration by viewing a plane from unknown orientations. In Computer Vision, 1999. Kerkyra: The Proceedings of the Seventh IEEE International Conference on IEEE; 1999:666–673.Google Scholar
- Wang Z, Bovik AC, Sheikh HR, Simoncelli EP: Image quality assessment: from error visibility to structural similarity. Image Process IEEE Transact 2004, 13: 600–612. 10.1109/TIP.2003.819861View ArticleGoogle Scholar
- Ertas M, Yildirim I, Kamasak M, Akan A: Digital breast tomosynthesis image reconstruction using 2D and 3D total variation minimization. Biomed Eng Online 2013, 12: 112. 10.1186/1475-925X-12-112View ArticleGoogle Scholar
- Fischer RE, Tadic-Galeb B, Yoder PR, Galeb R: Optical system design. 2000.Google Scholar
- Gramatikov B: Detecting fixation on a target using time-frequency distributions of a retinal birefringence scanning signal. Biomed Eng Online 2013, 12: 41. 10.1186/1475-925X-12-41View ArticleGoogle Scholar
- Cornelissen FW, Peters EM, Palmer J: The eyelink toolbox: eye tracking with MATLAB and the psychophysics toolbox. Behav Res Methods Instrum Comput 2002, 34: 613–617. 10.3758/BF03195489View ArticleGoogle Scholar
- Lewis JR: Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the ASQ. ACM SIGCHI Bull 1991, 23: 78–81.View ArticleGoogle Scholar
- Hart SG, Staveland LE: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Hum Mental Workload 1988, 1: 139–183.View ArticleGoogle Scholar
- Weiner IB, Craighead WE: The Corsini Encyclopedia of Psychology. Washington DC: John Wiley & Sons, Argosy University; 2010.View ArticleGoogle Scholar
- Lewis JR: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995, 7: 57–78. 10.1080/10447319509526110View ArticleGoogle Scholar
- Eberly D: Computing a Point of Reflection on a Sphere. 2008.Google Scholar
- Mulcahy KA, Clarke KC: Symbolization of map projection distortion: a review. Cartogr Geogr Inf Sci 2001, 28: 167–182. 10.1559/152304001782153044View ArticleGoogle Scholar
- FitzGerald MJT, Gruener G, Mtui E: Clinical Neuroanatomy and Neuroscience. Elsevier Health Sciences; 2011.Google Scholar
- Rybak IA, Gusakova VI, Golovan AV, Podladchikova LN, Shevtsova NA: A model of attention-guided visual perception and recognition. Vis Res 1998, 38: 2387–2400. 10.1016/S0042-6989(98)00020-0View ArticleGoogle Scholar
- Hunziker HW: Im Auge des Lesers: vom Buchstabieren zur Lesefreude: foveale und periphere Wahrnehmung. Transmedia Verlag; 2006.Google Scholar
- Chan D, Schmitt N, DeShon RP, Clause CS, Delbridge K: Reactions to cognitive ability tests: the relationships between race, test performance, face validity perceptions, and test-taking motivation. J Appl Psychol 1997, 82: 300.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.