Document Actions

Research

 

Labore-1

Laboratories

Click here to find out about our laboratories.

 

  

External funding

Click here to find out about our sponsors.

 

  

 

Introduction

Our research deals with visual and haptic information processing, including colors, textures, materials, objects and natural scenes. Our laboratories have measuring instruments and experimental setups for studies of visual and haptic perception as well as sensorimotor coordination. This includes state-of-the-art equipment for measuring eye movements (EyeLink 1000, Tobii Eye Tracker 4c) and other motor movements (Optotrak-3020 System, Zebris Tracking System, Qualisys Motion Capture System), for manipulating visual-proprioceptive information (PHANToM-force feedback device) and for creating virtual environments (HTC Vive). In addition to psychophysical measurements, a major focus of our department is the modeling of perception and behavior. For this purpose we also have state-of-the-art systems for computational modeling and machine learning at our disposal.

Our department is part of national and international collaborations and networks that are, for example, focused on investigating the most important aspects of human perception (Collaborative Research Center Cardinal Mechanisms of Perception) or multisensory integration in a dynamic environment (Research Training Network Dyvito).

 

Research Areas

 

Perception of material qualities

Prof. Katja Dörschner-Boyaci, Ph.D.

A crucial task of the human visual system is to determine the material that an object is made of. Correct identification of material qualities affects basic decisions such as whether food is edible, skin is healthy, or whether an object is pliable. Humans perceive these qualities in a split second. Yet, little is known about how the brain recognizes materials. Using a combination of psychophysics, image analysis, computational modeling, eye -/ hand tracking, and fMRI we investigate questions such as: What information does the brain use to estimate and categorize material qualities? How do expectations about materials affect how we perceive them, look at them or interact with them? Does freely interacting with a material change how it feels or appears?

 

Selected Publications

  • Schmid, A.C., Barla, P. & Doerschner, K. (2020). Material category determined by specular reflection structure mediates the processing of image features for perceived gloss. bioRxiv. DOI

  • Schmid, A.C., Boyaci, H. & Doerschner, K. (2020). Dynamic dot displays reveal material motion network in the human brain. bioRxiv. DOI
  • Schmid, A.C. & Doerschner, K. (2019). Representing stuff in the human brain. Current Opinion in Behavioral Sciences, 30, pp.178-185. DOI
  • Toscani, M., Yucel, E. & Doerschner, K. (2019). Gloss and speed judgements yield different fine tuning of saccadic sampling in dynamic scenes. i-Perception, 10(6): 2041669519889070. DOI
  • Cavdan, M., Doerschner, K. & Drewing, K. (2019). The many dimensions underlying perceived softness: How exploratory procedures are influences by material and the perceptual task. IEEE World Haptic Conference. DOI

 

Computational neuroscience of visual recognition

Dr. Katharina Dobs

From a brief glimpse of a complex scene, we recognize people and objects, their relationships to each other, and the overall gist of the scene – all within a few hundred milliseconds and with no apparent effort. What are the computations underlying this remarkable ability and how are they implemented in the brain? To address these questions, our research bridges recent advances in machine learning with human behavioral and neural data to provide a computationally precise account of how visual recognition works in humans. Specifically, we leverage the recent successes in artificial neural networks of vision to address long-standing questions concerning the functional organization of the human visual cortex, such as:

- Why do we have functional specialization in the human visual cortex?

- Is visual experience necessary to form functional specificity?

- Which aspects of visual recognition are idiosyncrasies of the human brain and which arise in any system optimized for visual recognition?

 

Selected Publications

  • Dobs, K., Yuan, J., Martinez, J., & Kanwisher, N. (2023). Behavioral signatures of face perception emerge in deep neural networks optimized for face recognition. Proceedings of the National Academy of Sciences, 120(32), e222064212. DOI

  • Dobs, K., Martinez, J., Kell, A. J. E., & Kanwisher, N. (2022). Brain-like functional specialization emerges spontaneously in deep neural networks. Science Advances, 8, eabl8913. [DOI]

  • Dobs, K., Isik, L., Pantazis, D., & Kanwisher, N. (2019). How face perception unfolds over time. Nature communications, 10, 1258. DOI

  • Dobs, K., Ma, W. J., & Reddy, L. (2017). Near-optimal integration of facial form and motion. Scientific Reports, 7(1):11002, 1-9. DOI

 

Perception & Action (PerAct) Lab

Prof. Dr. Katja Fiehler

The PerAct Lab studies how humans use sensory information to guide their actions, focusing on two themes: spatial coding for action and sensorimotor predictions. 

Spatial coding for action. To successfully interact with the environment, the human brain needs to build up a representation of where the action goal is located in space. We are interested in how humans spatially represent targets for actions and how they use this spatial information to plan and control eye and hand movements, e.g., directing gaze to a reach goal or grasping an object. Experiments in virtual reality allow us to expand our research on spatial coding for action from small-scale, static to large-scale, dynamic environments. 

Sensorimotor predictions. As processing of sensory information takes time, predictive mechanisms can help us to overcome these delays and to prepare our motor system to react at the right place to the right time, e.g. predicting the upcoming position of our moving arm when catching a ball helps us to quickly adjust our arm movement if the ball's trajectory suddenly changes. We investigate how well humans can establish and use sensorimotor predictions, and how these predictions influence the processing of sensory feedback. We believe that feedback signals are up- and down-weighted depending on the accuracy of sensorimotor predictions.

For our research, we use a variety of state-of-the-art methods in real and virtual environments, ranging from eye, hand and whole-body movement tracking to psychophysics and functional neuroimaging (fMRI).

 

Selected Publications

  • Fiehler, K., Brenner, E., & Spering, M. (2019). Prediction in goal-directed action. Journal of Vision, 19(9), 10, 1-21. DOI

  • Gertz, H. & Fiehler, K. (2015). Human posterior parietal cortex encodes the movement goal in a pro-/anti-reach task. Journal of Neurophysiology, 114, 170-183. DOI

  • Karimpur, H., Morgenstern, Y., & Fiehler, K. (2019) Facilitation of allocentric coding by virtue of object-semantics. Scientific Reports, 9, 6263. DOI

  • Klinghammer, M., Blohm, G., & Fiehler, K. (2015). Contextual factors determine the use of allocentric information for reaching in a naturalistic scene. Journal of Vision, 15(13), 24. DOI

  • Voudouris D., & Fiehler K. (2017). Enhancement and suppression of tactile signals during reaching. Journal of Experimental Psychology: Human Perception and Performance, 43(6), 1238-1248. DOI

 

Visual Perception of the Physical Properties of Objects

Prof. Roland W. Fleming, PhD

Research in my lab focusses on how the brain visually estimates the physical and functional properties of objects in our surroundings. When we look at things, we don't experience the world as a meaningless jumble of lines, colours or motions. Instead, whenever we open our eyes, we immediately gain access to a richly detailed world of meaningful visual sensations. We recognise objects; perceive what things are made of; identify risk and pleasures and can even work out how objects might respond to forces or actions. Based on how things look, we are able to make a remarkable range of subtle judgements about the physical properties of objects, such as whether food is fresh or stale or whether an object is stable or likely to topple over. Without touching an object, we can usually work out what it would feel like were we to reach out and touch it, based on the curves and contours of its shape and the way light plays across its surface. My research program aims to understand how the brain estimates the 3D shape of surfaces, and the material properties of objects such as elasticity, translucency or viscosity. In order to do this, we use a combination of computer graphics, image analysis techniques, neural modelling and psychophysical experiments.

 

Selected Publications

  • Phillips F* & RW Fleming* (2020). The Veiled Virgin illustrates visual segmentation of shape by cause. Proceedings of the National Academy of Sciences 201917565 (*the authors contributed equally to this work) DOI

  • Fleming RW & KR Storrs (2019) Learning to See Stuff. Current Opinion in Behavioral Sciences. 30: 100–108. DOI

  • Van Assen, JJ, Barla P & Fleming, RW (2018). Visual Features in the Perception of Liquids. Current Biology 28(3), 452–458. DOI

  • Fleming RW (2017). Material Perception. Annual Reviews of Vision Science 3(1). 365-388. DOI

 

Individual differences in perception

Dr. Ben de Haas
Visual perception feels like an objective window to the world. When we claim to have seen something ‘with our own eyes’, we mean to imply the objective certainty of what we saw, rather than the subjective nature of our impressions. Nevertheless, our perception of the world isn’t always the same. We can look at the same image, but move our eyes to different parts of it; look at the same face and disagree about its trustworthiness; and sometimes even look at the same object and disagree about its colour.

Our research aims to understand more about three related questions:

1)      How does perception vary from one person to the next?

2)      What are the mechanisms behind these differences?

3)      How do they shape who we are and how we interact with others?

To investigate these questions, we use eyetracking, psychophysics and neuroimging in healthy and clinical populations. More information can be found here.

 

Selected Publications

  • de Haas, B., Iakovidis, A. L., Schwarzkopf, D. S., & Gegenfurtner, K. R. (2019). Individual differences in visual salience vary along semantic dimensions. Proceedings of the National Academy of Sciences, 116(24), 11687-11692. DOI

  • Moutsiana, C., de Haas, B., Papageorgiou, A., Van Dijk, J. A., Balraj, A., Greenwood, J. A., & Schwarzkopf, D. S. (2016). Cortical idiosyncrasies predict the perception of object size. Nature communications, 7(1), 1-12. DOI

  • de Haas, B., Kanai, R., Jalkanen, L., & Rees, G. (2012). Grey matter volume in early human visual cortex predicts proneness to the sound-induced flash illusion. Proceedings of the Royal Society B: Biological Sciences, 279(1749), 4955-4961. DOI

 

Color Vision

Prof. Karl Gegenfurtner, Ph.D. , Dr. Thorsten Hansen
The perception of color is a central component of primate vision. Colour facilitates object perception and recognition, and has an important role in scene segmentation and visual memory. Despite the long history of colour vision studies, much there is still much to be learned about the physiological basis of colour perception. Recent studies are beginning to indicate that colour is processed not in isolation, but together with information about luminance and visual form to achieve a unitary and robust representation of the visual world.

 

Selected Publications

  • Hansen, T., & Gegenfurtner, K. R. (2006). Higher level chromatic mechanisms for image segmentation. Journal of Vision, 6(3), 239\u2013259. [PDF]
  • Hansen, T., Olkkonen, M., Walter, S. & Gegenfurtner, K. R. (2006). Memory modulates color appearance. Nature Neuroscience, 9(11), 1367–1368.
  • Gegenfurtner, K.R. (2003) Cortical mechanisms of colour vision. Nature Reviews Neuroscience, 4, 563–572. [PDF]
  • Gegenfurtner, K.R. & Kiper, D.C. (2003) Color vision. Annual Review of Neuroscience, 26, 181–206.[PDF]

 

Haptic and Multisensory Perception

Prof. Dr. Knut Drewing

Perception is an active process during that we purposively gather sensory information. Haptic perception is the prime example for this principle. For example, when we aim to haptically judge an object's softness, we first will have to appropriately explore the object in order to obtain the relevant sensory information. Often a single indentation of the object is not sufficient, but we repeatedly indent the object before we deliver a judgment.

In the HapLab we study how humans control natural explorations in active touch, and how information of different type is integrated into a percept of the stimulus. We found, for example, that humans perceive objects to be softer, when they already believe that these objects are relatively soft (Metzger & Drewing, 2019). We also modeled the integration of serially gathered texture information over the course of the exploration, using an ideal observer model (Lezkan, Metzger & Drewing, 2018). Other results suggest that people fine-tune their exploratory movements in order to optimize perception (Zoeller, Lezkan, Paulun, Fleming, & Drewing, 2019). Further research foci in the HapLab are on different dimensions of softness (Cavdan, Doerschner, & Drewing, 2019), haptic saliency and haptic search, multisensory integration (i.e. size-weights illusion), signals to haptic perception of environmental properties (time, space, softness, texture, shape) and on emotional effects of haptic stimulation (Drewing, Weyel, Celebi, & Kaya, 2018).

 

Typical visuo-haptic VR setup, 3D-printed and custom-made stimuli

 

The main competences of the lab are on behavioral and perceptual aspects of haptic and multisensory signal processing and of associated movement control. Presently we use in the first place a visuo-haptic VR/AR-setup including a force-feedback device, force sensors and a stereo display, a high-resolution 3D printer (Stratasys Objet Pro), vibrotactile actuators and various custom-made stimuli.

 

Selected publications

  • Cavdan, M., Doerschner, K., & Drewing, K. (2019). The Many Dimensions Underlying Perceived Softness: How Exploratory Procedures are Influenced by Material and the Perceptual Task. IEEE World Haptics Conference, WHC 2019 (pp. 437-442), IEEE. DOI

  • Drewing, K., Weyel, C., Celebi, H., & Kaya, D. (2018). Systematic Relations between Affective and Sensory Material Dimensions in Touch. IEEE Transactions on Haptics 11(4), 611-622. DOI

  • Lezkan, A., & Drewing, K. (2018). Processing of haptic texture information over sequential exploration movements. Attention, Perception, & Psychophysics, 80(1), 177-192. DOI

  • Metzger, A., & Drewing, K. (2019). Memory influences haptic perception of softness. Scientific Reports, 9(1), 1-10. DOI

  • Zoeller, A. C., Lezkan, A., Paulun, V. C., Fleming, R. W., & Drewing, K. (2019). Integration of prior knowledge during haptic exploration depends on information type. Journal of Vision, 19(4), 1-15. DOI

 

Perception and eye movements in natural scenes

Prof. Karl Gegenfurtner
We study the principles underlying the selection of fixation targets under natural viewing conditions. We study fixation patterns and saccadic latencies of human subjects viewing under natural images and videos of natural scenes and ask how stimulus features like contrast, color and spatial frequency content interact with top-down mediated expectations.

 

Selected Publications

 

Eye movements and Visual Perception

Dr. Jutta Billino, Dr. Doris Braun,  Prof. Karl Gegenfurtner

Humans frequently move their eyes, either to fixate a new location in the visual field (saccadic eye movements), or to keep fixation on a moving object (smooth pursuit eye movements). These eye movements pose two problems. First, an appropriate target location and execution time has to be selected for the eye movements. Hence we study, which visual signals are used to guide these eye movements, i.e. how visual perception influences the control of eye movements. Second, the execution of eye movements changes the visual image on the retina. To maintain a clear and stable perception of the world, the visual system has to cope with the retinal image motion. In this context we study how visual perception is affected by the execution of concurrent eye movements.

Our experimental approach comprises psychophysics measurements under simultaneous tracking of eye movements to investigate the bidirectional relationship between perception and eye movements.

 

Selected Publications

  • Schütz, A.C., Braun, D.I., Kerzel, D. & Gegenfurtner, K.R. (2008) Improved visual sensitivity during smooth pursuit eye movements. Nature Neuroscience, 11, 1211-1216. DOI
  • Spering, M. & Gegenfurtner, K.R. (2008). Contextual effects on motion perception and smooth pursuit eye movements. Brain Research, 1225, 76-85. DOI
  • White, B.J., Stritzke, M. & Gegenfurtner, K.R. (2008) Saccadic facilitation in natural backgrounds. Current Biology, 18, 124-128. DOI

Visually guided motor behavior

Dr. Jutta Billino

We investigate the complex mechanisms involved in interactions of humans with the environment. The versatility of the human visuo-motor system can be seen in the ease with which we perform everyday tasks such as reaching and grasping for objects under varying visual input. For example, we can easily grasp fragile objects like eggs (we might even learn to juggle them), or we might learn to adapt quickly to the distortions introduced by wearing left-right reversing prisms, etc. On the other hand, it is still very difficult to devise technical systems which are capable of only a subset of the capabilities of the human motor system.

One of the questions we have been studying intensively during recent years is whether the visual guidance of motor behavior is achieved by different processes (and neuronal substrates) as our conscious (visual) perception. Studies on neurological patients suggest such a division of labor in the human brain and it was suggested that this dissociation between vision-for-action and vision-for-perception can also be found in healthy humans. Support for this view came from studies which found that grasping is less affected by visual illusions than perception. Our results, to the contrary, suggest that the motor system uses very similar processes and neuronal signals as visual perception. This suggests that the brain is more coherent than currently proposed by a number of theories in visual neuroscience.

We use an Optotrak 3020 system to measure precisely position and timing of grasping and pointing movements. To control the Optotrak we use a specifically designed Matlab Toolbox (or C++).

 

Selected Publications

  • V. H. Franz. Planning versus online control: Dynamic illusion effects in grasping? Spatial Vision, 16(3-4):211 - 223, 2003. [PDF]
  • V. H. Franz. Action does not resist visual illusions. Trends in Cognitive Sciences, 5(11):457 - 459, 2001. [PDF]
  • V. H. Franz, K. R. Gegenfurtner, H. H. Bülthoff, and M. Fahle. Grasping visual illusions: No evidence for a dissociation between perception and action. Psychological Science, 11(1):20 - 25, 2000. [PDF]