Kurt-Koffka-Preisverleihung 2022
Ted Adelson’s contributions to the field of perception research are nothing short of legendary. And his research connects to and builds directly on the Gestalt foundations established by Kurt Koffka. Indeed, there are few living researchers who have made such fundamental and wide-ranging contributions to the scientific study of vision and perception as Ted Adelson has. His insights are the stuff of textbooks and perception courses. His illusions—such as the famous checker-shadow figure—have inspired and beguiled researchers and laypeople alike. From low-level mechanisms of retinal adaptation, to the development of the original motion energy model, as well as his revolutionary work on texture processing, lightness perception, pyramid decompositions, the plenoptic function, ‘things’ vs ‘stuff’ and material perception … practically everything Ted has done has defined or redefined the field for years afterwards. His work spans theory, psychophysics, computational modelling, and even some neurophysiology. He has without question made exceptional, lasting contributions to psychology. It’s no wonder that he was elected Member of the National Academy of Sciences in 2006.
Have you ever had the experience of watching an alternative intelligence performing actions that seem totally incomprehensible to you? Like watching whirligig beetles swimming around in complex spirals on the top of a pond … where are they going, what are they doing? Or watching naughty crows rolling sideways down the roof in the snow … what exactly are they up to? Well, that’s a bit like what it’s like watching Ted do research. Brilliance is sometimes totally inscrutable to mere mortals like the rest of us. It’s like an ant trying to work out what a human is doing.
“Sorry I haven’t been in the lab all week,” Ted once said to me. “I’ve been really busy shining lasers into cheese”. That was the origin of crucial insights into the perception of translucency!
Or that time when there were literally dozens of socks hanging from the ceiling in the lab. An entirely necessary step in lightness perception research, of course!
“Today we’re going to buy a lot of orange things. It doesn’t matter what it is, but it has to be orange”. See if you can work that one out for yourselves…
Anyway, there’s no way I can list of Ted’s research contributions, it would take far too long. But let me highlight a few of his most significant contributions.
Ted transformed our scientific understanding of lightness and brightness perception. It might seem strange, but despite hundreds of years of perception research, we still don’t really know how the brain tells the difference between black and white. A black piece of paper in sunlight can reflect more light than a white sheet of paper in shadow. Somehow the brain has to work out the reflectance of the surface. In Ted’s words, “lighting and reflectance are smooshed together in the image”. How does the brain tease them apart?
Ted demonstrated the central role of perceptual organization processes—like those proposed by Kurt Koffka—in lightness perception. It is in this area that he developed many well-known illusions, some of which perhaps we’ll see today. But these illusions are not mere eye-candy. Each one makes a profound theoretical point, using phenomenology to test specific scientific hypotheses about the cues and processes underlying surface perception—in the grand tradition of the Gestalt psychologists. Despite his background in retinal processes and image filters, Ted quickly recognized that low-level mechanisms cannot account for all lightness and brightness phenomena. Starting with his famous 1993 Science paper, Ted showed how the visual system does not compare and contrast patches against their backgrounds in spatially agnostic ways, but carefully segments images depending on the causes of light-dark transitions in the image. This was the birth of ‘mid-level vision’.
Later—radically extending this pioneering work on surface perception—Ted played a central role in founding the new field of material perception. In his seminal 2001 paper in Human Vision and Electronic Imaging, he laid out a manifesto for an entirely new research field. He rightly pointed out that while objects had received a lot of attention, much of what we experience—snow, sand or water—is not actually objects at all, but ‘stuff’. At that time, we knew essentially nothing about how the brain recognizes or represents materials and their properties. His highly influential 2007 Nature paper laid the foundations for a theory of glossiness perception based on image statistics. Since then, the field has moved on to ever more complex and fascinating material classes like translucent jade-like materials, textiles and liquids.
But in terms of universal impact, I think even these monumental contributions pale in comparison to his work on Motion Perception. Ted Adelson developed THE standard “Motion Energy Model”, showing quite simply how the brain computes the direction and speed of moving patterns from the everchanging array of light on the retina. This model appears in every textbook, and is the bedrock of essentially all current research on motion perception in psychology, neuroscience and computer vision. The 1985 article describing the model has been cited >4000 times!
His 1982 Nature paper on motion coherence also spurred an incredible variety of follow-up work from human psychophysics to neurophysiology and computational modelling. It established the main parameters for studying cortical representations of motion, and also inspired the development disparity energy models for binocular stereopsis. Not only that, but it also kicked off a huge and raging debate about how the brain combines motion component signals. For a couple of decades people squabbled over whether it was by “intersection-of-constraints (IOC)” or “vector averaging (VA)”. But fortunately, Ted came to the rescue again, resolving the debate with another seminal paper, this time in Nature Neuroscience. He showed how the brain combines noisy motion signals with a prior for slow and smooth motion. It explained all the data from both sides of the debate! That article is also a model of clarity, and is widely used in teaching to explain how Bayesian approaches can be applied to real-world vision problems.
I also have to mention Ted’s contributions to Multiscale and filter-based image representations. Content in images occurs at multiple scales—from the large-scale gradients that capture the bright sky above a darker floor, to the tiny fine details in the foliage of a tree—and at multiple orientations from vertical to horizontal. The brain separates these different image structures so they can be recombined and compared for all kinds of inferences about the outside world, like finding shadows, or working out 3D shape.
Ted developed algorithms that can separate out such different image structures using so-called ‘spatial filters’. The most famous are the concept of ‘steerable filters’, which has been cited over 4000 times, and his invention of the ‘Laplacian pyramid’, which has been cited over 9000 times!
His insights into the power of filter-based representations led to a beautiful Nature paper in 1988, which fundamentally transformed our understanding of texture perception. If you take photos of two different patches of grass, the pixels in the two images will be completely different: you can’t match them up with one another. Yet, when we look at them, it’s immediately obvious that they have the same visual texture. This is the mystery of texture perception. How do we tell when one texture ends—like the table—and the next texture—like the floor—starts? Ted showed how simple image-processing operations convert local variations in scale and orientation into readily detectable signals. This immediately explained a whole load of phenomena that previously had been explained with an esoteric, but ultimately meaningless concept called ‘textons’.
Nowadays Ted is revolutionising Tactile Sensing, making skin for robots so they can feel like the human hand. Ted is an astonishingly inventive thinker. During my PhD I would see him pottering around with little blocks of silicone elastomer, making casts of small objects and tweaking his recipes to alter the softness. Nobody knew what he was up to, but one day he turned up with a block that was coated on one side with a metallic paint. He showed how sticking his finger into it revealed the fingerprint in perfect relief. You could even make out the tiny embossed print on a dollar bill! This turned into the key innovation behind the company GelSight, which manufactures devices for microscale measurements of surface structure, and is nowadays developing novel tactile sensors and skin for robotics systems.
Ted is also an exceptionally gifted writer. The didactic lucidity of his prose, and the immaculately conceived figures to be found throughout his body of work, are the result of a deep investment of time and thought. I once heard him remark wryly that because his illusions are fun and easy to look at, people assume that they are fun and easy to make. They are not. They are the result of intellectual and aesthetic perfectionism applied repeatedly over many, many iterations.
Finally, in addition to his outstanding research achievements, Ted has also been an extraordinarily influential mentor and promotor of other researchers. Over a dozen of the most influential researchers in the whole field of perception have been through his lab as students or post-docs. Those of us lucky enough to have been mentored by him have fond memories of his guidance, his deep insights and deliciously intelligent sense of humour.
It is with great warmth and pride that I commend him for the Kurt Koffka Award and look forward very much to his talk.