Binocular disparity: how two eyes make depth

A coffee cup on your desk and the wall behind it both cast two flat images, one in each eye. The brain's problem is not simply to notice that the images differ. It has to decide which bit of the left image corresponds to which bit of the right image, then turn that tiny horizontal mismatch into perceived depth. That mismatch is binocular disparity.

The idea sounds like optics, but it became a neuroscience concept because it gives the visual system a sharply defined computation to solve. The left and right eyes are separated by a few centimeters. A nearby point projects to slightly different retinal positions in the two eyes; a farther point shifts less, or in the opposite direction, depending on where the eyes are fixating. A 2005 review by Jenny Read describes stereopsis as a useful model for perception because the geometry is well understood and can be tested in both humans and animals. 1

Binocular disparity is the difference between the retinal positions of matching features in the two eyes. Stereopsis is the perceptual result: depth seen from disparity. The distinction matters. A neuron can be sensitive to disparity without, by itself, explaining the experience of depth.

Two versions of disparity appear repeatedly in the literature. Absolute disparity is the mismatch for one point relative to the current fixation point. Relative disparity is the difference in disparity between two points. Human depth judgments often rely on relative disparity because it stays more stable when the eyes change vergence, while absolute disparity shifts with eye position. Cumming and Parker framed this distinction explicitly in awake-monkey V1 recordings. 2

A practical example: hold one finger near your face and another farther away. Fixate the far finger, then the near one. The doubled images and apparent depth relations change in a systematic way. Your brain is not reading a depth label from either retina. It is comparing two images and using eye position, receptive fields, and higher-level scene constraints to choose the most plausible 3-D layout.

Béla Julesz's 1960 Bell System Technical Journal paper changed the question. Julesz generated artificial stereo picture pairs on a digital computer. Each image looked random when viewed by one eye, but when the two images were fused, correlated regions appeared in depth. The IEEE record gives the publication date as 30 September 1960, pages 1125-1162, and DOI 10.1002/j.1538-7305.1960.tb03954.x. 3

That design stripped away a tempting explanation. If depth perception in stereopsis depended on first recognizing familiar objects in each eye and then comparing their positions, random-dot stereograms should have failed. They did not. Read summarizes the point sharply: random-dot patterns can contain no monocular structure, yet a square or surface can pop out when the images are combined binocularly. 1

The deep issue is called the correspondence problem. For any dot in the left eye's image, there may be many similar dots in the right eye's image. The visual system has to reject false matches and settle on a globally consistent interpretation. Julesz's stimuli made that problem visible.

Early visual processing keeps the two eyes partly separate. Signals from the retina pass through the lateral geniculate nucleus and enter primary visual cortex, where many neurons begin to combine input from both eyes. A 2023 review notes that disparity-tuned neurons are first seen along the visual pathway in V1. 4

The classic physiological step came soon after Julesz. Barlow, Blakemore, and Pettigrew recorded binocularly driven units in cat primary visual cortex. Their 1967 paper reported that a stimulus correctly positioned in both eyes drove cells more effectively than a monocular stimulus, and that 87 recorded units from seven cats had measurable preferred horizontal and vertical disparities. 5 Poggio and Fischer later recorded depth-sensitive neurons in striate and prestriate cortex of behaving rhesus monkeys. 6

This is the neuroscience leap: disparity is not just a geometrical relationship between two images. It is represented by populations of neurons whose firing changes when the left-right match shifts.

It is easy to overstate the finding. If V1 neurons respond to disparity, one might say V1 is where stereoscopic depth is perceived. The evidence is more careful than that.

Read's review argues that V1 neurons appear to perform important early processing, but the perception of depth reflects more global properties of the stimulus. Local V1 receptive fields can respond to matches that are false from the standpoint of the whole stereogram. 1 Cumming and Parker made the point experimentally in 1997 with anticorrelated random-dot stereograms, where dots in one eye are paired with opposite-contrast dots in the other. These displays do not normally produce depth perception, but disparity-selective V1 neurons still signaled their disparity. 7

That result separates neural sensitivity from conscious percept. V1 may supply a local disparity code. Later areas and broader circuits must help decide which local matches belong to a coherent surface in the world.

Three questions still keep binocular-disparity research from becoming a closed textbook chapter.

First, how much of the correspondence problem is solved early? Simple local models can produce disparity tuning, including tuning for random-dot stereograms, without solving the global matching problem. Read describes these models as useful approximations of early cortical processing, not full accounts of depth perception. 1

Second, where does the brain transform absolute disparity into the relative, surface-based signals that perception seems to use? Cumming and Parker's 1999 V1 study found that 45 of 53 cases maintained a consistent relationship with absolute disparity, and no recorded V1 neuron maintained a consistent relationship with relative disparity. They concluded that the relative-disparity signals used in primate depth perception are constructed outside V1. 2

Third, development is still under active study. Binocular vision depends on eye-specific wiring, matched orientation preferences across the two eyes, and disparity selectivity. The 2023 Frontiers review emphasizes unresolved circuit questions about how interocular matching and disparity selectivity develop, with much recent work using mice, ferrets, and tree shrews. 4

Binocular disparity is one of the cleanest examples of a perceptual computation. The input is concrete: two 2-D retinal images. The output is concrete: a 3-D relation between surfaces. The hard part sits between them, where the brain must match features, reject false correspondences, combine local signals, and stabilize the result as the eyes move.

That makes stereopsis a bridge between the early visual concepts in this channel. Ocular dominance columns show how V1 keeps eye inputs partly organized. Receptive fields explain why individual neurons only see a small patch of the image. Orientation selectivity shows how cortex builds feature detectors. Binocular disparity asks what happens when two matched feature maps must be compared.

It also gives a warning that applies across cognitive neuroscience: a neural response that correlates with a stimulus feature is not automatically the neural basis of a percept. V1 neurons can signal disparity in displays that do not produce depth perception. The brain still has to turn local evidence into a global interpretation.

Landmark paper: Béla Julesz, "Binocular depth perception of computer-generated patterns," The Bell System Technical Journal 39(5), 1125-1162, 1960. 3

Course connection: MIT 9.13, The Human Brain, uses visual perception and cognitive neuroscience methods as a backbone for asking how mental abilities are implemented in the brain; binocular disparity fits that sequence as a computation that links early visual cortex, psychophysics, and neural recording. 8