machine vision
TEN FILMS · READ AT EIGHT FRAMES A SECOND · ONE ATTENTION MODEL · ONE CORTICAL SURFACE · ONE SEARCH · NOTHING HERE WAS MEASURED IN A BRAIN
The film, and its reading.
Pick a film. Eight lanes are proxies built from what each cortical region is known to respond to, edges, motion, faces, bodies, loudness, lagged the way blood follows neurons. The red heat is a trained model of where eyes go. The thin white line is surprise. Beside it, the same film lights a standard cortical surface, 20,484 vertices; switch the source to the twin for the released encoder, one predicted response per vertex per three seconds. 112213
the screening room attention · reallanes · proxy
· LANES: EARLY VISION · MOTION · BODIES · FACES · PLACES · SOCIAL · VOICES · HEARING · SOUND · ATTENTION · TICKS ARE DETECTED CUTS · SCRUB THE LANES TO MOVE THE FILM
the cortex surface · reallit by · proxy lanes
· DRAG TO TURN · SCROLL OR PINCH TO ZOOM · REGIONS ARE DESTRIEUX ATLAS APPROXIMATIONS · THREE CARRY NO LANE AND STAY DARK: LANGUAGE, VALUE, CHOICE · THE TWIN: REAL PREDICTIONS, STILL IN SILICO, RESEARCH USE ONLY
Guided search, raced.
Making a video as a search: a candidate is a string of genes, rendered and scored by a model of a cortical region, the best kept, crossed and mutated. Genetic search beat random draws with an effect size of 0.46 and beat hill climbing. Here the three race on an eight-gene landscape with one crowded hill and two hidden peaks, same budget each. The terrain shows two genes; a candidate's height is its real score across all eight, so a dot reaches a spire only when the six unseen genes are right too. 1
the landscape synthetic
best so far
DRAG THE TERRAIN TO TURN AND TILT, SCROLL TO ZOOM · RUNS REPEAT ON THEIR OWN · THE MEAN OVER RUNS IS THE CLAIM, SINGLE RUNS VARY · MOVE A DIAL TO RESTART
The film, three more ways.
One cortex per three seconds of the film. Then the film as a route through 320 of the studio's finished stills, laid out by a vision model along its first three axes: a fast comet keeps changing what it shows, a slow one holds. Then the first-frame test on our own films: every found clip scored against its own first frame held still, motion worth 0.61 in the motion area and 0.14 in the face area. 31
the film as brains one cortex per three seconds
the studio map embeddings · real
DRAG TO TURN · 320 STILLS · THE FILM'S FRAMES PROJECTED INTO THE SAME SPACE, ONE EVERY EIGHTH OF A SECOND
does motion earn its cost
THE TEST: FREEZE THE FILM'S FIRST FRAME ON SCREEN FOR THE WHOLE RUN, THEN ASK WHICH THE REGION PREFERS, THE FREEZE OR THE MOVING FILM · EACH BAR = THE FILM'S MEAN RESPONSE MINUS THE FROZEN FRAME'S · ABOVE THE LINE THE MOTION EARNED ITS COST; BELOW IT THE FREEZE WOULD HAVE DRIVEN THE REGION HARDER · CLICK A BAR TO PLAY THAT FILM · THE PROXY: THE LANE'S MEAN OVER THE FILM MINUS THE HELD FRAME'S, IN STANDARD DEVIATIONS WITHIN THE FILM
The ideas, made live.
The research this page stands on, each idea rebuilt as a live diagram beside what it means for us. Drag to turn, hover for the numbers, replay when it ends.