How Do I Know?

Behind the curtain

Here is how I know

What the camera saw

start of the look

How the trick works

Find the face. About 30 times a second, a face-tracking model from Google (MediaPipe Face Landmarker) puts 478 points on the face in the video, including the edge of each iris.
Measure the eyes. From those points the program takes five numbers: where each iris sits between the two corners of its eye, how far the head is turned (measured two ways), and the face model's own estimate of where the eyes point.
Learn this person's eyes. The smoke moved from ball to ball, twice to each one, and your eyes followed it. While you watched, the program recorded your five numbers for each ball. It then fitted a simple straight-line rule (a linear regression) that turns those numbers into a spot along the row of balls.
Count the looking time. When I asked you to send me your ball, you stared at it. The program checked which ball your eyes pointed at, frame by frame, and added up the time. The ball with the most looking time was the guess. The shuffle is only showmanship.

Why the breathing and the smoke

The program needs a calibration: a few seconds of you looking at each ball while it knows which ball that is. Telling you "look at the red ball" would give the secret away. The ritual gives you a reason to do exactly that. Breathing with closed eyes keeps your head still, and the smoke tells your eyes where to go next. Magicians call this misdirection: the story hides the method. While the program records, the screen also shows no words and the magician speaks instead, because eyes jump to any text they can read.

Why the balls are so big

A webcam is a rough eye tracker. Lab eye trackers shine infrared light on the eye and film it with a fast, close-up camera. A webcam sees your iris as a small patch of pixels, so its guess is often off by a few centimetres on the screen. It can tell left, middle and right apart, but it could not tell which word you are reading. That is why the balls are big and far apart, and why the smoke ritual comes first.

Why choice researchers care

People tend to look longer at the option they end up choosing, and looking at an option can also nudge the choice toward it. Here you looked on purpose. In research, the same measure is taken while people simply decide, and it can predict the choice before they make it. Tools such as WebGazer.js let researchers run these studies online with ordinary webcams.

Shimojo, Simion, Shimojo & Scheier (2003), Nature Neuroscience, "Gaze bias both reflects and influences preference." Krajbich, Armel & Rangel (2010), Nature Neuroscience, "Visual fixations and the computation and comparison of value in simple choice."

Words used here

Calibration (the smoke ritual): You look at known spots so the program can learn what your eyes look like when they point there. Every face is different, so this is done for each volunteer.
Face landmarks: Points a computer model places on a face in a picture, such as the corners of the eyes and the tip of the nose.
Iris: The coloured ring of the eye. Where it sits between the eye corners shows which way the eye points.
Dwell time: How long the eyes stay on one thing.
Linear regression: A way to find the straight-line rule that best turns some measured numbers (here, iris position and head turn) into an answer (here, where you looked).
Self-test: The program learned from the first trip of the smoke and was tested on the second trip, then the other way round. The score is how often it named the right ball.