We train machines to visually detect the arrow of time, if a video is playing forward or backward, by watching millions of online clips.
At a small scale, the physics of the world is reversible. Most fundamental equations look the same whether time is playing forward or backwards. The collision of two particles looks the same if time is reversed. Yet at a macroscopic scale, time is not reversible and the world behaves differently whether time is playing forwards or backwards. A diver’s splash looks quite different in forward time and in reverse time. We are interested in how causality manifests itself visually. For most videos, people can very often detect if the video is playing backwards. What is it that lets us see that time is going the wrong way? Can we train a machine to make that same judgement?
