Optical Flow

Optical flow is the per-pixel motion vector field that describes how each pixel in one frame moves to its position in the next frame, used as the foundation for retiming, frame interpolation, motion blur synthesis, denoising and stabilization. Computing optical flow involves comparing two frames pixel-by-pixel and inferring the direction and magnitude of each pixel's displacement; modern algorithms use a combination of brightness constancy assumptions, smoothness constraints, and AI-based methods (RAFT, FlowNet) to produce dense, accurate flow fields. The primary use case in editorial is frame rate conversion: turning 24fps footage into slow motion at 60fps requires generating intermediate frames that don't exist in the original; optical flow synthesizes these by warping pixels along the computed motion paths. Tools include Twixtor (RE:Vision Effects, the longstanding feature standard), Twixtor's built-in flow algorithm, RE:Vision's RE:Flow, Adobe Premiere/After Effects' Pixel Motion, DaVinci Resolve's Speed Warp and Resolve's Optical Flow (used for retiming and deinterlacing). Quality optical flow requires sufficient texture (uniform regions like skies are hard to flow), absence of rapid occlusions (subjects crossing each other), and freedom from severe motion blur in the input. Failed flow produces visible artifacts: smeared edges, ghosting, melted edges on uncovered regions. Reference: 'A Database and Evaluation Methodology for Optical Flow' (Baker et al.) defines benchmark datasets.