Sony Patent Details Frame Prediction to Cut Input Lag

Sony filed a patent on October 15 detailing frame prediction technology intended to reduce input delay in frame generation, as spotted by Patentlyze and reported by wccftech.com.

The patent describes what it calls "frame generation super sampling," which generates an image frame embedding from a first image frame and then predicts a synthetic second image frame using that embedding. The system relies on an encoder that reads a rendered frame, optional motion vectors and controller input, plus a decoder that outputs a generated frame shown before the next real frame.

Rather than generating the next frame purely from what has already been rendered, the technology predicts it based on the player's expected input. The patent compares the principle to rollback netcode but describes it as more complex. It states that user input may occur within 15 ms of the last image frame generation, or between 15 ms and two seconds before it.

The filing also includes a non-AI fallback intended to mitigate incorrect predictions. It does not detail additional hardware requirements beyond current gaming systems.

Older context

Over the past few years, NVIDIA paved the way for upscaling technologies like DLSS Super Resolution, influencing PlayStation's PSSR and QSSR. Frame generation has made high-framerate experiences easier to achieve but suffers from input delay and image quality issues, while NVIDIA Reflex is a technology that mitigates input delay.