Researchers have developed a method for producing looping videos from one image. The technique is specialized in fluid motion seen in water, smoke or clouds. After training a deep neural network on thousands of images, the framework is capable of estimating motion. The solution posed several challenges, from which the most difficult to overcome was the employment of the splatting technique. Through it, holes in the top part of the images appeared. Using the previously obtained motion, the authors created a symmetric splatting methodology that merges the flow bidirectionally. A presentation of the published work can be seen in the following video.
For an in-depth understanding of the paper, please see the following resources:
Source (University of Washington. “Researchers can turn a single photo into a video.” ScienceDaily. ScienceDaily, 15 June 2021.)
Original paper: Holynski, A., Curless, B., Seitz, S.M. and Szeliski, R., 2020. Animating Pictures with Eulerian Motion Fields. arXiv preprint arXiv:2011.15128.