Tang Jingwei1,2, Aksoy Yagiz2, Cengiz Öztireli1, Gross Markus1,2, Aydin Tunc Ozan1
1Disney Research 2ETH Zu ̈rich
The goal of natural image matting is the estimation of opacities of a user-defined foreground object that is essen- tial in creating realistic composite imagery. Natural mat- ting is a challenging process due to the high number of unknowns in the mathematical modeling of the problem, namely the opacities as well as the foreground and back- ground layer colors, while the original image serves as the single observation. In this paper, we propose the es- timation of the layer colors through the use of deep neural networks prior to the opacity estimation. The layer color estimation is a better match for the capabilities of neu- ral networks, and the availability of these colors substan- tially increase the performance of opacity estimation due to the reduced number of unknowns in the compositing equa- tion. A prominent approach to matting in parallel to ours is called sampling-based matting, which involves gather- ing color samples from known-opacity regions to predict the layer colors. Our approach outperforms not only the pre- vious hand-crafted sampling algorithms, but also current data-driven methods. We hence classify our method as a hybrid sampling- and learning-based approach to matting, and demonstrate the effectiveness of our approach through detailed ablation studies using alternative network archi- tectures.
Links:
PDF