Automatically removing video background in real-time
Background identification of a real scene video is one of the more challenging and critical tasks in Computer Vision applications. Background subtraction is widely used as a technique where an image foreground is extracted for processing, moving objects are identified from the portion of a video frame that differs significantly from a background model.
Green screen and chroma-keying techniques are commonly used to eliminate the background from the subject of a photo or video. In particular, in the newscasting, motion picture and video gaming industries.
However, there are certain limitations to this method. For example, to obtain the best outcome a person should stand in front of a solid background and wear contrasting colors to the background.
Also, another problem can arise, when a person is in front of a white background and smiles. Not only the background but also the teeth are removed because they are white.
Our R&D engineers had an opportunity to work on a research project on removing the original background in the video and adding a custom image or another video without the need to use a studio or manually edit it in video editing programs for many hours. We have previous experience in the area having completed a project on object tracking in a video.
Computer Vision Solutions
Give meaning to images, analyze video, and recognize objects with the highest accuracy
The research began with an analysis and comparison of various background subtraction algorithms. We considered various approaches, from simple such as Running Gaussian average, Background mixture models to more complex such as Watershed Segmentation, GrabCut, and other graph-based algorithms.
The technical objective of the experiment was to develop and implement an algorithm that will not depend on colour or lighting to remove the background in the video in real-time. Among the set requirements were the following:
- The algorithm should work with any random background
- The object in the forefront should be chosen automatically.
- The video should be processed and rendered real-time at the desired level of 25 frames per second.
Based on our experience, our R&D team created the algorithm. See in action how the algorithm automatically removes the background from the video:
Results
The developed algorithm for background subtraction:
- Does not depend on background colour or texture of the surface behind a person; allows removing any reasonable random background
- Chooses an object in the foreground automatically by putting a frame around this object and remove the background from the video.
The algorithm can be applied both for PC-based standalone applications as well as for mobile apps (iOS and Android).
