How to Automatically Remove Background from Video in Real-time
Background identification of a real scene video is one of the more challenging and critical tasks in Computer Vision applications. Background subtraction is widely used as a technique where an image foreground is extracted for processing, moving objects are identified from the portion of a video frame that differs significantly from a background model.
Green screen and chroma-keying technique are commonly used to eliminate the background from the subject of a photo or video – particularly in the newscasting, motion picture and video gaming industries. However, there are certain limitations to this method, for example, to obtain the best outcome a person should stand in front of a solid background, a flat surface wall, wear contrasting colours to the background, and avoid too many shadows. Also, another problem can arise, when a person is in front of a white background and smiles, not only the background but also the teeth are removed because they are white.
Our R&D engineers had an opportunity to work on a research project on removing the original background in the video and adding a custom image or another video without the need to use a studio or manually edit it in video editing programs for many hours. We have previous experience in the area having completed a project on object tracking in a video.
The research began with an analysis and comparison of various background subtraction algorithms. We considered various approaches, from simple such as Running Gaussian average, Background mixture models to more complex such as Watershed Segmentation, GrabCut, and other graph-based algorithms.
The technical objective of the experiment was to develop and implement an algorithm that will not depend on colour or lighting to remove the background in the video in real-time. Among the set requirements were the following:
- The algorithm should work with any random background
- The object in the forefront should be chosen automatically.
- The video should be processed and rendered real-time at the desired level of 25 frames per second.
Based on our experience, our R&D team created the algorithm. See in action how the algorithm automatically removes the background from the video:
The developed algorithm for background subtraction:
- Does not depend on background colour or texture of the surface behind a person; allows removing any reasonable random background
- Chooses an object in the foreground automatically by putting a frame around this object and remove the background from the video.
The algorithm can be applied both for PC-based standalone applications as well as for mobile apps (iOS and Android).