Came across this blog from google about how they have managed to find out the background in a video. https://research.googleblog.com/2018/03/mobile-real-time-video-segmentation.html is the the link to the google research paper about this and https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf is the paper they have built upon. I have to look into this soon. Adding as a follow up.
