According to Oleksii Druzhynin, Lead Software Engineer, EPAM US, who identified this problem when the project first started and was the development lead for the solution, "We first tried solving the problem simply without neural networks, but nothing worked. Then we figured out how most types of neural networks work and taught them to do specific tasks. There are a lot of neural networks, but we were interested in those that process visual information."
Originally, we tested five neural networks and now, we settled on two. They analyze each frame and every bit of sound. When there’s a significant change in color or tone, the networks react, predicting the end of an ad and switching to another block of ads. For now, the network that determines transitions between ads is more accurate than the one that predicts the end of an ad. Together, these neural networks produce a 90% success rate. We also created a bot that can watch television and independently distinguish the content.
Everyone who worked on this project tested neural networks and different algorithms, connecting and synchronizing components. While the idea was born in California, the project engaged 70 engineers across several EPAM locations.
As a result, we created a great MVP that not only determines changes in ads, but is a universal solution for any task related to determining changes in video content, including potential video search solutions. It’s not just limited to ads; it could be used for a TV show with different scenes and settings, data from video cameras or digitized old movies that do not have metadata about the content of the video sequence.
Overall, this project led our team to better understand the need for open source code, as it would have helped with more complex aspects of the project. With the constant project changes and complexity of the solution, looping in new engineers and working with the open source community would have led to greater visibility of our project and would have attracted engineers who were interested in using their expertise by conducting regular checks for code reliability.
It all started with a question about how we could apply cutting-edge technologies to solve a very common challenge for the media and entertainment industry while collaborating with a large video content supplier. When traditional approaches didn’t work as efficiently as they needed to, our team turned to building a solution on neural networks and machine learning. Not only did we work together to figure out how we could improve the original concept, but we created a universal product that can be leveraged many video content companies.