Monetizing content through advertising has become a huge revenue source for TV channels and streaming platforms. However, ad segments ending too soon and overlapping with other advertisements has turned into a major frustration for advertisers, causing friction between companies.

This challenge inspired our team to ask, "How can neural networks and machine learning improve advertisement looping for TV channels and streaming platforms?" With skillful programming, of course. What started as an intriguing question turned into a minimum viable product (MVP) built on these technologies. While the solution was originally designed to determine the beginning and end of advertising blocks so that they appear at the right time, it can actually solve a wide range of other video content challenges.

One video content supplier that we collaborated with was achieving a 60% success rate, meaning 40% of the ad circulations had overlap. This led to a loss of advertisers, and therefore revenue, on their channels. The proposed solution was technically difficult, and at the time, we didn’t have any experts in these areas on the account so we weren’t sure that it would be successful.

According to Oleksii Druzhynin, Lead Software Engineer, EPAM US, who identified this problem when the project first started and was the development lead for the solution, "We first tried solving the problem simply without neural networks, but nothing worked. Then we figured out how most types of neural networks work and taught them to do specific tasks. There are a lot of neural networks, but we were interested in those that process visual information."

Originally, we tested five neural networks and now, we settled on two. They analyze each frame and every bit of sound. When there’s a significant change in color or tone, the networks react, predicting the end of an ad and switching to another block of ads. For now, the network that determines transitions between ads is more accurate than the one that predicts the end of an ad. Together, these neural networks produce a 90% success rate. We also created a bot that can watch television and independently distinguish the content.

Everyone who worked on this project tested neural networks and different algorithms, connecting and synchronizing components. While the idea was born in California, the project engaged 70 engineers across several EPAM locations.

As a result, we created a great MVP that not only determines changes in ads, but is a universal solution for any task related to determining changes in video content, including potential video search solutions. It’s not just limited to ads; it could be used for a TV show with different scenes and settings, data from video cameras or digitized old movies that do not have metadata about the content of the video sequence.

Overall, this project led our team to better understand the need for open source code, as it would have helped with more complex aspects of the project. With the constant project changes and complexity of the solution, looping in new engineers and working with the open source community would have led to greater visibility of our project and would have attracted engineers who were interested in using their expertise by conducting regular checks for code reliability.

It all started with a question about how we could apply cutting-edge technologies to solve a very common challenge for the media and entertainment industry while collaborating with a large video content supplier. When traditional approaches didn’t work as efficiently as they needed to, our team turned to building a solution on neural networks and machine learning. Not only did we work together to figure out how we could improve the original concept, but we created a universal product that can be leveraged many video content companies.