Academy blog

Testing Gives You Better Aim

In our previous articles we spoke about the ability to predict churn and support your ROI with lifetime value forecasts. It was clear the importance of defining longitudinal strategies involving business, technical and operational departments within your entire media service to reduce churn and maximize revenue.

Now, how to rest assure that you make the right call? How do you know that applying changes on advertising arrangement, on content, on your UI and make any other significant changes is good for your business? No longer need to use game theory before taking any major customer driven decision. A/B Testing and Multivariate Testing (MVT) will help you test decisions before implementing them into the wider audience reducing and mitigating risks bringing you closer to the real Nash’s equilibrium.  



A/B or split testing is the most standard practice when it comes to testing decisions. It allows you to show your users two variants of a same campaign, tracking the difference in your audience’s reaction towards it and therefore understand which version makes them tick.

A Multivariate test is more complex than an A/B test, it starts off with defining a goal. From there it allows you to change and monitor an unlimited range of variables that you think highly influence your goal over that specific segment of your audience. This practice is more user resourceful than others for it to be greatly accurate. MVT provides you with insights on which out of all variations perform the best. Based on the result you can see which combinations work best for the goal previously defined.



Both of these tests give you the ability to turn your business intelligence (BI) tool into a lab where you can run controlled experiments among segments of your customer base. Before taking any major decision, you would test it among a reduced sample of users and see how it affects their behavior. The ability to test decisions prior to taking them gives you a strong competitive edge as you reduce your chances to shoot and miss.



The effect of such tests in a real setting is mutually beneficial to your users and your business. Testing several parameters such as: editorial decisions, content arrangement, IO, number and positions of ads per page, CDN strategy, recommendation algorithms, etc. improves your service. Applying these methods leads to effective customer retention, increase in user interaction, engagement, satisfaction and higher revenue. Testing alongside predictive and descriptive insights provided by business intelligence puts you in the most accurate road towards success.


Answer the  “what are the odds?” question accurately.


Just another thing to think about from us here at NPAW.

Joaquin De Miguel on September 06th 2017

Related Post

NICE PEOPLE AT WORK will be live streaming the 080 Barcelona Fashion

It’s that time of year again, fashion designers and models are ready for today’s runway, and NICE PEOPLE AT WORK will be there too! From the 2nd to the 5th of February we will give you live, high quality and uninterrupted streaming of all the shows of the 080 Barcelona Fashion, so you won´t... 

2 Feb

NICE PEOPLE AT WORK and Brightcove become partners in Brightcove’s new player generation

YOUBORA Analytics is available in Brightcove’s new advanced player to maximize End User Quality of Experience in OTT delivery Brightcove and NICE PEOPLE AT WORK (NPAW) worked together to enhance Brightcove’s brand-new player with NPAW’s YOUBORA Analytics, the most advanced real-time metrics... 

10 Feb

Quality of Experience (QoE): Option or Need?

Quality of Experience has become an overused term in the online video industry, but also one of the big challenges that online broadcasters and OTT´s have to deal with. Even more now, a period in which users spend more time connected to the digital world than to reality. Moreover, expectations and requirements... 

16 Feb