How do you Determine a Winning Campaign?

Is it a winner?

Marketing is a science, and just like in science the only way to learn if something will have a positive or negative impact is to develop an experiment and test your hypothesis. At B2E we love to test and learn because, well, we are data scientists. And we enjoy helping our science marketing friends with their testing agendas and learnings as well. One question we often receive from them is – how do I know if my test is a winner?

There’s a calculation for that

The short answer is, there is a calculation to determine if your test is a winner. Across the marketing discipline, the most common practice for determining a winner is to calculate the statistical relevance of the outcome. Statistical significance is the likelihood that a relationship between two or more variables is caused by something other than random chance. Simply put, you want to determine if the outcome is repeatable.

Confidence is key

Some marketing organizations determine a relevant win as any test achieving a statistical confidence of 90% or more. This means that your test has a 90% chance of repeating the outcome you achieved in testing. The higher your confidence level, the more likely your test will carry out your expectations again and hopefully on a larger scale. It is not uncommon for marketers to strive for a 95% or above statistical confidence before they call a test a win.

Reaching 95%

Reaching 95% confidence can be a challenge if your testing quantities are too small. If your testing volumes are too small, then your win has to be even more dramatic in order to achieve this level of statistical significance. If you’re having difficulty achieving 95%, you may need to adjust your testing plan to extend its length or pump up the volume.

It’s a tie…now what?

If your testing results in a tie, or a non-significant win, re-testing would be best. Even though your test wasn’t a winner, it also wasn’t a loser. Take your learnings and determine your best path forward. You can choose to re-test the same test or a slight variation of the initial test to see if a greater response is achievable.

If you need assistance in designing a testing strategy or in analyzing your test results, contact us at B2E Direct – we have a lot of experience in this area.

Peace,

Greg