• Guest NEW FORUM SECTION! Click Here to learn more!

New A/B Testing Bug?

Tangerine Travels

Newbie Member
5
6
Hey Tubebuddy team!

I'm loving that you implemented one of the A/B improvements that I suggested a couple of weeks ago (watch time per impression).

I think there might be a bug in the calculation of this metric. In the attached screenshot you can see:
1. The variation has a significantly higher click-through rate
2. The variation has a slightly higher average watch time
3. The variation has a slightly lower watch time per impression

How could #3 be true if #1 and #2 are also true?
 

Attachments

Andrew

Superman
Administrator
4,533
31
youtube.com
Hey Tubebuddy team!

I'm loving that you implemented one of the A/B improvements that I suggested a couple of weeks ago (watch time per impression).

I think there might be a bug in the calculation of this metric. In the attached screenshot you can see:
1. The variation has a significantly higher click-through rate
2. The variation has a slightly higher average watch time
3. The variation has a slightly lower watch time per impression

How could #3 be true if #1 and #2 are also true?
Tagging @Phil
 
OP
OP
T

Tangerine Travels

Newbie Member
5
6
Also, I'm sure you guys are aware because you've been doing a lot of updates to it lately, but CTRs haven't been updating for the past couple of days.