TubeBuddy Suggestion Using historical data to speed up A/B testing

[Melanie]

It would be nice to be able to pull historical data on an old thumbnail and use this data in conjunction with data collected during the test to see if a new thumbnail performs better. The user would have to select a date to dig back to in order to avoid selecting past dates when maybe an even different thumbnail was used.

It would be nice though because instead of perhaps 12days of switching back and forth between a new thumbnail and an old one, TubeBuddy could just show the new thumbnail and collect six days of data for the new thumbnail (or even less given a mass amount of historical data) and compare it to historical data for the old thumbnail.

 

[Beanie Draws]

I like this idea as an option, but remember that some video topics aren't evergreen and are based on day to day trends and news, which is why day to day switching would give better results than a period of now, vs a period from say months previous. For example, Spinosaurus's tail discovery was trending in 2020... updating the thumbnail in 2021, and comparing it to 2020 wouldn't work because the views would change based on the theme of the video and the news and timing, rather than the visual of the thumbnail it'self.

BUT I do love this idea for more general themed videos. I've done some A/B tests, and been curious to see how the A/B test results would differ a year later, so having something to compare THAT to would be a cool addition for long term evergreen videos.

 

[Melanie]

I like this idea as an option, but remember that some video topics aren't evergreen and are based on day to day trends and news, which is why day to day switching would give better results than a period of now, vs a period from say months previous. For example, Spinosaurus's tail discovery was trending in 2020... updating the thumbnail in 2021, and comparing it to 2020 wouldn't work because the views would change based on the theme of the video and the news and timing, rather than the visual of the thumbnail it'self.

BUT I do love this idea for more general themed videos. I've done some A/B tests, and been curious to see how the A/B test results would differ a year later, so having something to compare THAT to would be a cool addition for long term evergreen videos.

CTR is independent of traffic, so CTR would theoretically stay roughly the same whether you've got low traffic or high traffic as it's not counting the NUMBER of people who are viewing your video, but rather the percentage. Otherwise, A/B testing would be a constant and often fruitless search.