Answered Split Testing Label Confusion - Can You Help?

[Zanet]

I use Split A/B Testing and I have a question
I find the Labels Confusing once the Thumbnail test has completed
All my tests involve a thumbnail variation UNTIL Statistically Significant
So there is no completion until that's acheived

There seem to be 3 results:
A) Complete
B) Complete Variation Set
C) Stopped, Variation Set

See image below

So whats the difference between B) Complete Variation Set and C) Stopped, Variation Set

 

[Stanley | Team TB]

How many cycles did this run for? In option B the test stopped due to it reaching statistical significance. But option C was simply stopped... if it was set for statistical significance but there was never that much of a difference then the test will only run for 99 cycles before stopping without picking a thumbnail.

 

[Stanley | Team TB]

Thanks for answering but i don't think that's right.

None of them had 99 cycles and they are all set until statistical significance
it seems that only 20 cycles took place on B
A was only 2 cycles then it stopped saying " This is considered Statistically Significant "
B was 20 cycles and it stopped on its own " This is considered Statistically Significant "
C was 24 cycles and then stopped " This is considered Statistically Significant "

I spoke to the developers... it appears as though there may be a bug causing different responses for the same thing. Basically, unless you have stopped any of these yourself they should all be considered 'Complete' and not 'Stopped.' (The difference being that if it is 'Stopped' it is because you stopped it yourself). They are working at resolving that now.