I have questioned this myself, but in my opinion (will need some confirmation on this thought) there's more to SEO and ranking than the SEO it'self. You can craft "perfect" SEO, but then it will be your thumbnail, topic and video it'self that will determine how it ranks long term. SEO ranking simply shows how high up the video will be presented, but that's before it's actually clicked, the thumbnail and topic etc will bring the actual clicks. The SEO will bring the impressions and those impressions bring the OPPORTUNITY for clicks. So in my opinion, even if two channels have the exact same perfect SEO for a video, the history of both channels, the reputation, the thumbnails and topics etc will still be different which will make the "perfection" of the SEO different.
I may be wrong, but that's my understanding.