Your SEO Score is a reflection of the Best Practices that you have followed. It shows you what you have done and what you need to do in order to best give YouTube the information it needs in order to correctly place your video in front of the correct audience.
But the score of how well you achieved this is less important than the work you actually put into it. Your focus needs to be on finding the best possible keyword phrases to center your video's topic around. You need to make thumbnails that are going to completely stand out against your competition and you need content that is going to keep these viewers around longer than the videos that your competitors are making.
Think about it like this; if you optimize 100% for the search term 'NBA' do you think you stand a chance against ESPN? There is no way you are going to rank for that term, it's terribly overused. Far too competitive. But if you optimize 85% for the term 'Oliver Millers Best Day in the NBA' you are going to stand a far better chance of ranking.
So it is important to get the best SEO Score possible, but the real work isn't your score so much as the quality of work that was put in to get that score.