Au contraire, if you can't see that a countries respective population size offers only a very limited explanation of their success on the sporting field you're an idiot.
If it were population based, how would you explain the relative success of Australia (Tennis, Swimming, Rugby, Hockey, etc) New Zealand (Basketball, Rugby, Golf), The Netherlands (Hockey, Soccer, Volleyball)
None of these countries have a population of more then 20 million, and with the exception of perhaps rugby, none of them have anywhere near the highest total amount of people playing each respective sport.
Culture, State of the art sporting facilities, World Class Coaching, a top domestic league, and an abundance of resources (monetary or otherwise) are far more important factors.
quote: "assuming they are countries who actually play sport to similar levels"
That's just a weak as disclaimer.