Well, since genetics are pretty much arbitrary, and the environment that shapes ones early upbringing is unpredictable, most people won't have infinite potential. There are too many uncontrollable factors which predetermines an individuals competence and ability.
So I would agree that no, not everybody has the potential to do something.
HaSaSSin by your logic someone with a trait with thin fast fingers and a photographic memory and quick reasoning would be able to be faster than someone who dedicates exponential time to the puzzle.
Yeah, I agree with whoever mage quoted. People could become competent or highly competent even in areas in it would logically follow that they should not be that good at. But within logical parameters.
Of course some people who are dumb (can't speak, not the other kind) can't sing. That's physically impossible. A blind person can't be the best video game player, because he/she can't see the images. A deaf person can't be a music producer because he/she can't hear music. All of those are examples of the limitations of the human body. Normal people can do all those things but they can't be professional in all those things because they have to choose one path to go on. They can develop the skills they need to get a profession but they can't specialize in all of them unless they spent their lifetime practicing and studying.