The Manchester United legend, part of England's 1966 World Cup-winning team, sympathises with England boss Roy Hodgson due to his limited selection options.
"The number of players we have to choose from is ridiculous," said Charlton.
"Didn't Fabio Capello say there were only 94 footballers he could select out of the whole of the league?
"You can't expect them all to be world-class players.
"I would like to think that one day we will be favourites going into a World Cup, but we are not any more.
"I am not happy about it, but I don't know how you can solve it either.
"When I am asked to give my opinion, I always say yes (we can win the World Cup) but it is a fool's errand.
"People wonder whether we will ever win it again. I worry about that as well.
"You need good players. But if all the spaces at English clubs are taken by foreign players, you have no chance."