The Manchester City defender took the decision last summer to represent Team GB at the London Olympics, rather than go on standby for the European Championship.
Richards insists he made the right call for him, with a 'once-in-a-lifetime' opportunity presenting itself.
He is, however, prepared to acknowledge that his England days may now be over at the age of 25 - after earning just 13 caps - with Glen Johnson and Kyle Walker the favoured options at right-back for Roy Hodgson.
Richards told The Sun: "It was one of those. Do I go on standby or play in the Olympics? I wanted to go and play under (Stuart) Pearce as I knew him from here at City.
"It was a decision I made, people will always think negatively about it.
"But, from my point of view, we'd won the league that year and the FA Cup the season before. Kyle Walker got injured and I was still being asked to go only on standby.
"Was I wrong for making the decision? Maybe. But I wanted to play in the Olympics. It was a once-in-a-lifetime chance and I enjoyed it. It was one of the best times of my life."
He added: "I still want to play for my country, everyone does. But what can I do now?
"Of course, there's still a lot of time. But time waits for no-one. We've got young players coming through now like Kyle and Nathaniel Clyne.
"So, yeah, time is still on my side. But sometimes with England, if you don't get in when you're young and stay in, they look to go younger. All I can do is play well.
"People ask me the question about getting back in the England squad and I think, 'I won the FA Cup and then the Premier League in 12 months and I still didn't get in'. So what can I do? I couldn't do any more than that."
Any international recognition at this stage would be a bonus for Richards, with the City man's focus locked on nailing down a regular role at club level rather than chasing a 2014 World Cup dream.
He said: "I'm coming back from injury and I've only played a few games this season. So do I expect to be in the England squad right now? No.
"On ability maybe I should be. But if you go on form and if people are playing every week, probably not."