The 42-year-old former Egoalkeeper is one of the most high-profile black players of the last 20 years and is now plying his trade with the eighth club of his career, Bournemouth.
James is planning a career in management after completing his UEFA Pro-Licence and does not think ethnicity will prove an issue.
"I struggle with the racist issue in football because I don't see it, and that's not because I've got my head in the sand," he said.
"In the earlier days, yes, but the game's changed.
"There are some wonderful organisations out there which have helped football become a much more enjoyable game for everyone.
"Stuff in the crowd being aimed at players - that's gone, or pretty much gone. I don't hear it any more.
"With regards to the playing side of things, I don't look at myself as any different from the guy who gets changed next to me, and I'm not going to fly anyone's flag in order to join some 'gang', which doesn't need to be joined .
"If you want to go on a coaching course to become a manager then give yourself a chance.
"If you don't want to go, and moan about not getting jobs, well, probably because you haven't been on the course is the reason why you haven't got a job."
James admitted there were "not many ethnic players" on his coaching courses but also pointed out there were not many English managers in the Premier League either.
"I don't think it's a racist issue," he added. "I just think it's a case that the numbers aren't there.
"A bit like the goalie situation. Not many goalies go on the course, so, therefore, you're not going to have many goalies make it."
James was speaking yesterday at the Leaders in Football conference at Stamford Bridge, where he accused football's anti-racism campaigners of making too much of the issue in order to keep their jobs.
Incidents involving John Terry and Luis Suarez have seen racism become one of the biggest topics in football over the past year, although James does not believe it remains an issue.
"I think the organisations which have done so good on the terraces are still employed looking for stuff to be shouted about," he said.
"And a lot of the issues that we've gone on about in the last season or so, it's more about people driving the issue than the issue being a real focus."
Asked to elaborate, James added: "Well, some people get paid for doing certain jobs and certain jobs involve bringing stuff to public notice.
"The JT thing, it could've been dealt with quietly and done. He could've served a ban if he was going to serve a ban, rather than it being a six or eight-month thing."
