Cameron told the Jewish Chronicle that Spurs supporters should not be prosecuted for chanting the word so long as they are not motivated by hate.
"There's a difference between Spurs fans self-describing themselves as Yids and someone calling someone a Yid as an insult," Cameron said.
"You have to be motivated by hate. Hate speech should be prosecuted - but only when it's motivated by hate."
Tottenham fans have called themselves "Yids" "Yiddos" or the "Yid Army" for years as an act of defiance towards those who ridicule the club's links with the Jewish community.
The Football Association says that such chanting could lead to a banning order or criminal prosecution.
But to prosecute, police would have to prove that the person chanting the word does so with the intent of causing offence.
Tottenham insist there is no malice intended by the chanting, although they are planning to distribute a questionnaire to season ticket holders asking them if they think it is time to stop the chants.
Spurs boss Villas-Boas says he did not object to the Prime Minister having his say on the issue.
"He can get involved anywhere, he is the Prime Minister," Villas-Boas said. "I think his intervention was probably what Spurs fans would want to hear.
"It was straightforward in what he came out with and it was clear."
Villas-Boas hopes the debate does not drag on without conclusion. "Hopefully this won't be an ongoing debate that will lead to nowhere," the 35-year-old said.
"I think our fans sing it with pride, it is something that they defend. It is not sang with offence.
"I see no problem with it. The problem is finding out what is seen as an offence. That is why the FA has come out and made a statement so hopefully this won't become a debate that leads nowhere."