Toure warned that black players could boycott the tournament after he complained of being racially abused by CSKA Moscow fans during Manchester City's Champions League clash on Wednesday.
UEFA president Michel Platini has demanded to know why his officials failed to act on Toure's complaints, while CSKA, who have denied any racist abuse took place, have been charged over the conduct of their supporters.
Chelsea boss Mourinho claimed black players were so integral to the history of the game that to hold a World Cup without them would be unthinkable.
"I respect Toure's opinion, but I disagree," said Mourinho.
"The history of football was made equally by many races. And the black players have a fantastic contribution of what football is.
"The World Cup is the biggest expression of national team competitions - races, people from different parts of the globe, people from every continent. And the black players are very, very important for that.
"Who is more important, the billions of people in love with the game around the world? Or a few thousand that go to football stadiums and have disgraceful behaviour in relation to black players.
"If I was a black player I would say the other billions are much more important, so let's fight the thousands, but let's give to the billions what the billions want. And the billions want the best football and the best football without black players is not the best football, for sure.
"The players must realise that football is a beautiful game, they must realise that a huge percentage of the people that go to football stadiums are pure people that respect the difference and respect everybody and those people are more important than small groups that express themselves in a negative way."
Mourinho emphasised the need to continue the fight against racism and said he had "sympathy" for Toure's situation.
But the Portuguese insisted football was trying to combat the problem and that it was not just the responsibility of the sport to do so.
"What football can do against racism football is trying to do," he said. "This country tries to do, other countries try to do. UEFA and FIFA have campaigns.
"But I think racism is about principles. And principles don't start in football. They start in your house, when you're a kid. That's where it starts. Football is not guilty and football should not be responsible to clean racism out. I think families can do it. Parents can do it."
Mourinho's stance was supported by his Arsenal counterpart Arsene Wenger, who claimed it was too early to think about a boycott.
Wenger said: "I don't know what really happened there, but you cannot tolerate racism of any kind and you have to fight against it. How (do) you do that?
"Honestly I believe UEFA is taking action in the right way and to go as extreme as (a boycott), it's a bit early to do that because it's not proven what happened.
"I believe that Russia itself has to fight against that and of course you want everybody to be active on that."
Chelsea v Manchester City is live on Sky Sports 1 HD from 3:30pm on Sunday.