Aron Gunnarsson levelled Edin Dzeko's opener in the Welsh capital before Fraizer Campbell struck twice and despite a late strike from Alvaro Negredo, Mackay's side held firm to record their first win in the Premier League.
"I spoke to the players about the fact that they have to believe they belong here," he said.
"We were undoubtedly playing against one of the top teams in Britain and Europe and a team filled with top, top talent.
"But you want to test yourself against the best. I want to test myself against the best managers, and my players should want to test themselves against the best players in the world and be in the best league in the world.
"They are here on merit, they won the Championship last year and they were out there against City and you saw the belief.
"I have got technically very gifted players, and when we got the ball back we could counter-attack and hurt a team who are a top side.
"The best have come here and we deserved to beat them. That should give the players confidence we are here to compete every week and make sure it will be difficult for opposition to play against us."
Mackay reserved particular praise for Gunnarsson, who played a crucial role in the victory and has emerged as one of his key performers.
"Aron became captain of Iceland when he was just 23, but he is a player who just goes quietly under the radar," added Mackay.
"As last season went on he became more and more important to our team. I think it was about him growing up as a player over the last two years and he has done that.
"He has become a more mature player and with an experienced player in Gary Medel alongside him, he showed that maturity against top players."