The hosts were forced to play 75 minutes with just 10 men after Mitrovic was shown a red card for a stamp on Francis Coquelin, with the Arsenal midfielder the subject of numerous fouls from Newcastle players.
Arsenal, who had struggled to break down Steve McClaren’s side beforehand, toiled without reward against an even more resolute outfit, but finally found the breakthrough after constant pressure.
Coloccini was the unlucky figure after Aaron Ramsey’s snapshot was palmed away by the impressive Tim Krul, with Alex Oxlade-Chamberlain’s follow-up effort deflecting in off the Argentinean defender’s heel.
Theo Walcott struggled to make an impact after being handed a start by Arsene Wenger ahead of Olivier Giroud, with the England international, Santi Cazorla and Ramsey thwarted by Krul and a strong Toon defence on more than one occasion.
Arsenal started the brighter of the two sides, having a penalty appeal turned down within 10 minutes after Hector Bellerin was felled by Florian Thauvin; the Frenchman marking his first Premier League start for his new side.
Despite their dominance in possession – reaching 80% at numerous points – Wenger’s side could boast only half-chances, with Newcastle staying firm in the face of their much-fancied opposition.
Moussa Sissoko was the first of six Newcastle players to be booked, with his third minute foul on Nacho Monreal setting the tone.
The hosts were reduced to 10 men just 10 minutes later. With a raucous home atmosphere setting in, new striker Mitrovic appeared to stamp on the shin of midfielder Coquelin, and was shown a straight red card by busy referee Andre Marriner.
After being booked just over 15 seconds into his Premier League debut, and cautioned after two minutes in his second appearance, Mitrovic will now find himself facing a ban after a heavy-handed introduction to the English game.
Despite their man advantage, Arsenal seemed to lack a cutting edge during the clash, with their first real opportunity falling at the feet of lone striker Walcott. The England man skied his chance high and wide after following up Alexis Sanchez’s 25-yard effort.
Newcastle attempted to break on the counter on a handful of occasions, with Sissoko and Georginio Wijnaldum providing the impetus, but the side were understandably fearful of committing too many numbers forward.
Just seven minutes into the second half, Arsenal eventually found a way through. Ramsey’s low drive was palmed away by Krul, with Oxlade-Chamberlain collecting the loose ball before firing in a shot which looked to be off-target before Coloccini’s inadvertent intervention.
Sanchez nearly doubled the visitors’ advantage five minutes later, but Arsenal could not find a decisive second goal, with second-half substitute Giroud struggling to make an impact. Papiss Cisse was handed 10 minutes after being brought on for Sissoko, but the Magpies failed to create any real goal scoring opportunities.
The win sees Arsenal record their third unbeaten game in a row after falling to defeat against West Ham on the opening day, while McClaren remains without a league win at Newcastle.