Following their 2-1 Champions League defeat by Benfica in Lisbon in midweek Hoops boss Neil Lennon had claimed fatigue was impacting on his squad and it looked that way as they never got going against the neat and tidy visitors.
Kris Commons and Mikael Lustig missed good chances for the home side in an otherwise dull first half but there was no second-half surge from the champions and indeed, after McKay notched in the 65th minute from close range, he later spurned a good chance to double their lead when he hit the bar with only Fraser Forster to beat.
Caley defender Graeme Shinnie fired a 25-yard free-kick over the bar early on before Commons had an effort which failed to trouble Caley keeper Antonio Reguero.
The Highlanders' goal almost came under threat in the 17th minute when Beram Kayal played in Lustig but his attempted cut-back from the byline to Gary Hooper was thwarted by a fine Josh Meekings tackle.
The champions then squandered two good chances to break the deadlock. In the 39th minute, as the Parkhead side broke swiftly, Hooper sent Commons through but the Scotland midfielder's angled-shot from 12 yards was wayward.
Two minutes later Lustig had a similar and possibly better opportunity when Hooper cleverly allowed a Kayal pass to run through to the right-back but he hit the side-netting with his unconvincing effort.
The visitors, resolute in defence, carried more than a hint of danger on the break in the second period, with Parkhead defender Efe Ambrose coming to Celtic's rescue in the 54th minute when he cut out a dangerous cross from midfielder Aaron Doran and although the corner came to nothing it served to accentuate the sense of frustration inside Celtic Park.
Caley's goal was not a surprise and came through another swift counter.
Roberts took a pass from Doran inside the Celtic box, turned inside Adam Matthews and Mulgrew, and fired the ball across for McKay to guide the ball past Forster, with the home fans claiming offside.
Paddy McCourt replaced Tony Watt and Brown came on for Matthews but before the second substitution there was another scare for the Hoops when Andrew Shinnie played in Tudur Jones but he could not set himself properly and his effort was easily saved by Forster.
Celtic, urged on by their fans, stepped up the pressure and Gary Warren hit his own post under pressure before Reguero made a good save from what looked to be a Samaras header.
With nine minutes remaining and the Caley goal under siege, Reguero made a fine save from Commons powerful drive from 16 yards with the Hoops midfielder firing the rebound over the bar.
In the 83rd minute Caley should have scored the second when they broke in a two-versus-one situation. Doran set up McKay who hit the bar with only Forster to beat before the Caley midfielder's shot from the rebound was cleared off the line by Ambrose.
The Highlanders were made to defend for the rest of the match but held on for the win, as Celtic Park echoed to the sound of boos at the final whistle.