Kris Commons scored after just six minutes at St Mirren Park in a match that saw Victor Wanyama dismissed early in the second half before referee Bobby Madden was at the centre of further drama with the spot-kick award.
Gary Hooper almost scored inside three minutes with an audacious chip from just inside the penalty area which had St Mirren keeper Craig Samson well beaten.
Samson was helpless again three minutes later as the Buddies failed to clear the danger from a Celtic corner and Efe Ambrose lofted the ball into the box for Commons to plant a downward header into the corner of the net.
St Mirren responded strongly as Fraser Forster deflected Paul Dummett's net-bound shot past the post with his thigh, and Marc McAusland's header was cleared off the line by Beram Kayal at the back post.
Dummett bundled Forster into the net before the ball followed as St Mirren continued to press, and the hosts also had a penalty claim waved away when the ball hit Georgios Samaras on the shoulder.
Celtic had lost Commons when he was carried off with an ankle injury after an accidental tangle with Kayal, before Wanyama (55) was harshly given a straight red card for catching McGowan on the shin as they challenged for a bobbling ball.
The visitors appeared to be seeing the game out when Madden pointed to the spot after Esmael Goncalves hit the turf in a tussle with Emilio Izaguirre, and McGowan drilled the penalty home.