Kris Commons scored from the spot in the 18th minute after Gorgie midfielder Jamie Hamill had been penalised for handball after a challenge with Anthony Stokes.
The former Kilmarnock player, who was only available after successfully appealing the red card he picked up against Inverness, was as surprised by the decision as most inside the stadium - although suggestions later were of his hand being in an unnatural position.
And after receiving a yellow card, Hamill's frustration was compounded when former Nottingham Forest player Commons drove the subsequent penalty past Jamie MacDonald.
Jason Holt equalised in the 56th with a deflected drive from the edge of the box but goals from Anthony Stokes and debutant Teemu Pukki took the points back to Glasgow.
Afterwards Locke said: "Unfortunately for us, there was only one man in the stadium who thought it was a penalty and he was the most important one.
"I think you can see with the reaction of players and not one Celtic player appealed for a penalty and no one in the Celtic dug-out appealed for a penalty but, unfortunately for us, the referee gives it so there's not a lot you can do about it.
"There's no point (talking to him) because at the end of the day, the decision is not going to get changed.
"The problem is you don't get an explanation from anyone and if Hamill's hand was in an unnatural position that puzzles me because his hands weren't in an unnatural position from where I was.
"It is difficult enough to beat Celtic but when you get a decision like that it makes it tougher.
"But there is no point in moaning about it, there was plenty time to get back into the game.
"I am not going to use that as an excuse and there is a lot more positives than negatives I can take out of the game."