The Jambos were 2-1 down with 11 minutes to play but three goals in six minutes ensured Locke would be celebrating his first victory since securing the job on a permanent basis.
Michael Ngoo scored twice and substitute Jason Holt was also on target, with Evangelos Ikonomou putting through his own net to give Hearts the lead in the contest at 3-2.
"It's a great feeling. I said to the players afterwards that I want more from what I saw of them in the second half and less of what they gave me in the first," he said.
"We tried to be more positive in the second half. I wasn't happy with what I saw in the first. We tried to go more attacking.
"Sometimes as a manager you get lucky with your substitutes. In the first half we didn't play well enough. Second half I was a lot happier.
"County will feel aggrieved that they didn't get anything out of the game and justifiably so.
"I want to reserve special praise for our goalkeeper, Jamie MacDonald. He made three outstanding saves that kept us in the game."