Kallum Higginbotham's acrobatic early strike looked like being enough to separate the sides until the visitors were awarded a penalty in the final minute.
William Gros was upended in the box by a crude challenge from Aaron Taylor-Sinclair, who was shown a straight red card, and Muirhead was on hand to stab home after Kris Boyd's penalty was saved.
There was little indication of the drama to come after Thistle had been rewarded for their bright start with a superb opening goal in the third minute.
Stephen O'Donnell's long-throw was half-cleared by Manuel Pascali's header, but Higginbotham took to the air and connected with a sublime overhead kick that Craig Samson was powerless to keep out.
O'Donnell came close to adding a second with a rasping drive that narrowly cleared the bar, while Paul Gallagher - making only his second appearance for the hosts - dealt with a Sammy Clingan free-kick from distance.
Samson did well to hold a long-range curler from the confident Higginbotham after excellent set-up play from Steven Lawless, while Muirhead sliced wide from Boyd's flick at the other end.
Taylor-Sinclair's free-kick from the edge of the box was palmed away by the alert Samson shortly after the break.
The visiting keeper was pressed into action again to save another effort from Lawless, and Gallagher then pulled off the save of the match to deny Boyd and Conrad Balatoni cleared the rebound off the line.
But the game turned in the last minute after Gros was scythed down by Taylor-Sinclair, although Gallagher can count himself unlucky after blocking Boyd's penalty into the path of Muirhead.