Goals from Yoann Arquin and Yann Sango'o gave County a first-half lead before Boyd pulled one back shortly after the restart.
Filip Kiss saw red for the away side with 15 minutes to play before Boyd grabbed his second of the game with seconds left to earn Kilmarnock a point.
The first chance fell to Kilmarnock's Rory McKenzie who drew a save from Michael Fraser after 15 minutes before Michael Tidser forced a save from Craig Samson at the other end minutes later.
Brian McLean and Richard Brittain both called Samson into action shortly after as County began to dominate proceedings and they didn't have to wait long to take a deserved lead.
Kiss played a measured ball to Arquin on the half-hour mark before the Frenchman composed himself and fired home from 18 yards.
The visitors continued to take the game to the home side and six minutes later extended their lead when Sango'o headed past the helpless Samson.
That's the way it stayed until the break but the home side came out with a greater side of purpose and pulled a goal back three minutes after the restart.
Former Rangers striker Boyd rifled the ball into the top-left corner, thereby scoring in all three fixtures contested between the sides this season.
And the home side were almost on level terms three minutes later when McKenzie shrugged off his marker before striking the ball from 20 yards only to hit the post.
Muirhead almost grabbed an equaliser after beating two players and firing the ball towards the left corner only for an excellent save from Fraser.
Kilmarnock's hopes of getting something out of the game were boosted when Kiss was dismissed for the visitors after a second bookable offence.
They looked like slipping to defeat but Chris Johnston's stoppage time cross was headed in by Boyd as the home side snatched a point at the death.