With 12 minutes remaining, County were seemingly on their way to three vital points thanks to two goals from striker Steffen Wohlfarth.
But after Holt levelled matters on 78, Evangelos Ikonomou bizarrely put through his own net and Ngoo killed off the Staggies with a clinical strike from the edge of the box.
County had opened the scoring midway through the first half when Wohlfarth, who missed an excellent early chance to break the deadlock, made amends with a wonderful individual effort on 22.
Controlling the ball close to the touchline, he fended off two defenders before finding the top right-hand corner of the net with a fierce strike from a tight angle.
Ivan Sproule should have added a second four minutes later when he raced clear of the home defence only to see his indecisive shot easily smothered by keeper Jamie MacDonald.
The closest Hearts came to scoring before the break was from the right foot of Kevin McHattie who flashed a curling long-range shot off the angle of the post and bar in the 36th minute.
Hearts did level matters four minutes into the second half when the powerfully-built Ngoo rose highest to head home McHattie's precise cross.
But County were back in front six minutes later when Wohlfarth clinically headed Iain Vigurs' corner past a helpless MacDonald from seven yards.
The visitors remained in control until 12 minutes from time when Hearts equalised again thanks to an opportunist strike from Holt who volleyed past keeper Michael Fraser from the edge of the box after County had failed to clear a corner.
Within 70 seconds the hosts were in front following a comical own goal through Ikonomou.
The hapless defender, seemingly under no pressure inside the six-yard box, sliced the ball into his own net as he vainly attempted to clear Walker's weak goal-bound shot.
Ngoo scored his second goal of the match, and his side's fourth in the 83rd minute, when he found the bottom right-hand corner of the net after latching on to John Sutton's assist.
The stunned visitors were unable to respond in the final few minutes as Hearts claimed only their second win in 10 SPL games.