The Welsh club appeared on the ropes after Jimmy Ryan had cancelled out Danny Crow's earlier effort.
But Liam Cooper hauled down Conor Washington in the penalty area, resulting in his dismissal, and Adam Chapman duly fired home from the spot.
Andy Sandell then completed the job before Marc Richards claimed an injury-time consolation to set up a frantic finale.
Newport made the perfect start, with Crow opening the scoring after just six minutes. The forward guided Robbie Willmott's cross past Tommy Lee with a pin-point flicked header.
The visitors began to establish a foothold on the encounter but they fluffed a succession of chances. James O'Shea struck the woodwork after 40 minutes and the rebound fell to Armand Gnanduillet but he headed over.
Lenny Pidgeley made a host of saves in the second half but he failed to deal with O'Shea's long ball just after the hour mark and Ryan pounced to equalise.
Cooper then saw red after 74 minutes and Chapman sent Lee the wrong way from the penalty spot, tucking the ball into the bottom corner.
Sandell tapped in a rebound from close range after 87 minutes to put County 3-1 ahead before Richards struck from the centre of the box, but it was not to be for the second-placed Spireites.