The game turned in a six-minute spell either side of half-time after the home side had opened up a 2-0 lead midway through the first half.
Defender Tony James sliced James Hurst's clearance past keeper Jamie Stephens to make it 1-0, then midfielder Mike Jones intercepted Stephens' clearance to score a stunning second from just inside his own half.
But Chris Zebroski tapped in a cross by Robbie Willmott to give Newport a lifeline and three minutes after the break Hurst pulled back Danny Crow inside the box as he raced through and was shown a straight red card.
Adam Chapman made no mistake from the penalty and Newport's fightback was completed on the hour when Willmott's cross was sliced into his own net by defender Connor Essam.
Willmott then missed a glorious chance to score a fourth when he was clean through and Newport needed late saves from Stephens to deny Jamie Proctor and Mark Connolly to ensure their progress.