County drew first blood after 32 minutes with a quality strike. Yoann Arquin met Alan Sheehan's cross from the left and struck a sweet volley on the angle from 20 yards to claim the opener and his seventh of the season.
Three minutes before the break Rotherham levelled after their best period of the half.
Daniel Nardiello's determination paid off when he sent Michael O'Connor down the left and his inviting low cross was met by Ben Pringle on the run to drill home.
"Don't write us off in the replay," said Evans defiantly.
"Credit to both teams, it was a fantastic cup tie from start to finish. It was a fantastic tie and it is far from over and we look forward to another one in 10 days' time.
"Notts County started better than us. They got the goal and for 20 to 25 minutes they were better than us.
"Then a couple of our players managed to get their foot on the ball and we passed it and moved it.
"In the remaining part of the half, we equalised and perhaps should have scored when Mark Bradley hit the post.
"Then I think for the second half we were by far the better team. We made a chance in the 90th minute when Daniel Nardiello blasted it high and wide.
"If you wanted any player in the camp on that, it would be him."
