Rovers capped off a positive start after 16 minutes with John-Joe O'Toole handed an easy finish after Eliot Richards had seen his effort hit a post.
One minute before the break the Pirates scored a second with Oliver Norburn finding the corner with a low effort after the ball had broken to him on the edge of the area.
Rovers thought they had put the tie to bed with Chris Beardsley capitalising on a loose ball to fire home four minutes after the restart before the game changed direction.
York were handed a lifeline with 20 minutes to go as Lewis Montrose was fouled in the area with Wes Fletcher sending Steve Mildenhall the wrong way from the spot.
Fletcher then turned the game on its head one minute later as he struck from distance but it wasn't enough as York exited the competition despite a brave fightback.