Shaun Harrad's goal in first-half injury time looked to have secured the win for the Whaddon Road hosts on Tuesday night, but a dazzling long-range thunderbolt made it four without defeat for John Ward's side.
It was Rovers who started the better with John-Joe O'Toole forcing a good save from Robins keeper Scott Brown, who then blocked a close-range Tom Parkes effort.
But after weathering the early storm, Cheltenham began to take control with Harrad drilling an effort at visiting keeper Steve Mildenhall before a Steve Elliott header was cleared off the line by defender Michael Smith.
The breakthrough came with the final move of the first half, Harrad pouncing on the loose ball after Kaid Mohamed's shot had been blocked and drilling his effort home from 12 yards.
Jermaine McGlashan was denied a second Cheltenham goal 10 minutes after the break when Mildenhall saved from close range while O'Toole went close at the other end only for Brown to deny him with another excellent stop.
But with the last move of the game, Norburn let fly and his effort nestled into the top right-hand corner.
