Having dominated the game Shaun Morrison popped up to head in Alan Sheehan's corner in stoppage time to win it.
The Robins were the only team in it for the vast majority of the game, but looked like being stifled by Lee Clark's men.
After dominating the first half, Swindon nearly took the lead 30 seconds after the interval but although Thomas Dossevi's 25-yard effort looked to be heading in, his strike went just wide.
Dossevi and the impressive Vincent Pericard saw their shots blocked as the game entered its final third, while another miss by Pericard was greeted by groans by the Reds' faithful.
But just as time looked to be running out, up rose defender Morrison to head into the bottom corner to raise the roof at the County Ground.