Left-back Golbourne, who moved to Molineux for £700,000 from Barnsley last month, and striker Kevin Doyle struck in the first half as Wolves recorded a fifth successive league win for the first time since November 2008.
But it was rough on Swindon, who dominated proceedings before an 83rd minute strike from Dany N'Guessan got them back into it.
Wolves substitute Kevin Foley then looked to have ended Swindon's challenge in the closing stages before an injury-time strike by Ryan Mason set up a frantic finale.
Wolves took the lead after just three minutes through Golbourne in somewhat fortuitous circumstances. What was intended as a cross wrong-footed Swindon goalkeeper Wes Foderingham and dropped over his head into the corner of the net.
Despite that early breakthrough, Wolves then found themselves under some intense pressure as Carl Ikeme pulled off smart saves to deny Mason and Alex Pritchard, while Nile Ranger saw his penalty appeal waved away following a challenge from Bakary Sako.
Having managed to ride their luck, Doyle increased Wolves' advantage two minutes before half-time when he nodded home a cross from Leigh Griffiths at the far post.
While Wolves' luck was in, Swindon's was very much out as Mason then went on to squander a hat-trick of second-half chances in the space of six minutes, including one that hit the post.
N'Guessan finally earned Swindon some reward when he lashed home a pass from Mason before Foley converted a cross from fellow substitute Bjorn Sigurdarson.
Having failed to score from close range, Mason then found some success from 30 yards with a shot that whistled past Ikeme but it was too little too late for Swindon.