The visiting Saints had the better of the chances throughout and deservedly led through debutant Adam Campbell - only for Rory McKenzie to equalise in the first half and Boyd to snatch the points at the death.
Lee Ashcroft nodded an early chance at Marian Kello and McKenzie rattled the bar with a vicious long-ranger - but home fans were left stunned in the 19th minute when Campbell opened the scoring.
The 19-year-old, who is on loan from Newcastle, swivelled sharply to lose his marker and send an accurate finish low into the corner of the net from 20 yards out.
The excellent Craig Samson prevented the Buddies from quickly doubling their advantage - the home goalkeeper saving from John McGinn and Conor Newton before McKenzie's superb effort restored parity on the half-hour mark.
Steven Thompson and Paul McGowan further tested Samson before the interval and after it, veteran home striker Boyd failed to find the target at the climax of a rare Killie attack.
Danny Grainger had already had a shot blocked by the time he forced another excellent Samson save on the hour-mark while at the other end, Kello reacted smartly to keep out efforts from Robbie Muirhead and McKenzie.
Campbell, a live wire throughout, saw two late shots blocked having worked openings for himself but the killer blow came seconds before the final whistle, when Boyd clinically rifled a winner into the far corner having sprung the offside trap.