The impressive Conor Sammon's early goal gave Kilmarnock a richly-deserved half-time lead but McGinn volleyed home following slack defending just after the interval.
McGinn further stunned the hosts with a deflected second in the 69th minute and St Mirren produced a much-improved defensive display in the second half to hold on.
Kilmarnock were rocked before the game by news of Kyle's knee cartilage problem, which will rule him out of Scotland's World Cup qualifiers against Macedonia and Holland.
But the Ayrshire side took less than four minutes to suggest they could cope without their skipper as Sammon added to his midweek cup double against Morton.
The Irishman ran on to a clever through-ball by Kyle's replacement, David Fernandez, that caught the Saints defence square and slotted the ball into the far corner from 20 yards.
St Mirren, who had Hugh Murray back in midfield in place of Tom Brighton, settled somewhat after a disorganised start but Kilmarnock continued to take the game to them with the energetic Sammon putting the visitors under enormous pressure.
Fernandez cut inside from the left and fired a shot that Paul Gallacher held and Gavin Skelton's cross just evaded Sammon following good build-up play.
Killie were also without the injured Jamie Hamill and Danny Invincibile but Garry Hay had recovered from a back problem to take the captain's armband and Manuel Pascali slotted well into central midfield.
Alan Combe's first real test came in the 32nd minute when he dived ow to hold Billy Mehmet's 30-yard free-kick but Kilmarnock remained in firm control.
Sammon again threatened when he drove the ball across the six-yard box following a one-two with Mehdi Taouil but the Saints defence cleared. However, Kilmarnock's impressive first-half efforts were undone within three minutes of the restart when McGinn volleyed the equaliser from 10 yards.
Hay made a mess of a clearance from David Barron's deep cross and Dorman cut the ball back for McGinn to smash into the top corner with his weaker right foot.
Pascali was wide with a deflected volley and a header as Killie sought an immediate reply and Gallacher pulled off a excellent diving save from Sammon's powerful first-time effort from a difficult angle.
Killie looked the likelier team but St Mirren came back strongly just before McGinn stunned the hosts again. Dorman's cross fell for the midfielder on the edge of the box and his low shot deflected off Tim Clancy and past Combe at his near-post.
Frazer Wright's header was inadvertently blocked by Michael Higdon from Craig Bryson's corner as Kilmarnock quickly reasserted themselves as an attacking force.
But, with St Mirren now having something to defend, they did not have th space to play as they did in the first half.
Most of Kilmarnock's pressure came from high balls and the visiting defenders showed they had fully recovered from their shaky start.
Jack Ross produced a crucial clearance from Clancy's deep cross and Lee Mair knocked away Hay's cutback before Saints enjoyed a comfortable final 10 minutes.