Killie's fine recent run looked like coming to an end when Peter MacDonald and Graham Gartland put Saints 1-0 and 2-1 ahead in Saturday's clash.
But equalisers from Chris Maguire and Garry Hay - the latter's first goal since December 2008 - set the stage for Kyle to bag his fourth against Derek McInnes' men this season.
Killie recalled Danny Invincibile and Jamie Hamill, while Saints handed Paul Sheerin his first SPL start for five months and drafted in Murray Davidson, Cillian Sheridan and Gartland.
Killie were looking to complete a hat-trick of victories over St Johnstone.
During a 21-game sequence in which they won only two SPL games, both successes came against this afternoon's opponents.
Saints midfielder Jody Morris had the first effort on goal today, his volley dipping just over Cameron Bell's crossbar in the third minute.
Scott Severin then needed all his experience to avoid conceding a penalty when Sheridan bore down on goal, the defender timing his tackle to perfection.
Gartland almost gifted Killie a 12th-minute lead with the most spectacular of own goals, a horrible sliced clearance that was looping in but for a magnificent save from Graeme Smith.
MacDonald sent a shot on the turn straight at Bell before Hay drove just over from distance.
Sheridan had the first intentional clear-cut chance when he got in behind the Killie defence after 26 minutes but Bell smothered the striker's poked finish from a tight angle.
Saints took the lead nine minutes before half-time when Dave Mackay fed MacDonald, whose attempted lay-off deflected back to him off an opponent for him to sweep home left footed.
MacDonald injured himself in the process and was replaced three minutes later by Kenny Deuchar.
Severin had to bail out Bell with a clearance behind when the new man skipped past the goalkeeper in the box.
But the home side levelled in the final minute of the half when a Deuchar header and Duberry slice from a Killie free-kick inadvertently set up Maguire to lash home from close range.
The same player almost caught out Smith with a mishit cross five minutes after the restart.
But Saints were back in front three minutes later when Gartland claimed the faintest of headers on a floated Danny Grainger free-kick that looked like it was going in anyway.
At the other end, Kyle glanced Hay's free-kick wide before manager Jimmy Calderwood threw on Connor Sammon for Tim Clancy.
Maguire saw his 20-yard shot blocked behind after a lightning Killie breakaway, while Morris became the first player booked for a cynical 64th-minute foul on Liam Kelly.
It proved costly as Maguire fooled everyone by passing to Hay instead of shooting and the full-back's first-time strike flew into the far corner.
Mehdi Taouil had replaced Invincibile in the meantime as Killie went on the attack.
Sammon hooked over on the turn and Deuchar saw an attempted volley blocked as both sides sought a winner.
Kelly should have put Killie ahead when he blazed over after a superb run and cutback from Hamill.
The visitors immediately withdrew Sheerin for Liam Craig but the hosts were looking more of a threat and they completed a tremendous comeback when Kyle got his customary goal against Saints 13 minutes from time.
Taouil got free down the right and stood the ball up at the back post for the striker to thump home a trademark header.
Kyle, who had earlier hurt himself in a challenge with Duberry, was replaced by Steven Old, while Saints brought on fit-again Collin Samuel for Sheridan.
Samuel, making his first appearance for more than two months, almost combined with Deuchar to equalise but Severin rescued Killie with a goalline clearance.
A frustrated Duberry was booked for dissent late on and Bryson was also carded for deliberate handball.
Smith stopped Maguire making the game safe in the closing moments before Craig curled a stoppage-time free-kick wide.
Sammon had the ball in the net deep into added time but was flagged offside.