The Scotland international pounced on a cutback from strike partner Mark Burchill to make St Johnstone pay for some slipshod defending after only 15 minutes.
Saints enjoyed the upper hand throughout and carved out the better opportunities but failed to score for the first time in a home match this season as the visitors claimed a first victory in five matches.
Kilmarnock forced the game's first corner but it was St Johnstone who carved out the opening opportunity.
Danny Grainger's cross eluded intended target Kenny Deuchar, who appeared to be held down by Frazer Wright, but Filipe Morais wasted the chance when he blasted high and wide from the right side of the box.
The attack sparked a flurry of activity at either end as both sides probed for the breakthrough.
Saints captain Kevin Rutkiewicz slipped in trying to deal with a Mehdi Taouil through ball and was relieved when goalkeeper Graeme Smith's subsequent clearance cannoned wide off Mark Burchill's charge down.
A minute later, Kilmarnock goalkeeper Mark Brown was forced into action as he diverted a powerful shot from Morais round his near post.
There was no let up in the action and there was no surprise when the visitors took the lead in the 16th minute following some dreadful defending from the hosts.
Burchill, making his first start after an injury-ravaged opening to his Rugby Park career, nipped in to gather possession too easily inside the box and Kyle trundled in his cutback despite the best attempts of Rutkiewicz and Liam Craig to block on the line.
Saints midfielder Jody Morris tried to restore parity in the 20th minute but Brown palmed away his stinging drive from 20 yards out.
St Johnstone's shaky defence was all at sea again just after the half-hour mark when Garry Hay was allowed to canter down the left wing before spearing a low ball into the goalmouth.
Smith dived to block and would have been thankful to see his fumble land at the feet of a team-mate rather than the lurking Kilmarnock attackers.
Saints' encouraging start had given way to misplaced passing and hopeful punts up the park and their visitors were slowly gaining the upper hand.
But four minutes from the interval, the hosts almost grabbed the equaliser.
Morais curled a dangerous free-kick into a crowded penalty area and Brown had to look smart to gather Rutkiewicz's glancing header as it drifted towards the far corner of his net.
Deuchar then slipped as he looked poised to score when Grainger's cross fell at his feet 10 yards out to sum up Saints' half.
St Johnstone began the second period hunting relentlessly for the equaliser and they came within a whisker of getting it in the 50th minute.
Morais drifted in from his right-wing and turned superbly past James Fowler's challenge on the edge of the box before arrowing a fierce shot towards the bottom corner.
But Brown was equal to the effort with a magnificent one-handed stop to tip the ball round the post.
Kilmarnock were always likely to threaten on the break, however, and Grainger blocked Burchill's shot after Bryson's cutback.
At the other end, Kilmarnock almost inflicted damage of their own when Simon Ford came close to turning Grainger's low cross into his own net as the ball spun up and narrowly wide.
The McDiarmid Park side continued to enjoy the lion's share of possession as the game drew to a close and they should have grabbed the equaliser in the 74th minute.
Morais chipped a tempting cross to the edge of the six-yard box but Rutkiewicz could only nod down and wide from close range.
And, with the clock running down, the Perth men could not make their dominance pay as they failed to create anything meaningful in a tame climax to a poor 90 minutes.