The conspiracy theorists had this one marked down as a draw even before kick-off, but the visitors came here to win and will feel unlucky not to have collected at least a point.
The result leaves Inverness very much still in the relegation dogfight.
It was Kyle's eighth goal of the season and it delighted the home supporters in the 6,096 crowd.
Neither side pulled any major surprises in their starting line-ups.
Killie boss Jim Jefferies fielded nine of the 11 that defeated St Mirren during the week, with Frazer Wright and James Fowler the only new faces in place of Craig Bryson and Simon Ford.
Inverness ran with the exact same side that picked up a point at Fir Park three days earlier.
The visitors did not threaten much but made much of the early running as Dougie Imrie purposefully ran at the Killie defence.
Inverness midfielder Brian Kerr tried his luck from 25 yards after eight minutes but his shot was well wide of the target.
After 11 minutes the Highlanders came close when Richie Foran lunged into the six-yard box but failed to connect with a dangerous ball across the face of the goal by Roy McBain. It was the closest the game had come to the opening goal.
Kerr came close after 20 minutes when his right-footed shot was deflected just wide of Alan Combe in the home side's goal.
Fifteen minutes before the break Mehdi Taouil won a free-kick in a dangerous position but again Inverness won the ball back and broke. Filipe Morais caught the home defence napping but fired just wide.
Kyle had the chance of a trademark header 12 minutes before the break, but he could not connect with the floated ball into the box.
A 22-man stand-off then followed a foul by David Proctor on Garry Hay which saw Richie Foran and Willie Gibson join the initial offender and the protesting victim into referee William Collum's book on 40 minutes.
With less than a minute left of the first half, Kyle nearly put the home side ahead with a header from eight yards but his effort was cleared off the line.
Inverness left-back Lionel Djebi-Zadi came close, firing into the side-netting after 51 minutes.
Five minutes later Morais looked to have sneaked through the home back four, but his shot was easily saved by Combe.
Next up for Terry Butcher's team was Djebi-Zadi, who tried his luck from distance, but again Combe held the effort.
A foul by Ian Black on the edge of the area allowed Killie a rare chance but Wright blasted the effort two yards wide.
As Inverness tired and chances dried up the home side searched for the goal with Kyle the main target, but Ross Tokely and Grant Munro marshalled the Killie striker well for the first 75 minutes.
With less than 15 minutes remaining Killie again threatened from a set-piece situation, but Kyle and Lilley managed to get in each other's way and the chance was gone.
However, the hosts then got the goal they barely deserved.
With 10 minutes remaining Taouil beat his man and supplied Hay, who crossed for Kyle to knock the ball home at the back post.
Taouil nearly made in 2-0 in the last minute but his effort hit the post.