Kyle netted on the hour mark after David Templeton had been brought down by Stephen Craigan to pile more misery on Motherwell following last week's departure of manager Craig Brown to Aberdeen.
Keith Lasley had levelled for Motherwell in the final stages of the first half, immediately after Mark Reynolds had put the ball into his own net following Templeton's overhead kick.
But Motherwell's comeback hopes were extinguished in the 83rd minute when Steve Jennings was sent off by Stevie O'Reilly for making one of several complaints about the referee's performance.
Hearts move three points behind Celtic in third place and nine ahead of Motherwell, who were being led by youth development chief Gordon Young.
Hearts made two changes after their 5-0 win over Aberdeen, with Ryan Stevenson and Darren Barr in for Ian Black and Eggert Jonsson, and started well as Motherwell looked to be paying for their weather-enforced blank spell of three and a half weeks.
Stevenson and Rudi Skacel fired just wide from long range for Hearts before visiting striker Stephen Elliott missed a great chance in the 29th minute when he sliced a volley over from inside the six-yard following excellent wing play from Templeton.
Motherwell had a decent chance six minutes later when the ball fell for Jamie Murphy after Ruben Palazuelos had cut out Alan Gow's attempted through ball, but the 21-year-old pulled his shot beyond the far post.
Murphy then fired another shot across the face of goal after a determined run with Nick Blackman appearing to hold back from following in.
Kyle volleyed just beyond the far post and Hearts went ahead in the 44th minute.
Kyle robbed Stevie Hammell as the left-back waited for the ball to fall near the byline and Elliot set up Templeton's acrobatic effort, which was stopped brilliantly by Darren Randolph.
However, the Motherwell goalkeeper and Reynolds got in each other's way as they tried to clear and the ball crossed the line.
Motherwell levelled within 60 seconds when Lasley stabbed the ball home from 12 yards after running onto Alan Gow's through ball.
Tom Hateley stopped Elliott converting Kyle's low cross with a brilliant sliding challenge moments after the restart and the Republic of Ireland striker squandered another glorious opportunity.
Elliott turned and blasted against the bar from 12 yards following Barr's penetrating run and then headed the rebound over.
Hearts did go back in front after Stephen Craigan was adjudged to have brought down Templeton just inside the box, although the Motherwell skipper protested. Kyle sent Randolph the wrong way from the spot.
O'Reilly further incurred the wrath of the home team when he turned his back on play as Motherwell took a free-kick, missing Skacel running across the ball and diverting its flight. Bizarrely, none of his fellow officials alerted him and Blackman was booked for protesting.
Templeton again went down inside the box under the challenge of Jennings but O'Reilly waved play on despite the penalty shout appearing to be stronger than the one given.
Motherwell were struggling to break their visitors down but Hearts needed a great block by Marius Zaliukas after Murphy had worked space inside the box.
The home side's hopes all but disappeared when Jennings, booked in the early stages for a foul on Kyle, was sent off after complaining about O'Reilly's decision to turn down a penalty request when Zaliukas slid in on Gow following the striker's one-two with substitute John Sutton.