Kyle converted substitute Arvydas Novikovas' deep cross as Hearts claimed a 10th win from 12 games to move three points behind Celtic and two adrift of Rangers at the top of the Scottish Premier League.
Hibs frustrated their hosts with men behind the ball and their best chance fell to Colin Nish after 76 minutes, the striker shooting narrowly wide as Colin Calderwood suffered a seventh loss in 11 games in charge.
Hibs lost 2-0 to Hearts at Easter Road in November in the former Scotland defender's third match in charge, and fortunes had barely improved since.
Calderwood's men did not build on successive wins over Rangers and Motherwell and had picked up two points since - in a 0-0 draw with St Johnstone and last Wednesday's 2-2 draw with Dundee United.
Calderwood made five changes for the second straight game, with Sol Bamba, who recently returned from a hamstring problem, not even among the substitutes amid speculation linking him with a move to Leicester.
But while Hibs were 2-0 down after 35 minutes in midweek before rescuing a draw, they appeared far more sturdy at Tynecastle.
Hearts, who defeated St Mirren 2-0 last Wednesday, pressed forward but Hibs kept men behind the ball and continually snuffed out the hosts' attempts to attack.
After Rudi Skacel, returning despite a slight calf tear, fluffed two set-piece opportunities, Derek Riordan shot straight at Hearts goalkeeper Jamie MacDonald with a free-kick from the edge of the area.
Ian Murray then led with his arm and clattered Ian Black in an aerial challenge after eight minutes. While Black was undergoing treatment, the Hibs captain was booked and may have been fortunate to stay on the field.
Black was able to continue and Hearts then missed a chance to go in front after 13 minutes.
David Templeton, Skacel and Ruben Palazuelos combined on the left and neither Kyle or Calum Elliot could apply the final touch to Skacel's cross.
Liam Miller was booked for tripping Black before Darryl Duffy, starting for Hibs for the first time, fired a testing volley which MacDonald had to turn around for a corner at the other end.
Valdas Trakys became the third Hibs player booked and Adrian Mrowiec the first for Hearts before Skacel shot wide and then shot straight at Mark Brown.
Francis Dickoh shot wide from a corner in first-half stoppage time and Brown stopped another Skacel effort as the half ended level.
Elliot was denied by Brown from Kyle's knockdown of Black's free-kick as Hearts resumed on the attack.
Zaliukas shot wide with an ambitious long-range effort before Steven Thicot cleared the Hearts skipper's header off the line following Black's corner.
Kyle headed weakly towards goal from Mrowiec's cross before Templeton, who had switched to the right, was tripped on the edge of the box by Miller.
Skacel's curling effort towards the far post hit the top of the bar as Hibs survived once again.
Nish replaced Trakys after 64 minutes and headed on Riordan's free-kick after 69 minutes, but Dickoh contrived to fire wide.
Zaliukas shot straight at Brown from long range before a similar effort from Mrowiec which barely troubled the Hibs goalkeeper.
David Wotherspoon replaced Duffy with 15 minutes remaining before Nish shot narrowly wide when played through by Riordan.
Gary Glen replaced Elliot and Templeton was brought off in place of Novikovas with 11 minutes remaining.
Black's shot hit Grounds, with muted appeals for handball, before Skacel's cross was cleared by Murray.
Glen then missed a straightforward opportunity when failing to hit the target from a Novikovas cross five minutes from time, but his blushes were spare when Kyle netted the winner moments later.
Novikovas was the provider as Kyle headed beyond Brown from the substitute's deep cross for his 10th of the season.
A Riordan ball hit Ismael Bouzid in the area but Hibs appeals were waved away as the visitors returned across the capital frustrated and beaten.