The Rugby Park side had a decent record at Tynecastle last season and were confident of a victory despite losing their last three league outings.
Mehdi Taouil provided a deserved lead which was cancelled out by Laryea Kingston, before Bryson helped himself to the decisive goal.
Mike Tullberg earned his first start for Hearts - and his Tynecastle debut - as Jambos boss Csaba Laszlo made two changes from the side who lost to Dundee United. Tullberg and Jason Thomson were drafted into the starting line-up in place of the suspended Eggert Jonsson and Audrius Ksanavicius.
Killie made just one change after their reverse in Inverness. James Fowler's red card meant he was banned and was replaced in the starting XI by Danny Invincibile.
Both teams were up for this game right from the outset and Manuel Pascali - wearing the captain's armband - was fortunate to escape punishment after clattering into Christophe Berra in the opening minutes.
The Hearts skipper recovered enough to meet a Kingston corner at the near post but his header only managed to ripple the side-netting as the home side carved out the first real chance of the game.
Kingston was again the provider when he embarked upon a surging run down the right flank before delivering into the path of Tullberg, who connected awkwardly and chested wide of the target.
Instead, it was Kilmarnock who opened the scoring with 19 minutes gone. Jamie Mole picked up the first booking of the game for a foul on Bryson just outside the box and the challenge proved to be costly.
Taouil stepped up for the resultant free-kick just outside the box and delivered a lovely curling effort into the top corner of the net, leaving goalkeeper Marian Kello with no chance.
Killie pushed forward in numbers and looked likely to add to their tally - only to suffer a blow when Hearts hauled themselves level with 33 minutes gone.
Stewart burst into the box and had a go with a powerful effort that cannoned off the body of Bryson and fell kindly to the feet of Kingston, who gratefully lashed home the rebound from 10 yards.
The visitors tried to restore their lead before the break but Jamie Hamill's effort at the near post was weak and comfortably held by Kello.
Killie could have been back in front shortly after the restart when Hearts lost possession in the middle of the park, allowing David Fernandez to hook the ball out to Conor Sammon.
The striker burst into the box from the right but was denied the sight of the net bulging by a timely block from Berra at the post.
At the other end, Saulius Mikoliunas took advantage of David Lilley's failure to clear the danger and saw his low shot scrape wide before Kingston volleyed over as he eyed his second of the afternoon.
Sammon then had another go, with Fernandez the supplier again, but his header bounced harmlessly on the roof of the net with Kello covering anyway.
Kingston missed another great chance to hand Hearts the advantage when he collected a lovely pass from Mikoliunas and found himself one-on-one with Alan Combe, but the keeper pulled off a great point-blank save to keep the score level.
It was Bryson who snatched the winner with nine minutes to go, starting the move by picking out substitute Donovan Simmonds before making a well-timed run into the box and collecting the return pass.
Bryson then finished in style by sending a low, left-footed shot past Kello to claim maximum points.