Hearts manager Csaba Laszlo had warned against a repeat of last season's home defeat by Dunfermline's Irn-Bru First Division rivals Airdrie, but they trailed to Graham Bayne's goal at half-time.
However, Suso Santana headed an equaliser despite the visitors' appeals over an earlier infringement, and Hearts moved in front in the 74th minute after Brian Winter adjudged Neil McGregor to have fouled David Obua.
McGregor claimed his challenge was a fair shoulder charge but Stewart sent Hearts through.
Hearts had struggled to penetrate the Pars defence after losing Andrew Driver to injury before the game, with Gary Glen coming in to support defender Ismael Bouzid up front.
Dunfermline have lost their last three league matches without scoring a goal but they started confidently and Hearts centre-back Jose Goncalves got away with a strong penalty-box barge on Nicky Phinn.
Greg Ross ran on to Scott Muirhead's long throw but he failed to get enough purchase on his volley and Janos Balogh saved.
But Hearts were behind in the 15th minute when Bayne ran on to Ross' reverse pass and fired a shot that Balogh let slip through his body.
Hearts continued to struggle with Glen seeing little of the ball and Bouzid unable to hold it up.
Dunfermline manager Jim McIntyre's 4-2-3-1 formation was getting the better of Laszlo's tactics, while central midfielders Graeme Holmes and Alex Burke were impressing with and without the ball.
However Hearts did create some first-half chances from set-pieces as Dunfermline sat deeper.
Zaliukas headed home Craig Thomson's 35th-minute free-kick but the offside flag was raised and the Lithuanian volleyed well over from the same source a minute later.
Burke cleared Obua's header off the line following right-back Thomson's corner and Greg Fleming blocked the Uganda midfielder's follow-up.
Obua then turned provider with a long throw that fell to Glen, but his acrobatic volley just cleared the crossbar.
Hearts lost Lee Wallace to injury at half-time, with Ian Black coming on as the support striker and Bouzid going back into defence.
Dunfermline centre-back Andy Dowie was fortunate to avoid punishment for a penalty-box block on Santana that looked at least an obstruction.
But the Spaniard netted in the 57th minute when he deftly headed home Obua's delivery after the midfielder had retrieved an overhit cross.
McIntyre complained bitterly on the touchline, apparently over Obua's challenge on Callum Woods when the original cross came in.
Dunfermline re-emerged as an attacking force and Balogh parried a powerful effort from substitute Joe Cardle at his near post.
Thomson was not far away with a 45-yard chip after seeing Fleming off his line but Hearts soon took the lead.
McGregor gave the ball away to Obua but appeared to have recovered possession with a shoulder charge on the Hearts midfielder.
However, Winter pointed to the spot and Stewart sent Fleming the wrong way.
Dunfermline pushed forward in the closing stages but Hearts held on.