Hearts played without a recognised striker for the whole match, but Stewart drove home a 62nd-minute winner with their only shot on target.
United were handed a lifeline in the shape of a controversial late penalty, but Janos Balogh saved Francisco Sandaza's poor effort.
The away win was all the more rewarding for Hearts, given their selection problems - exacerbated on Friday with the news that Laryea Kingston faces two months out with a torn hamstring.
With Christian Nade suspended and Gary Glen injured, Hearts started with midfielder Adrian Mrowiec up front on his own - Bruno Aguiar supporting. The Pole was handed the unfamiliar role in only his second start for Hearts, and his first since suffering a knee injury in September.
The visitors also had Christos Karipidis and Eggert Jonsson in a makeshift central defence but they survived against a United side lacking confidence in the final third.
They were grateful for Balogh's double save in the 13th minute. Paul Caddis found Sandaza inside the box with a clever reverse pass. But Balogh blocked his effort and held Caddis' first-time follow-up at the second attempt.
Hearts survived another scare in the 20th minute, after being caught out by Danny Grainger's quickly-taken free-kick. The full-back's low cross found Sandaza at the near post, and he helped it on for Darren Dods - whose close-range shot was blocked by Robbie Neilson.
On-loan Leicester midfielder James Wesolowski soon shot well over from 20 yards on his United debut, but the home side failed to build on their pressure.
Caddis weakly shot a 25-yard free-kick into the wall as Hearts comfortably survived a scrappy phase of football until half-time, without providing any attacking threat of their own.
United strikers Warren Feeney and Sandaza even looked unconvincing during some half-time target practice - the Northern Ireland international then volleying high and wide after the ball broke for him 15 yards out after the restart.
Mrowiec failed with a hopeful penalty appeal as he tried to run between two defenders, but United's sloppy passing was allowing Hearts more possession in dangerous areas.
They took an unlikely lead, after Andrew Driver's persistence inside the box paid off. The ball fell into Stewart's path 20 yards out, and he shot powerfully into the bottom corner after his initial effort had been blocked.
United brought on winger Craig Conway for Wesolowski as they sought much-needed inspiration, and they began to push Hearts back.
Sandaza's shot trundled wide, after he had turned Jonsson just outside the box, and the Spaniard came closer when he fired inches wide of the far post.
Sandaza soon won a controversial 80th-minute penalty from Eddie Smith when he fell over Jonsson's leg as the Hearts man appeared to take the ball with a sliding tackle.
As Hearts unsuccessfully appealed for Smith to allow treatment to the Mrowiec, the United striker saw his weak penalty saved by Balogh.
The game threatened to spiral out of control as United played on, with Mrowiec still lying in the middle of his own half.
But Hearts saw out the remainder of the game - including five minutes of injury time - with Balogh making a comfortable save from Dods from 10 yards out.