In a match where Rudi Skacel was supposed to be the central protagonist on his Tynecastle return, Stevenson put Hearts in front after 15 minutes and added a second from the penalty spot after 31 minutes.
Keith Watson pulled a goal back after 57 minutes but United could not find a second as Hearts climbed above their opponents with victory in their final home match of a tumultuous calendar year.
The most popular man on the pitch was, curiously, a visiting player as Skacel started for an unchanged United.
The Czech international's final appearance of his second spell at Hearts was the 5-1 Scottish Cup final defeat of Hibernian on May 19 and he selected squad number 51 at United, while he was the subject of an ovation from the home faithful in the 51st minute.
It was the second time Skacel had faced Hearts this term since moving to United. He was withdrawn from the Scottish Communities League Cup quarter-final penalty shootout by United boss Peter Houston.
From the first corner, Skacel tackled Stevenson and cleared, his opponent going to ground in search of a penalty, with appeals ignored.
Moments later Stevenson scored, benefiting from John Sutton occupying the attentions of two defenders and Brian McLean's failure to clear a ball from deep. The ball broke for Stevenson, who simply sidefooted in.
Jamie MacDonald made an instinctive save when Jon Daly met Barry Douglas' free-kick from the left and McLean's torrid afternoon continued as he could only hit the post with the follow up from two yards out.
Dylan McGowan won a penalty when he was bundled over by Willo Flood trying to meet brother Ryan's long throw. Stevenson took the resulting spot-kick and sent Cierzniak the wrong way to score.
McLean headed over from a Flood cross as United resumed on the attack and Hearts were repelling an aerial bombardment when there was the bizarre spectacle of a Tynecastle chorus of "Rudi" complete with the 'we-are-not-worthy' bowing gesture.
Skacel acknowledged with a clap of his hands during a break in play before having a role as United replied. A Douglas free-kick was only half-cleared, Skacel sliced a shot into the ground and Watson turned to fire into the top corner beyond MacDonald.
In the fourth minute of stoppage time, Russell had a chance to equalise when the ball evaded Marius Zaliukas, but he could only fire over as Hearts finished victorious.