The Dens outfit looked set for their 20th defeat of the Clydesdale Bank Premier League campaign after losing Declan Gallacher following his last-man tug on the scorer of Saint's opening goal, Steven MacLean.
But former Falkirk man Stewart popped up with a last-minute leveller to keep the basement club 15 points adrift of safety.
MacLean had headed the Perth side ahead 35 minutes in after home goalkeeper Rab Douglas sullied his 300th appearance for Dundee with a handling gaffe, but John Baird cancelled that out with his own header just before the break.
Saints struck again through skipper Dave Mackay's free-kick on 68 minutes as he made sure it was a double punishment for the hosts following Gallacher's dismissal. However, just as the visitors thought they had done enough for the Tayside derby win, Stewart struck at the death.
Nicky Reilly and Baird both passed up reasonable chances for the hosts early on but, against the run of play, it was the Perth side that struck first with 10 minutes of the first half remaining.
Paddy Cregg struck a 25-yard drive low at Douglas that the Dens keeper should have held. Instead, it spun up off him and MacLean was hungrier than any of Dundee's defenders as he leapt to nod the rebound over the line.
Brown's new charges levelled after as Conroy's dangerous left foot provided another opening. From a free-kick on the right, he picked out Lewis Toshney and the defender's backward flick fell perfectly for Baird to head home unmarked at the back post.
Conroy's set-piece deliveries continued to cause consternation in the Saints' defence when his corner on 62 minutes broke for Toshney again. But the defender could not beat Mannus as the keeper blocked bravely, just as he did once more to deny the supplier of that chance when Conroy himself was put through by Harkins seconds later.
Within four minutes, though, it would be Douglas who was beaten again as the home side saw their numbers reduced by one.
MacLean looked set to latch on to Tam Scobbie's through ball on the edge of the Dark Blues' box until he was pulled back by Gallacher. Referee Willie Collum produced the straight red and, as if that was not bad enough for the home team, Mackay swept home a superb free-kick into the top corner.
Bad could have marched straight on to worse when Chris Millar had a volley at goal, but Douglas came up with a solid stop. He did the same when Saints substitute Gregory Tade swivelled and hit another fierce drive straight at the former Scotland international with 20 minutes left.
Brown did sample a portion of the resentment that has surrounded his appointment when he was treated to a round of boos as he replaced playmaker Harkins with striker Colin Nish for the remaining portion of the game.
But the jeers were replaced by cheers as Dundee levelled at the death when Stewart appeared to be offside as he collected a ball into the St Johnstone box, but the last touch had come off a visiting player and Collum allowed the midfielder to set himself before firing past Mannus to hand Brown a late reprieve.