The Hoops looked to be heading for their first home league defeat in a year thanks to Stuart Armstrong's stunning first-half goal but their resistance was broken when keeper Radoslaw Cierzniak was finally beaten with just seconds remaining.
The outcome was tough on Jackie McNamara's side who had been looking to record their first victory at Celtic Park since a Duncan Ferguson goal earned them all three points on Boxing Day 1992.
United enjoyed early possession and were denied an opener when Fraser Forster thwarted Brian Graham with a point-blank save in the 14th minute.
The let-off sparked the home side into life and Amido Balde squandered a great chance when heading a Darnell Fisher cross straight into the arms of Cierzniak six minutes later.
And the Portuguese striker was against guilty of slackness when he put another header over after being picked out unmarked in the box from a Mulgrew corner.
Anthony Stokes was next to try his luck but he shot wide and with the home crowd becoming increasingly more nervous United took the lead after some fine work from Nadir Ciftci.
The Turkish striker, who is facing a lengthy suspension after being sent off against Inverness last week, picked out Armstrong with a delightful chip which the former Caley midfielder collected on his chest before chipping over Forster with the static Celtic defence looking for an non-existent offside flag.
However, they were almost level just 60 seconds later when James Forrest was presented with a golden chance but he shot wide of Cierzniak's goal.
After the break United continued to threaten the home goal and but for the miss of the season would have doubled their advantage early in the period.
With the goal at his mercy, Graham, who was picked out by another cross from Cifcti, managed to head the ball straight at Forster from three yards out with the goal at his mercy.
Celtic upped the pace after the lifeline and Stokes drew a great save from Cierzniak who turned his near-post effort over the bar from the Hoops' 10th corner of the game.
Substitute Dirk Boerrigter, and Stokes again squandered further opportunities before Mulgrew headed Emilio Izaguirre's cross beyond Cierzniak two minutes into injury time to stretch his side's unbeaten start to the season to 11 matches.