Burns drove home his first SPL goal from 20 yards as the Pars leapfrogged Rangers in the table.
The home side had enjoyed the better of the contest but were denied by a superb first-half save from Paul Gallacher to push Scott Robertson's volley on to the post.
And United generally struggled to get in behind a stubborn Pars defence to leave them with just two points from three home games.
Dunfermline's defensive display merited them maintaining their unbeaten record, although they had failed to trouble the United goal before the 85th-minute winner.
United handed Gavin Gunning his first start, the former Blackburn player lining up in central defence to allow Sean Dillon to return to right-back.
Dunfermline made an attacking change following their 1-0 win over St Johnstone with defender John Potter dropped to the bench following his red card, which was rescinded on appeal. Winger David Graham took his place.
Johnny Russell looked lively in the opening stages and Willo Flood had a powerful long-range strike held by Gallacher, but the visitors went on to enjoy the better of the first quarter.
Joe Cardle, Graham and Martin Hardie were providing good support to striker Andy Kirk and the Pars were winning the midfield contest, although Barry Douglas had a curling effort saved by Gallacher midway through the half.
The visitors created their first clear chance when Jason Thomson's cross found Hardie but he headed over from 10 yards.
Douglas, who was in for Scott Allan, and Jon Daly headed just wide following crosses from either flank as United improved.
Gallacher blocked at the near post from Daly after John Rankin had forced his way to the byline, and soon pulled off an outstanding save from the resulting corner in the 31st minute.
The former United goalkeeper punched clear Flood's delivery, which fell to Robertson 20 yards out. The midfielder arrowed a volley towards the bottom corner and Gallacher threw himself down to his right to push the shot on to the post despite appearing unsighted until the last moment.
United came close again five minutes before half-time when Rankin ran on to meet Dillon's cross but headed a yard wide.
Russell found some space in the box just after the restart but sliced wide from a tight angle and United kept up the pressure with a number of balls into the box, one of which fell to Rankin to turn and shoot wide from 12 yards in a crowded area.
However, the Pars were coping comfortably with the pressure, with Thomson and Austin McCann producing two particularly impressive blocks, and United manager Peter Houston brought on Danny Swanson for Douglas in the 64th minute in a bid to find a creative spark.
None was forthcoming though, and Hardie was wide with a 35-yard free-kick and a back-post header as Dunfermline sensed they could perhaps sneak three points.
United's desperation was evident when Daly went down theatrically after the slightest of contact from Andy Dowie just inside the box, and the home fans vented their frustration when Russell was taken off for Ryan Dow.
And the United fans were soon streaming out after summer-signing Burns netted.
Cardle sent the ball in from the left, behind everyone in the box, but it fell invitingly for Burns to run on to and fire low into the net from 20 yards, although Dusan Pernis got a hand to the shot and should have done better.
Gunning headed wide from Flood's corner as United failed to claw back the deficit.