Former Hearts midfielder Kenny Black, now manager at United, watched his 10-man side win on penalties to progress to the third round and inflict an embarrassing defeat on new Hearts boss Csaba Laszlo after a goalless draw.
Former Scotland Under-21 goalkeeper Stephen Robertson denied Lee Wallace and David Obua with saves in the shoot-out - the last from the Ugandan - with Darren Smith the only man to miss for Airdrie.
The Lanarkshire side put up a brave, defensive play to frustrate lacklustre Hearts throughout, but they went closest to winning the tie in normal time when Paul Di Giacomo struck the bar in the first half.
Hearts, for all their possession, lacked a cutting edge even after Airdrie lost Kevin McDonald to a red card in extra-time.
As Airdrie celebrated, boos echoed around the stadium at the finish as Hearts joined Edinburgh rivals Hibernian as unlikely early cup casualties.
Laszlo made fewer changes from Saturday's win over St Mirren than anticipated, with only Andrew Driver and Jason Thomson rested. Saulius Mikoliunas came in, with Eggert Jonsson also tried at right-back.
Airdrie, without a goal in four league games this term, understandably arrived with a defensive approach, content to hand Hearts control in a first meeting between the sides since United were re-formed in 2002.
Despite the hosts' often patient passing play, they made hard work of creating chances.
In fact, it was United who first threatened when Stephen McKenna headed over following Steven McDougall's corner.
Nineteen minutes had elapsed before Hearts carved out a first opportunity, yet Christos Karipidis was off balance as he diverted a weak shot into Robertson's arms.
Home fans howled for a penalty five minutes later after Michael Stewart went to ground under a challenge from Marc Smyth, but referee Eddie Smith was not interested.
Black's men, backed by a vociferous travelling support, grew more confident as they frustrated Hearts and almost stunned Tynecastle just after the half-hour mark.
The ball broke kindly for Di Giacomo 15 yards out and the former Kilmarnock striker was only denied by a superb blocking save from Marian Kello.
Di Giacomo went even closer on 42 minutes. His fierce 25-yard drive was seemingly destined for the top right corner before the ball hit the underside of the bar, bounced down on the line and went away to safety.
Hearts upped the tempo after the restart and Obua, Mikoliunas and Laryea Kingston all quickly registered efforts.
The Diamonds, though, remained a threat on the break and McDonald tested Kello after 56 minutes.
Seconds later, Hearts should have made the breakthrough, but Robertson superbly denied Jamie Mole with a clawing save.
Mikoliunas then somehow contrived to spoon wide at the far post when it looked easier to score.
Hearts continued to press and after 82 minutes Mole headed straight at Robertson from substitute Deividas Cesnauskis' cross.
Airdrie replacement Michael Brown saw an 18-yard effort deflected just wide, before Robert Donnelly was lucky to get away with a tug on Audrius Ksanavicius' jersey in the area at the end of the 90 minutes.
Instead, Smith booked the Lithuanian for diving.
Obua and substitute Christian Nade passed up chances in extra-time, before Airdrie's caused was hampered by the dismissal of McDonald for a second booking after 107 minutes.
As time drifted away, Robertson stood tall, keeping out efforts from Cesnauskis and Nade with fine stops before maintaining his form in the shoot-out.