Fraser Fyvie's opportunist goal gave the Pittodrie club a deserved 1-0 half-time in this fifth round replay as the sides met again following a 1-1 draw in the original tie at Pittodrie.
But Queen of the South's Ryan McGuffie equalised with a penalty and Aberdeen survived a few scares before Considine struck two minutes into injury time.
Craig Brown's side will now face Motherwell at Fir Park in an all Clydesdale Bank Premier League last-eight tie.
Doonhamers manager Gus MacPherson was without the cup-tied John Potter, replacing him with Craig Reid, while Aberdeen manager Brown welcomed back the influential Kari Arnason into central midfield.
The Pittodrie club's recent history is littered with cup defeats to lower division sides and they were desperate to avoid another damaging loss.
The visitors settled quickly and Fyvie almost caught out Lee Robinson with a looping free-kick, but the
goalkeeper did well to recover and tip over.
Aberdeen continued to apply pressure to the Queens defence and Fyvie was inches away from connecting with Mitch Megginson's cross in nine minutes.
The home side were under pressure and Arnason and Stephen Hughes both came close with long-range efforts.
A goal seemed inevitable and Aberdeen made the breakthrough in the 21st minute. Rory McArdle's cross was headed back across goal by Rory Fallon and Fyvie finished from six yards after Scott Vernon's initial effort was blocked.
Queens' first attack came midway through the half, McGuffie's cross having just too much pace for Nicky Clark.
But Aberdeen soon found their stride again and Robinson pulled off a superb reaction save from Vernon after great work from Megginson.
The home side had another brief flurry and Sam Parkin fluffed his shot after meeting Daniel Carmichael's cross eight yards out.
Impressive Aberdeen winger Mitch Megginson limped off five minutes before the break, giving Daniel Uchechi his debut.
McGuffie's long-range drive caused Aberdeen goalkeeper Jason Brown mild concern within seconds of the restart.
Uchechi then showed superb skill to break free on the left wing before finding Vernon, but his shot lacked pace and power.
Fyvie was unlucky with a first-time volley from Mark Reynolds' long pass as Aberdeen tried to kill the tie.
But the Doonhamers hauled themselves back into the match in the 59th minute when Ryan McGuffie scored from the penalty spot after Youl Mawene handled Nicky Clark's cross.
The Aberdeen players were furious at that award but referee Alan Muir was left with little option but to penalise Mawene, who was also booked.
The Irn-Bru League Division One side were now full of confidence and Clark crashed a 20-yard volley off the bar as Aberdeen lived dangerously.
The home side had their SPL opponents rattled and Parkin's curling shot skimmed the bar after sublime trickery from Carmichael created the opening.
Clark was a constant menace for Queen of the South and his driving run and cross caused chaos in the Aberdeen box with nine minutes remaining. Parkin and Scott McLaughlin were both waiting to pounce but Dons midfielder Stephen Hughes produced a last-ditch clearance.
With extra-time looming, Considine struck. Arnason picked out the defender at the back post with a perfect cross and Considine buried his header past Robinson from six yards to steer Brown's men into the last eight.