Dougie Imrie converted a 97th-minute penalty after Efe Ambrose was adjudged to have handled inside the area, and the Ton withstood a late barrage to register their first victory at Celtic Park since 1971.
Despite making five changes, Neil Lennon's side controlled proceedings for large spells but they squandered a plethora of chances and were punished by Imrie in extra-time.
The Hoops quickly assumed the ascendancy in this third round tie, with Teemu Pukki having a shot gathered at the second attempt by Nicolas Caraux, who was again alert to save Charlie Mulgrew's 25-yard free-kick.
Nir Biton, who was making his first start for Celtic, went close on the half-hour when his volley whizzed inches over the crossbar, but Morton's defensive strategy was frustrating the home side.
The onslaught continued after the interval, with Tomas Rogic testing Caraux, but he was replaced on the hour mark as Lennon summoned top-scorer Anthony Stokes from the bench.
Mulgrew twice went close during the dying stages of normal time, but Morton rode their luck before then taking a shock lead in extra-time.
Referee Bobby Madden penalised Ambrose for handball inside the area and Imrie blasted the resultant spot kick down the middle of Lukasz Zaluska's goal.
Celtic piled bodies forward and should have equalised in the final minute of the first period. Kris Commons found Virgil Van Dijk at the back post but the defender failed to hit the target.
It was one-way traffic during the second period, with Van Dijk having a shot cleared off the line and Mikael Lustig seeing an effort go inches wide.
Lustig went even closer in injury time when his shot came back off the post, but Morton held on for one of the biggest results in their history.