Newcastle had the better of proceedings over 90 minutes but they could not break the deadlock and instead left St James' Park digesting an unflattering defeat.
And while Alvaro Negredo and Edin Dzeko deserve praise for neat finishes in the 99th and 115th minutes, goalkeeper Costel Pantilimon also did everything that was asked of him.
City made 10 changes from the side that was beaten by Chelsea at the weekend, the least surprising of which involved the struggling Hart stepping down.
A first-half save from Shola Ameobi, when partially unsighted, was good but a firm block when in a one-on-one with Papiss Cisse was even better.
Newcastle showed five changes of their own and the teams contested a scrappy 90 minutes, with neither doing enough to seal the win in regulation.
Both had chances to do so, not least a pair of close-range headers in the space of a minute late on. Negredo saw his beaten clear by Tim Krul but an unmarked Yoan Gouffran had no such excuse as he diverted Mathieu Debuchy's cross wide from inside the six-yard box.
City lost Stevan Jovetic to injury after just 10 minutes, with Manuel Pellegrini summoning Negredo from the bench.
The visitors started lethargically, with Pantilimon making a good save from Ameobi on the half-hour.
Ameobi had the ball in the net a minute later but was ruled offside when he tapped home Cisse's shot across goal.
Navas created City's best chance in the 63rd minute when he came away with the ball following an ill-judged foray from Krul, but James Milner curled his effort wide.
With 12 minutes left in normal time, Negredo made his first major contribution, waltzing into the box and floating in a delicate close-range chip that Krul flicked around the post.
Both sides might still have won it in normal time, Negredo denied by Krul at close range and Gouffran nodding a glorious chance wide.
Pantilimon made sure the shutout continued in extra-time when he saved well from Cisse, and that paved the way for the belated opener.
Dzeko drilled a low cross into the Newcastle area and Negredo edged in front of Mapou Yanga-Mbiwa to apply the finishing touch.
The hard-working Dzeko got his own reward just before the break in extra-time when he got on the end of Milner's pass and successfully rounding the keeper.
Joleon Lescott cleared a Ben Arfa volley off the line in the closing stages but City held on for victory.