Johnson, who turned down a January move to the Stadium of Light to join City instead, came off the bench to level in stoppage-time and finally end the home side's stubborn resistance.
It was hard luck on Black Cats goalkeeper Craig Gordon, who pulled off five fine second-half saves, three of them from Newcastle old boy Craig Bellamy, to protect the lead given to his side by Kenwyne Jones' ninth-minute header.
Sunderland were within seconds of clinching back-to-back Premier League victories for the first time since December 2008, a run of 48 games, and the home fans recognised their efforts despite the last-gasp heartbreak.
They were warmly applauded from the pitch by a crowd of 41,398 after very nearly adding City's scalp to those of Liverpool and Arsenal five days after ending their 14-game winless league run against Bolton.
The visitors, who have a game in hand on fourth-placed Tottenham, are now just two points adrift, but they came desperately close to leaving the north east empty-handed despite rallying well after a dismal first-half display.
Having assessed the enviable resources at the disposal of opposite number Roberto Mancini in the run-up to the game, Steve Bruce had nevertheless pointed to Real Madrid's midweek Champions League exit as evidence that money alone cannot guarantee success.
It was a message which appeared to have got through to his players as they set about the task of building upon Tuesday's 4-0 demolition of Bolton in confident style.
They got their noses in front within nine minutes when Jones rose majestically to power in a header from Steed Malbranque's pinpoint cross, and rarely looked like surrendering their lead as a City side packed with expensively-acquired attacking talent drastically under-achieved.
Their dominance was all the more impressive as Bruce, who has only just started to enjoy a little respite from the injury crisis which has blighted his defenders in recent months, found himself short of midfielders.
Lee Cattermole's lingering hamstring problems and Lorik Cana's suspension left him with Kieran Richardson, who has himself been struggling with a calf injury, and largely untried youngster David Meyler as a central pairing.
However, both men seemed to be galvanised by the challenge of meeting Nigel de Jong and Gareth Barry head on as the Black Cats took the game by the scruff of the neck.
Jones was a constant threat, as was the pace of leading scorer Darren Bent, while wide men Malbranque and Fraizer Campbell made life distinctly uncomfortable for Micah Richards and Wayne Bridge, who was replaced by striker Roque Santa Cruz with 33 minutes gone.
The Paraguayan's introduction added his goalscoring prowess to a side which already included Carlos Tevez, Bellamy and Shaun Wright-Phillips.
However, none of the four saw enough of the ball to do any real damage with Sunderland defending tigerishly, and it was Barry who came closest to restoring parity with a 31st-minute snapshot which whistled inches wide of the post.
Sunderland were forced into a reshuffle at the break when Jones failed to re-appear and was replaced by midfielder Jordan Henderson, who took over from Campbell wide on the right as he joined Bent in attack.
But whatever Mancini had said to his players at the break seemed to have done the trick as they returned in determined fashion.
Three times inside the opening seven minutes of the half, Gordon had to make vital saves, keeping out efforts from Santa Cruz, Wright-Phillips and Bellamy in quick succession after being left exposed by his defenders.
Barry and Meyler earned themselves a ticking-off from referee Chris Foy after an angry confrontation off the ball, and the temperature was rising all the time.
Mancini replaced his other starting full-back, Richards, with Patrick Vieira with 64 minutes gone, but it was Bellamy and Tevez who almost dragged his side back into it with 20 minutes remaining.
The Welshman drilled in a low cross from the right which the Argentinian prodded towards goal, only for Gordon to block with his legs from point-blank range.
Bellamy might have equalised himself twice inside the last 10 minutes, but lost out in one-on-one battles with the Black Caps goalkeeper on both occasions.
But there was nothing the Scot could do to keep out Johnson at the death, the former Middlesbrough winger curling a long-range effort high over his out-stretched arm into the top corner to snatch two points from Sunderland's grasp.