Chelsea, who lost Salomon Kalou and Yossi Benayoun through injury, had taken a sixth-minute lead through young left-back Patrick Van Aanholt only for United to storm back and lead at the interval thanks to goals from Nile Ranger and Ryan Taylor.
Shola Ameobi made it 3-1 to United four minutes after the re-start but two goals from Nicolas Anelka (69, 86) looked to have taken the tie into extra-time.
However, Ameobi headed the winner from a Jonas Gutierrez corner in the 90th minute to earn United a deserved place in round four.
Chelsea made a dream start when left-back Van Aanholt crowned his full debut with a goal.
The left-back drilled an angled shot into the corner after being set up superbly by Gael Kakuta.
But Newcastle were level against the run of play in the 26th minute.
Shane Ferguson's left-wing cross caused problems in the Chelsea defence and when John Terry failed to clear, Ranger stole in on the blind side of Van Aanholt to level.
Ranger then squandered a great chance to put Newcastle 2-1 in front on the half-hour.
Great work by Gutierrez on the left saw him deliver a low cross into the Chelsea six-yard box but the ball spun off the foot of Ranger to the relief of home fans.
Chelsea centre-half Jeffrey Bruma was lucky to escape with just a yellow card for a foul on Ameobi in the 32nd minute as Newcastle continued to dominate.
Taylor punished Bruma and Chelsea for the foul when the Newcastle midfielder sent the ball around the wall and into the top corner.
Ameobi then caused more problems for Chelsea in the 35th minute but his angled shot was cleared off the line by Van Aanholt.
Taylor was lucky to remain on the pitch in the 40th minute when he brought down Van Aanholt but referee Phil Dowd opted to lecture him instead of producing a second yellow.
Chelsea made two changes at half-time with striker Kalou and defender Alex replacing Gael Kakuta and Terry.
But the changes had little time to take effect as United went further in front in the 49th minute.
Paulo Ferreira's pass went straight to the feet of Ameobi who ran on unchecked to fire a 25-yard drive beyond Ross Turnbull.
It was the first time Chelsea had conceded three goals at home since Manchester City's 4-2 demolition of the Blues in February.
The night got worse for Chelsea in 53rd minute when Kalou went down with what appeared to be a hamstring injury.
The Ivory Coast striker was taken off on a stretcher and replaced by teenage midfielder Josh McEachran.
The youngster was quickly into the game and Tim Krul had to be alert to save his 20-yard shot in the 59th minute.
The misery continued for the Blues in the 62nd minute when Benayoun turned his ankle and was forced to leave the field.
It left the home side down to 10-men but still they battled on with Alex sending a 20-yard effort against the outside of the right-hand upright.
Chelsea reduced the deficit in the 69th minute when Van Aanholt's cross was turned home by Anelka.
In the 82nd minute, a cross from Lovenkrands was met by Ranger at the far post but Turnbull saved at the second attempt.
It allowed Chelsea to sensationally equalise in the 86th minute when substitute Cheik Tiote was harshly adjudged to have fouled Alex and Anelka scored from the spot-kick.
Ferreira smashed an 88th minute volley against the upright as Chelsea looked for a winner.
But it was Newcastle who earned a place in the fourth round when Ameobi rose above Bruma to head the winner from a Gutierrez corner.