Palace took the lead from the penalty spot on 25 minutes as Dwight Gayle confidently picked out the bottom corner after Wilfried Zaha was tripped by Darryl Janmaat inside the area.
But Newcastle soon hit back as Emmanuel Riviere equalised with his first goal for the club since joining from Monaco, holding off Brede Hangeland before curling a fine finish beyond Wayne Hennessey, who was left rooted to the spot.
Riviere was able to double his Newcastle tally after the break, slotting a cool penalty past Hennessey following Paddy McCarthy’s foul on the advancing Sammy Ameobi.
Alan Pardew’s luck appeared to be turning as Newcastle moved closer to a rare away win, but 19-year-old winger Sullay Kaikai stepped off the bench to scramble home a stoppage-time equaliser and send the game to extra time.
The size of the task facing Newcastle increased in the first half of extra time as Mehdi Abeid was shown a second yellow card for hauling down Zaha, while Gayle had an acrobatic effort ruled out for offside.
But Dummett stooped to meet Adam Armstrong’s inviting cross inside the area and head a dramatic 112th-minute winner to book a fourth-round meeting with Manchester City.
If Pardew can transfer this manner of performance and victory in the league, he is unlikely to face further fan protests.
Palace manager Neil Warnock meanwhile will probably remain relatively unfazed in defeat, considering the former Sheffield United boss’ continued drive for stability at Selhurst Park.
Newcastle started the brighter despite Pardew’s continued Tyneside troubles, but the visitors offered no finishing power in the opening half-hour.
Gayle dragged a snapshot wide before Janmaat failed to find anyone with a right-wing cross after neat work from Gabriel Obertan.
Riviere then wasted two threatening crosses by failing to connect with any kind of power.
Janmaat’s tame side-foot left Hennessey with little to do to mop up, and Newcastle immediately paid the price for profligacy.
Zaha’s left-wing break was seemingly meandering to a close when Janmaat hacked his shins and conceded a needless penalty.
Gayle stepped up and despatched the spot-kick low and to Elliot’s right to hand Palace the lead against the general run of play.
Full debutant Armstrong failed to reach a right-flank cross, then Riviere fluffed Ameobi’s dangerous ball.
For all that flailing though, Newcastle then levelled the match courtesy of a perfectly-placed strike from Riviere.
So wasteful beforehand, the Frenchman found pristine poise to spin off Hangeland and clip beyond a motionless Hennessey from 20 yards.
Palace captain Paddy McCarthy gifted Newcastle the lead after the break, the fit-again defender scything down Ameobi in the box to hand the visitors a soft penalty.
Riviere stepped up and side-footed home just as calmly as he had converted his opener, again making a mockery of his early travails in front of goal.
Palace rallied in frustration, with Gayle heading against the base of the post.
And just when the hosts looked like slipping to defeat up popped late replacement Kaikai to send the game into extra time.
Mehdi Abeid was then sent off after receiving a second yellow card as Palace pressed for a winner when the match resumed for its third instalment.
Gayle lashed the ball into the net in the second period of extra time, only for his fine finish to be chalked off for offside.
As both sets of players started preparing for penalties, Armstrong clipped the ball to the far post – and Dummett caught the Palace defence unawares to nod home and steal the tie.