Richard Hughes struck the decisive blow with Pompey's second sudden-death spot-kick after Julian Speroni had saved from Nathaniel Clyne and Adam Barrett.
Substitute Nadir Ciftci and Carl Dickinson previously had the opportunity to send Steve Cotterill's side through but both missed their penalties.
David Nugent put Portsmouth ahead just before the hour-mark but Ibrahima Sonko's own goal with nine minutes remaining sent the tie into extra-time.
Palace were reduced to 10 men five minutes after the equaliser when Wilfried Zaha was shown a straight red card for a late challenge on Dickinson.
Former Holland international Edgar Davids, who signed for Palace on a pay-as-you-play deal last week, made a surprise debut considering his lack of competitive football in the past two years.
He was involved in most of what George Burley's side did. Owen Garvan and Pablo Counago, also making his Palace debut, sent shots over the crossbar before the half-hour mark.
Kieran Djilali and Andy Dorman also threatened before the break as Palace monopolised the first-half chances.
They forced Jamie Ashdown into action for the first time two minutes after the break when he had to tip Zaha's shot over.
But Nugent struck against the run of play after 57 minutes with Pompey's first real effort on goal. Richard Hughes provided the assist from the left and Nugent struck for the first time this season from 12 yards out.
It sprung Portsmouth into life. Tommy Smith and Dickinson tried their luck from long range before Smith turned the ball over the crossbar from inside the six-yard box.
Ciftci then went close with a shot from the edge of the area.
Palace were level when Sonko turned Kieron Cadogan's cross into his own net but their joy was shortlived as Zaha was sent off.
Speroni saved from West Brom and Stoke target Marc Wilson in extra time before Hughes won it from the spot.