Walters had an afternoon to forget at the Britannia Stadium on Saturday when he scored two own goals and missed a penalty in Chelsea's 4-0 romp.
But after Glenn Murray's 87th-minute penalty forced the extra period Walters made up for his weekend woe with goals on 95 and 110 minutes.
Cameron Jerome made it 4-1 in the final minute to complete a win that sends Stoke into a fourth-round tie at home to Manchester City later this month - a repeat of the 2011 final.
Palace had their chance to force an upset with Jermaine Easter missing a first-half penalty, won by sought-after midfielder Wilfried Zaha, before goalkeeper Lewis Price made a hash of Kenwyne Jones' header on 69 minutes as Stoke took the lead.
There were two Premier League casualties on Tuesday as West Brom and Sunderland joined Newcastle and Southampton as the top-flight clubs to have crashed out at the first hurdle.
The Baggies lost their all-Premier League tie with QPR 1-0 after Jay Bothroyd scored his first goal of the season for Harry Redknapp's side on 75 minutes.
Sunderland fell 2-0 at home to Bolton with Marvin Sordell netting twice in the second half.
Fulham narrowly avoided following them out as they needed extra-time to beat Blackpool 2-1.
After substitute Kieran Richardson scored a stoppage-time equaliser for Fulham Brede Hangeland snatched the winner on 116 minutes. Nathan Delfouneso had put managerless Blackpool ahead on 82 minutes.
FA Cup holders Chelsea will make a short fourth-round trip to Brentford after the League One high-fliers beat Southend 2-1.
Clayton Donaldson scored the decisive goal 14 minutes from the end to set up a home clash with the European champions.
Barry Corr had levelled for Southend after the Bees, third in League One, led through Paul Hayes (26) and saw their former goalkeeper Paul Smith save a Harry Forrester penalty.
Wigan avoided an upset at Bournemouth as Mauro Boselli's 18th-minute goal was enough to ensure they will go to Blue Square Premier club Macclesfield next.
The MK Dons reached the fourth round for the first time since they relocated from Wimbledon with a 2-0 win at home to Sheffield Wednesday.
Shaun Williams' first-half penalty and a fortuitous second goal from a Dean Bowditch free-kick set up an away trip to QPR.
Leeds will welcome Premier League opposition to Elland Road for the fourth time in cup competitions this season after they came from behind to beat Birmingham 2-1.
El-Hadji Diouf netted a 76th-minute winner from the penalty spot after Ross McCormack (70) cancelled out Wade Elliot's 36th-minute opener.
Leeds will now meet Tottenham after having already hosted top-flight Everton, Southampton and Chelsea in the Capital One Cup this term.
Yorkshire rivals Hull also progressed 2-1 after extra-time against League One Leyton Orient.
Tom Cairney ensured City avoided penalties with just three minutes to spare. Orient forward Dean Cox (87) sent the game into extra-time after Nick Proschwitz (41) smashed in the opener.