Marouane Fellaini struck the winner deep into extra-time as Everton finally saw off battling Stevenage 2-1 at Goodison Park.
The Belgian came off the bench to score in what could prove his final appearance for the Toffees after 115 minutes of an absorbing second-round tie.
League One Stevenage had taken the lead against the run of play afer 36 minutes through Luke Freeman but debutant Gerard Deulofeu levelled just before the break.
Brothers Shola and Sammy Ameobi teamed up with both the goals as Newcastle's won 2-0 at Morecambe to reach the third round.
Shola came off the bench to open the scoring with a deflected 84th-minute shot, and Sammy made sure in the fourth minute of stoppage time to book the Magpies' place in the next round, something which had looked far from certain for much of the previous 80 minutes or so.
It was a first win of the season for manager Alan Pardew, who fielded a much-changed side, but was far from convincing.
Aston Villa went some way to laying to rest the ghosts of last season's semi-final exit to Bradford by easing past League One Rotherham 3-0 at Villa Park.
Paul Lambert branded that two-legged defeat to the then League Two club his lowest point as Villa boss, and he was not prepared to risk it happening again as he fielded a strong team which included Christian Benteke, Gabby Agbonlahor and Andreas Weimann.
And they were far too good for Steve Evans' side, with Lambert's selection justified by half-time as Weimann opened the scoring with a fine 20-yard effort in the 19th minute and then Benteke headed home Agbonlahor's cross 20 minutes later to ease the nerves.
Fabian Delph put the game to bed shortly after the interval with a fine team goal, latching on to Benteke's scooped pass to finish with aplomb.
It could have been a greater margin of victory, but Villa were wasteful in the final 30 minutes, with Agbonlahor and Joe Bennett missing good opportunities.
A much-changed Cardiff side also progressed as two second-half goals in the space of a minute eliminated plucky Accrington 2-0 at the Crown Ground.
Strikers Nicky Maynard and Rudy Gestede, who had earlier come on for the injured Andreas Cornelius on the stroke of half-time, struck to ensure the 2012 finalist's passage.
Kenwyne Jones netted a hat-trick as Stoke saw off Sky Bet League One side Walsall 3-1 at the Britannia Stadium.
The Potters took the lead in the 22nd minute when the visitors squandered possession and Jones made the most of the opportunity, cutting inside and firing home.
It was his first goal since January, and the Trinidad and Tobago striker took only nine minutes to add another, rifling into the net from close range after the ball had dropped at his feet from a corner.
The visitors got themselves back in the game through Ashley Hemmings' 57th-minute strike but Jones then completed his treble in the 84th minute, racing on to a through-ball and finishing coolly.
Watford are through to the third round for the first time since 2008 after they beat Bournemouth 2-0 at Vicarage Road.
A memorable five-goal second-half display saw the Hornets inflict a 6-1 drubbing on the Cherries when the two sides met in the league earlier this month.
A repeat never looked likely as the visitors arguably gave a better account of themselves this time around, only to be undone by a bizarre own goal from Eliot Ward and Cristian Battocchio's second-half effort.
New-look Nottingham Forest edged into the third round of the Capital One Cup - although they needed extra time to see off Millwall 2-1.
Reds boss Billy Davies altered his entire starting XI from the one that had claimed a creditable 1-1 draw at Watford but eventually saw his reshuffled side provide a demonstration of their strength in depth, as an extra-time goal from academy product Jamaal Lascelles steered them to victory.
A 58th-minute header from Matt Derbyshire looked to have settled the tie but Liam Feeney's drive, possibly deflected by substitute Andy Keogh, levelled matters for the Lions with just four minutes left.
And it needed Lascelles' 94th-minute header, after Greg Halford had headed a Jamie Paterson corner back across goal, to win the game for Forest.