Villa were looking to bounce back from their Capital One Cup disappointment against Bradford in midweek and looked set to do so in style when Darren Bent gave them the lead midway through the first half.
But Danny Shittu headed home from a corner just five minutes later and John Marquis netted an 89th-minute winner to send the Championship side into the last 16.
Villa started like a team under pressure and looked nervy in possession but Paul Lambert's side seemed to shake off their early worries and went ahead in the 22nd minute through Bent.
Fabian Delph robbed Liam Feeney of possession and Andreas Weimann capitalised, running on to the ball before forcing David Forde into action for the first time.
The Lions' goalkeeper could only palm Weimann's effort into the path of Bent, whose miskicked effort crept over the line to give the beleaguered top-flight side the lead.
But just five minutes after falling behind the hosts were level, as James Henry's corner found Shittu at the back post and the Millwall skipper rose to power a header past Shay Given.
The equaliser led to the nerves creeping back into Villa's game and meant the Championship promotion hopefuls went in level at the break.
A couple of set-pieces early in the second half again tested the visitors' resolve, and Feeney came close to putting Millwall ahead when he fired just wide following a corner before the hosts' momentum was brought to a halt by their own supporters.
The game was stopped for five minutes just before the hour-mark due to a disturbance amongst Millwall fans in one corner of the Dockers Stand, with several bottles thrown in the direction of the assistant referee Mark Scholes after he awarded a free-kick to Villa.
Once play had resumed Villa substitute Barry Bannan arrowed in a shot that Forde again decided to push away but this time the ball bounced to safety.
The tie looked set for a replay at Villa Park until Millwall striker Marquis headed Adam Smith's cross on to the bar before reacting quickest to nod home the rebound.
The earlier incident from the Millwall fans meant that Kenny Jackett's side had to negotiate six minutes of stoppage time before becoming the first name in the hat for round five, but they did so with relative ease to leave Lambert suffering one of his worst weeks in management.