Romain Vincelot's header three minutes from time sees Orient join Bradford in humiliating Villa in this competition, with the Bantams famously defeating the Premier League team over a two-legged semi-final in 2013.
Adam McGurk struck a superb late free-kick as League Two Burton claimed a shock 1-0 win over Premier League side QPR.
The Brewers striker, who recovered from a hip injury to feature, rewarded the hosts for a bright performance as he curled home a fine set-piece with just 13 minutes remaining at the Pirelli Stadium.
QPR's Junior Hoilett hit the bar before the break but the Londoners did not look like winning for a first time this season as the game wore on.
James Hanson's late winner heaped further pressure on Leeds head coach Dave Hockaday as Bradford claimed a 2-1 victory in a fiery West Yorkshire derby to progress to the third round.
Beaten finalists two seasons ago, the Bantams were again keen to impress in front of the television cameras as they recorded their first win over United for almost 28 years.
Hockaday, who was given a stay of execution by club president Massimo Cellino following Saturday's 4-1 defeat at Watford, looked a relieved man when Matt Smith headed them in front eight minutes from time.
But a stunning equaliser from Billy Knott soon lifted the home crowd and again, and they were simply delirious when Hanson prodded over the line with four minutes remaining.
Sunderland's late show sent the Black Cats through - and left Birmingham fuming.
Jordi Gomez, Adam Johnson and Connor Wickham struck in the final 13 minutes to stun the hosts and give the Premier League side a 3-0 win.
But Blues were angered after Gomez's opener, claiming the ball had gone out in the build-up.
Forward Jonathan Walters scored twice and made a late third for Peter Crouch to ease Stoke into the third round as on-loan Chelsea winger Victor Moses enjoyed a promising debut against Portsmouth at the Britannia Stadium.
While the prospect of seeing Moses - who had not started a club game since scoring for Liverpool in an FA Cup fourth round tie at Bournemouth in January - for the first time hardly enticed the locals to the game in their droves, there were some encouraging elements to be seen in a 3-0 victory.