Wanderers were heading out in the first round until Craig Davies' spot-kick in the seventh minute of added time cancelled out Ryan Lowe's opener and ensured extra-time.
Substitute Neil Danns then scored twice - the first via a huge deflection off Chris Sedgwick - while Jim McNulty pulled one back as Dougie Freedman's men won the first meeting between the two sides in 12 years.
The visitors started brighter and got their reward when Tim Ream's slip allowed Andrew Tutte to pick out Lowe at the far post, and the 35-year-old finished with aplomb.
Bury were playing with plenty of freedom, although goalkeeper Shwan Jalal and McNulty were caught overplaying at the back with the former recovering well to deny Jermaine Beckford from point-blank range.
Jalal sustained an injury so Rob Lainton, released by Wanderers last summer, came on for the second half and produced a brilliant one-handed stop to repel David Wheater's prod.
After Beckford had squandered another glorious opportunity, the hosts were handed a controversial lifeline by referee Jeremy Simpson when Lainton tangled with Davies as he moved away from goal. The Welshman drilled his spot-kick into the corner to take the game into another 30 minutes.
Three goals in four minutes soon followed with Sedgwick unable to stop Danns' touch going over the line before the former Leicester midfielder curled in a fantastic free-kick. However, David Flitcroft's side hit back within 74 seconds, McNulty bundling in Nicky Adams' free-kick.
Bury, who were only able to name six substitutes, struggled to rekindle the promise they showed in the first half and went out ruing Davies' penalty, and the decision not to give a spot-kick when Nathan Cameron was hauled down in extra-time stoppage-time.