This was always likely to be a tricky tie for the Hammers with their opponents, leading the Sky Bet Championship, in such buoyant mood after recent results and bidding to win an 11th straight League Cup tie at Turf Moor.
Yet, in a much-improved second-half showing, Sam Allardyce's substitutions helped turn the tide in the visitors' favour with Kevin Nolan, one of those replacements, winning the all-important first spot-kick and Collison earning and converting the second.
The hosts had shaded the first half but found stubborn resistance in the form of Hammers goalkeeper Adrian, who twice repelled efforts from Junior Stanislas, while a stop from Danny Ings sandwiched in-between was the pick of the bunch.
Even with two recognised strikers in their line-up, the visitors were struggling to fashion openings, a rusty Carlton Cole also needing treatment half an hour in for a knee injury.
At the other end Sean Dyche's men almost got their reward before the interval.
Kieran Trippier's cross was met by Sam Vokes' head and although he planted his effort against the post, the assistant referee's flag brought a halt to proceedings.
Sensing the need for extra invention, Allardyce replaced Mohamed Diame with Collison for the restart but he must have felt deflated when debutant Pelly Ruddock's wild effort was dragged out for a throw.
They were beginning to finally show some attacking intent, though, and had the ball in the net, only to be denied by the offside flag, as Joe Cole stopped Modibo Maiga's pot-shot before dispatching a neat overhead kick.
A tense cup tie suddenly sparked into life, Keith Treacy's well-struck volley just failing to stay low enough to trouble Adrian.
The stopper was all at sea after the hour, though, as Burnley again hit the woodwork.
Tomkins' needless foul on Ings set up Trippier's set piece and the full-back's viciously swerving effort beat Adrian before cannoning off the crossbar.
Allardyce continued to turn to the bench in a bid to wrestle the game's momentum and it was Nolan whose introduction did lead to the opening goal 76 minutes in.
The former Bolton midfielder collected the ball in the area and went to ground under a challenge from Clarets captain Shackell, who looked to have nicked the ball away fairly.
Nevertheless referee Madley pointed to the spot, booking Shackell, and allowing Taylor to convincingly slam home the spot-kick in front of the away fans.
As the visitors tried to force extra-time, Collison sprung on the break and was hauled down by Treacy, with the winger being sent off, before the Welsh midfielder himself stroked home the penalty to hand Dyche's men their first defeat since August.