Adi Yussuf and striker Taylor twice put Gary Rowett's side ahead at Bramall Lane on Saturday, only for debutant Nick Blackman and Neill Collins to bring the Blades back level and force spot-kicks.
The visitors opened the scoring just after the half-hour mark when midfielder Yussuf finished Robbie Weir's pass from the edge of the box.
Former Blackburn man Blackman was introduced at half-time and headed home Shaun Miller's cross from six yards to restore parity six minutes later.
Blades full-back Tony McMahon fired over the crossbar before the game edged into extra time, during which Taylor slotted in another Weir pass to restore the League Two side's advantage.
Defender Collins wasted a penalty in the 103rd minute as his kick was saved by Ross Atkins, but the Scot redeemed himself two minutes later to take it to penalties, with Albion's Marcus Holness dismissed in the ensuing melee.
And, after the shootout had gone into sudden death, Taylor proved to be the hero as he struck the decisive kick following Michael Doyle's failure.
