Paul Quinn, who had been nursing a groin injury for most of extra-time, slotted home the decisive penalty after Gelson Fernandes had missed for the Foxes.
Don Cowie opened the scoring for Cardiff with a header, only for Steve Howard to poke home his first goal of the season to equalise.
Lloyd Dyer slammed in Leicester's second before Rudy Gestede's first goal for Cardiff forced extra-time.
Fernandes skied his penalty, allowing Quinn to win the match with the 14th spot-kick.
Cardiff made seven changes for the game and Leicester 10, and it was the visitors who dominated the early exchanges.
An inviting cross from Franck Moussa picked out Jeffrey Schlupp, but the striker's header cleared the crossbar.
Cardiff hit back and some great build-up play ended with Gestede firing in a shot from 12 yards which forced a decent save from Chris Weale.
Stephen McPhail was forced out of the action through injury in the 29th minute, but the Bluebirds took the lead shortly afterwards.
A cross from Dekel Keinan was only half-cleared and the ball fell kindly for Cowie to nod home his third Carling Cup goal of the season from 10 yards.
Their joy was short-lived, though, as Leicester levelled five minutes before the break.
The impressive Schlupp beat two defenders and cut the ball back from the byline for Howard to force the ball in off the post from close range.
Dyer saw a shot deflected wide and Sean St Ledger headed over in a bright start to the second half.
Cardiff goalkeeper David Marshall comfortably gathered a 25-yard drive from Neil Danns, but the Foxes midfielder played a part in his side's second goal shortly afterwards.
Danns bustled his way into the penalty area and saw his shot blocked but the rebound fell to Dyer and he smashed the ball into the bottom corner.
The Bluebirds equalised in the 82nd minute when Darcy Blake played the ball to Gestede on the edge of the area and he turned and tucked the ball past the keeper.
A chipped effort from Howard was tipped over, while Schlupp almost snatched the visitors victory in the last seconds of normal time but Marshall saved well.
Substitute Yuki Abe was played in by Schlupp at the start of extra-time but Marshall saved again as the Foxes looked the side more likely.
Howard set up strike partner Schlupp but he headed wide as time ticked away and penalties loomed.
After 10 successful spot-kicks it moved into sudden death, where Fernandes missed and Quinn kept his nerve to send the Bluebirds through.
