The Rams were the better side for the majority of the match and comfortably led through Chris Martin's first-half goal.
A first triumph on Leicester soil since a 3-0 success at Filbert Street in February 2002, when this was a Premier League fixture, looked firmly on the cards - until the introduction of Anthony Knockaert.
The Foxes substitute equalised with a stunning free-kick after 78 minutes and then had a hand in Danny Drinkwater's winning goal just three minutes later.
There was a big scare for Derby in the opening minute. After surrendering possession straight from the kick-off, Kieron Freeman had to clear the ball off his own line after Tom Hopper headed Liam Moore's long throw goalwards.
Zoumana Bakayogo returned the favour in the ninth minute when he was in the right place at the right time to keep out Jake Buxton's header.
The Rams were on the attack again shortly afterwards and a clever run across the box and shot from Craig Bryson was pushed to safety by Kasper Schmeichel.
Nigel Clough's side then had a couple of penalty appeals midway through the first half, but both were waved away by referee Dean Whitestone.
First Bryson tumbled over Leicester debutant Marcin Wasilewski and then Paul Coutts went down under a challenge by Ignasi Miquel.
The Derby midfielder was penalised for handball in the second instance but more worryingly for Clough he lay in some pain on the ground before having to be helped from the field and replaced by Ben Davies.
Derby deservedly made the breakthrough three minutes before half-time. It was good work inside the box from Johnny Russell to set up Martin who let the ball run across his body before beating Schmeichel with a low shot.
However, the Rams needed a brave piece of defending from Freeman to ensure they went into the break ahead.
Another Hopper header from a Moore throw-in looked like it would be poked in by Gary Taylor-Fletcher but Freeman got ahead of the Foxes' new striker to deny him a goal on his home debut.
Home boss Nigel Pearson made two changes at the break as Lloyd Dyer and David Nugent replaced Bakayogo and Hopper.
Leicester improved immediately but never really tested goalkeeper Lee Grant despite their spells of sustained pressure.
Pearson's final throw of the dice from the bench saw Knockaert come on for Taylor-Fletcher after 63 minutes, and it would prove to be a telling switch.
Derby came within a whisker of extending their lead after 66 minutes when Russell diverted Craig Forsyth's shot just wide.
Knockaert levelled up the tie with a thunderous free-kick in the 78th minute. His 25-yard left-footer crashed in off the crossbar leaving Grant with no chance.
It was Leicester's first notable effort on goal and cruel on Derby - but not as cruel as what was to follow.
Just three minutes later the game turned on its head when another Knockaert free-kick found the back of the net via a couple of deflections with the final touch coming off Drinkwater.