City were given the perfect start with a goal after just 64 seconds when a miss-hit cross from Kerrea Gilbert flew into the far corner of Owain Fon William's net.
But Stockport were level after five minutes when the Foxes defence failed to clear to allow Chris O'Grady to hook home from 18 yards.
On-loan Arsenal full-back Gilbert almost grabbed a second after 14 minutes but this time Fon Williams was alert enough to claw away the ball from beneath the bar.
The second half began slowly with the first real chance not arriving until the 66th minute when Stockport substitute Greg Tansey fired a 20-yard free kick just over.
Leicester's on-loan goalkeeper David Stockdale produced a fine save to keep out a close-range effort from Oli Johnson and saved his best for the final minute when he denied Gary Dicker with an instinctive save.