The Terriers took the lead after five minutes when goalkeeper David Martin failed to collect a Gary Roberts corner and Phil Jevons pounced to hook home the ball from seven yards.
City levelled after 11 minutes when a Bruno Berner drive cannoned into the net off Michael Morrison from six yards.
Town recaptured the lead in the 51st minute as Keigan Parker took full advantage of a hesitant home defence to lift the ball over Martin from 18 yards, but City equalised for the second time after 59 minutes when Matt Oakley hoisted in a free-kick and Steve Howard nodded on for Berner to drive home at the far post.
Leicester had their noses in front for the first time after 63 minutes as Matty Fryatt chested the ball home from five yards following Morrison's deflected shot and soon after Jack Hobbs headed home from close range to seal the win.
Huddersfield's frustration was complete after 88 minutes when Gary Roberts was first show a yellow card for kicking the ball away and then a straight red for abusive language aimed at the referee.