The Foxes were on the front foot from the start with Lloyd Dyer letting fly with a 12-yard shot which was pushed around the post for a corner by goalkeeper Peter Gulacsi.
But it was Hereford who went in front in the 28th minute after Jennison Myrie-Williams had been brought down in the penalty area by Joe Mattock and Steve Guinan drilled home the penalty.
Leicester levelled seven minutes later when Matt Oakley fired a free-kick over the head of Gulacsi into the top corner of the net.
The Foxes then took the lead in the 73rd minute when Dyer drilled the ball home after defender Richard Jackson headed a shot from Steve Howard against the crossbar.
And with four minutes remaining Howard secured the three points when he tapped a close-range shot from King into the net to increase the Foxes lead over Peterborough at the top of the table.