The Blue Square Premier side had their moments early on with Steve Morison nearly taking advantage of a badly hit backpass from Aleksandar Tunchev.
Morison then dragged a shot wide of the post before City opened the scoring in the 37th minute when Matty Fryatt beat two defenders on the right before crossing for Lloyd Dyer to score with a crisp drive.
Chris Day kept Borough's hopes alive after the restart with quality saves to deny first Fryatt and then Matt Oakley.
But the keeper was beaten after 52 minutes as Dyer raced down the left before finding Fryatt who scored with a carefully placed shot from just inside the area.
Leicester wrapped up the game with 21 minutes remaining when Andy King spotted the keeper off his line to score with a spectacular 25-yard lob which went in off the bar.