Newly-appointed captain Morgan, signed from East Midlands rivals Nottingham Forest in January, celebrated taking over the armband from the departed Matt Mills over the summer by powering home a header early in the second half.
It was Leicester's first real chance of the game after Posh had dominated possession in a largely uneventful first half without troubling home goalkeeper Kasper Schmeichel.
Substitute Andy King made sure of all three points for Nigel Pearson's side with a superb strike from the edge of the box with 15 minutes remaining.
Both managers saw their sides win 4-0 in the Capital One Cup in midweek and named unchanged teams.
Foxes boss Pearson handed full league debuts to defender Ritchie De Laet, midfielder Matty James and striker Jamie Vardy while another summer signing - midfielder Anthony Knockaert - was on the bench.
Goalkeeper Bobby Olejnik and midfielder Michael Bostwick made their league bows for Peterborough.
Posh manager Darren Ferguson gave teenager Kgosi Ntlhe his first Championship start, with the 18-year-old lining up at left-back.
Former Manchester United trainee De Laet headed over an early chance from Ben Marshall's free-kick.
Peterborough enjoyed some good spells of possession inside the opening half-hour but they were wasteful with the final ball in the last third.
Centre-half Shaun Brisley almost undid their good work when he presented the ball to Vardy midway inside his own half.
Former Fleetwood forward Vardy broke into the box and squared a pass to strike partner Jermaine Beckford but defender Gaby Zakuani got there first and Olejnik had to react brilliantly to stop his skipper scoring an own goal.
Posh's first shot of any note came four minutes into the second half. Paul Taylor beat Liam Moore to a ball down the left and cut inside before hammering in a shot that Schmeichel beat away to safety.
Morgan gave Leicester the lead after 51 minutes.
The big centre-half powered a header in off the underside of the bar after connecting perfectly with Marshall's corner from 10 yards out.
The Foxes had their tails up now. De Laet crossed from the right but Beckford could not quite climb high enough and Vardy headed over.
Olejnik then had to produce an excellent save to deny Vardy after he tried to lob the Posh keeper from a ball over the top.
Leicester's second goal came after 75 minutes.
Vardy showed great determination to carry the ball away from Brisley down the left before playing an intelligent cut-back towards the edge of the box where King arrived and steered a first-time, side-foot shot into the top corner.
Peterborough substitute Emile Sinclair could have got the visitors back into the game almost immediately but he shot straight at Schmeichel after scampering into space behind the Foxes backline.
