Bouazza struck after five minutes before Andy Keogh found the scoresheet for the third successive match with 20 minutes remaining.
Harry Kane completed the scoring in the 89th minute with a superb 30-yarder.
The result gave Kenny Jackett's side a eight-point cushion on the teams in the bottom three.
Peterborough, meanwhile, continue to labour and are just a point better off than tonight's opponents after a sixth defeat in their last nine league matches.
Emile Sinclair was recalled to the Posh starting XI in the only change from the weekend. He came in for Daniel Kearns, who dropped down to the bench.
Millwall made three changes.
Jack Smith returned from a jaw problem along with fellow defenders Darren Ward and Scott Barron as Tony Craig, Shane Lowry and Nadjim Abdou were replaced. Craig was amongst the substitutes.
Posh, with just one home win since mid-December, made an awful start.
James Henry found space down the right-hand side and crossed for Bouazza to head home from 12 yards out.
Darren Ferguson's side struggled to get going and could only respond with long-range efforts from Paul Taylor and Matthew Briggs - both of which failed to worry Millwall goalkeeper David Forde.
Forde's opposite number Paul Jones produced a finger-tip save to prevent Kane from extending the lead midway through the first half.
Jones denied Kane again early in the second half before the teenager struck the post with a 25-yarder in the 53rd minute.
Ferguson changed things around soon afterwards with the introduction of Lee Frecklington and then Kearns. Sinclair and Briggs were the players to make way.
However, Millwall struck again in the 70th minute.
Henry was again the provider, with Keogh heading home his free-kick from the right-hand side of the penalty spot.
Keogh also scored against Peterborough at London Road in October during his loan spell at Leeds when he was a Wolves player.
Taylor tried his luck again from outside the area but, like during the first half, the Posh striker could not work Forde.
Kane capped a fine performance with a deserved goal a minute from time when he collected Forde's clearance and belted the ball past Jones from well outside the penalty area.
