The League Two visitors had dumped Barnsley out in the previous round in a 2-0 success at Oakwell but there was to be no upset at the New Den on Tuesday night.
Bouazza opened the scoring in the 34th minute with a superb free-kick and Mkandawire added a second on the stroke of half-time with another fine finish.
Morecambe arrived in the capital on the back of three successive league wins and looked full of confidence as they kept the Lions at bay during the early exchanges and fashioned chances of their own.
The Shrimps went close to opening the scoring inside the second minute when striker Danny Carlton hit an 18-yard shot which flashed narrowly wide.
Moments later Millwall went close at the other end of the field as Bouazza hit a shot from 12 yards which drew a fine parried save from visiting goalkeeper Barry Roche.
Josh McQuoid went close from a corner for Millwall and Bouazza also threatened before he opened the scoring in the 34th minute.
Kieran Charnock fouled McQuoid and Bouazza stepped up to send a superb left-footed free-kick into the top corner from 25 yards.
McQuoid then saw a shot parried by Roche but the second goal arrived in first-half stoppage time.
Chris Hackett found Mkandawire with a fine pass and the Lions midfielder hit a right-footed shot into the net from 20 yards.
Roche kept the score down after the break with another smart stop from McQuoid but the Shrimps had chances at the other end as winger Kevin Ellison flashed a shot just wide on the hour.
Carlton then saw a shot parried by Steve Mildenhall in the 68th minute but the Lions held on with relative comfort.