Orient were presented with a golden opportunity to take an early lead in the League One clash at The New Den when Tony Craig upended Charlie Daniels as he broke into the area after eight minutes but Scott McGleish missed the penalty.
Orient went ahead when keeper Jamie Jones launched a long kick forward to McGleish, who in turn put Simon Church through on goal and the striker drilled his low shot beyond Forde.
Dave Martin should have equalised on the hour-mark but he dragged his shot from close range wide of the post after connecting with Neil Harris' cross.
Millwall were back on level terms 10 minutes later when Lewis Grabban's pass found Alexander and the striker buried his shot past Jones from 18 yards.
And Alexander took his league goal tally for the season to 13 and sealed victory for the Lions two minutes from time when he rose at the far post to head home James Henry's cross.