The Lions went into the match against the in-form Shrimpers with only one win in seven games.
But that all changed in the 39th minute when midfielder James Henry, on loan from Reading, delivered the ball into the Southend box.
Alexander hit the ball into the bottom right-hand corner from six yards for his eighth goal of the season. Southend had forced several saves out of Millwall goalkeeper David Forde before Peter Clarke hit a post with a header.
And Forde produced a fine save to deny Franck Moussa just before half-time.
The Shrimpers stepped up their hunt for a equaliser, but the Lions soaked up the pressure and forced a save from Steve Mildenhall after David Martin set up Ashley Grimes for a shot.