The 30-year-old has been overshadowed by striker partner Neil Harris so far this season and that looked set to continue after Harris netted his 10th goal for Millwall after three minutes.
But Alexander made sure it was his time to step into the limelight at the New Den heading the Lions home in the 83rd minute after Patrick Kisnorbo's first-half equaliser.
Leeds travelled to south London as the only unbeaten team in English football but were warned that may change when Danny Schofield's shot was blocked by Richard Naylor 60 seconds in.
However, Harris made no mistake minutes later, heading Chris Hackett's cross over Casper Ankergren only for Kisnorbo to return the favour nine minutes later when he converted a Robert Snodgrass corner.
Introduced in the 61st minute, Alexander rushed onto Schofield's pin-point cross and headed home from six yards to extend Millwall's unbeaten run to five matches and end Leeds' unbeaten start to the season.