The Dons took the lead when the Lions failed to clear Peter Leven's corner after nine minutes.
The ball fell to Sean O'Hanlon and the defender blasted home his second strike of the season from close range.
Millwall pushed forward in search of an equaliser but their dominance did not last.
Baldock doubled Dons' lead when he converted Luke Chadwick's cross from the left in the second minute of first-half stoppage-time.
The Lions' afternoon went from bad to worse when Paul Robinson put the ball past his own goalkeeper after 62 minutes for Dons' third. Baldock then grabbed his second in the final minute to complete Millwall's misery.