Despite a goalless first half, a penalty from Gary Alexander and cool finish from David Martin were enough for the points.
Former Lions striker Barry Hayles was shown a straight red card after a late aerial challenge on Millwall skipper Paul Robinson in the 28th minute.
And the away side were also unlucky not to break the deadlock when Alex Russell's shot from the edge of the box struck a post before the half-time whistle.
The breakthrough eventually arrived in the 69th minute when Gavin Caines fouled Alexander and the frontman picked himself up to fire his penalty past Cheltenham keeper Shane Higgs.
The points were wrapped up when Martin was found by Nadjim Abdou with nine minutes to go and he converted left-footed from 18 yards out.