Former Millers striker Deon Burton netted the only goal of the game in the 22nd minute as the Gills extended their lead over fourth place from ten points to 11.
The 36-year-old striker, who played for Rotherham seven seasons ago, buried Danny Kedwell's flick on from a long Chris Whelpdale throw for the winner in the first half.
Gillingham are also eight points clear of Port Vale in second with four games remaining to their five however boss Allen will not be celebrating anything just yet.
"We need to keep doing it right and keep going," he said. "We cannot let our standards drop. Rotherham are good, they have invested a lot, but they weren't that good.
"They gambled as you do at the end so it was tricky and testing. Apart from one long-range effort though, they never got near our goal.
"It was a top, top performance. We made the decision at Fleetwood (last time out) that there is no room for excuses, let up or sleep because Rotherham had a day extra to recover.
"We dominated from start to finish and we should have had loads of goals. Deon will get the headlines but this is a squad, on and off the field we were very, very strong.
"Other results have gone our way. Everybody knew it was April Fools' Day and there was no way we were going to be the fools."