The Millers dominated the first half but could not turn the pressure into goals until Josh Walker grabbed the winner in the 80th minute.
Inside the first minute, Adam Le Fondre tested Wayne Brown with a shot and Ian Sharps later somehow hoisted a shot over the crossbar from inside the six-yard box following a Nicky Law corner. From another Law corner, Walker and Le Fondre both had shots blocked and Sharps had a header cleared from yet another Law corner.
By contrast Bury never managed an effort on target and their only serious attempt at goal saw Tom Elliott head high over the crossbar from Tom Newey’s cross.
Brown came to Bury’s rescue nine minutes into the second half with a magnificent diving save from Marcus Marshall and as Rotherham continued to dominate without real penetration, Le Fondre steered a shot just past the post after 72 minutes.
The pressure finally paid off 10 minutes from time. Bury looked to have cleared the danger yet again but Walker picked up the loose ball 25 yards out, side-stepped the advancing defender and whipped a low drive into the corner of the net.