Rotherham grabbed the initiative with a goal after just five minutes. A quick throw out by goalkeeper Adam Collin found Kieran Agard wide on the left and as he cut into the box Ashley Eastham got in a half tackle that only deflected the ball straight to Lee Frecklington and he gratefully slotted the ball into the corner of the net.
And Rotherham could have had two more in quick succession. In the 13th minute Agard unleashed a fierce drive that was well held by Josh Lillis and seven minutes later Agard looked offside but the flag never came, but Lillis and Michael Rose combined to clear the danger.
The visitors did not manage a first half shot on target and a minute before the break Agard again tested Lillis following a great run through the heart of the Rochdale defence.
But Rochdale improved dramatically after the break and equalised 10 minutes after the re-start. Rose floated in a high cross from the left which evaded everyone and Matthew Lund turned the ball into the net off his chest from a yard out.
And Rotherham were reduced to 10 men with 17 minutes to go when Michael Tidser collected his second yellow from referee Graham Scott, and it was just the spur the visitors needed with twin substitutes Joe Bunney and Peter Vincenti combining for Vincenti's 83rd minute winner from Joe Bunney's cross.