Despite dominating from start to finish, the promotion-chasing Millers left it until the 76th minute to get the only goal.
Lionel Ainsworth's left-wing corner was flicked on at the near post by Kari Arnason and Alex Revell arrived late to head home.
"This is a tough place to come and we thought we had got through the worst of it. We knew what Rotherham would do and how good they would be at it, but it was a catalogue of little things that led to the only goal," Ardley said.
"We talked about it on Thursday and again at half-time, so it is disappointing and all the hard work and all the effort we put in to get through to that stage went out the window.
"It was down to a bit of group thinking. We dropped too deep, gave away a corner and then two players lost their markers and you cannot do that.
"That's what the league is about. That was early season form for us, giving away goals from set-pieces, and it's cost us here and we still have a lot of work to do."
AFC Wimbledon may have dropped two places to 18th in the League Two table but they remain seven points clear of the drop zone.