Kick-off was delayed until 8pm after a traffic accident held up some of the Dons squad, with the home side forced to replace goalkeeper Ross Worner with deputy stopper Brown.
And it was Brown's error that allowed Callum Wilson to equalise Michael Smith's opener before Carl Baker scored one and claimed a second for the League One side.
Worner eventually arrived at the ground on the final whistle and while eager not to criticise Brown, Ardley admitted the error changed the game.
"If we don't concede the goal we win the game, I know that sounds funny to say but I genuinely couldn't see them getting back into the game," he said.
"They would have committed more men forward and we would probably have had more chances at the other end but it is gone now.
"They are all humans and everyone makes mistakes. We have to be careful, he knows, I don't need to tell him.
"These things happen in football, I've told him to get his head up and I've told the lads how proud I am of them, goals change games.
"Their goalkeeper was probably man of the match with some of his saves.
"It would be disappointing if my team let me down, it would be disappointing if we didn't do our club proud but unless I'm watching a different game I thought we were fantastic.
"That was close to being as good a performance from my team since I've been in charge."