Mark Roberts gave the hosts a two-goal lead moments before half-time after James Dunne and a Pim Balkestein own goal had earlier put the Dons on the back foot.
Brown's men were unable to force their way back into the game in the second half and their manager admitted the game was already lost by that point.
He said: "I was disappointed with the first half because I thought Stevenage tore us to shreds at times.
"We were chasing the game and I didn't ever think we were really going to recover.
"We came here knowing that this would be a difficult place to come and we struggled to cope with the pace of their two wide players."
