The Shrimpers were beaten 2-0 at Wembley in Brown's third game in charge of the League Two club.
With previous manager Paul Sturrock sitting among the fans, Brown was subject to criticism for his starting XI, but he is ready to move on from the defeat.
Brown told Echo News: "In hindsight maybe that was the wrong decision and it's something we'll look to redress in the next five games.
"[But] you can't judge yourself on hindsight as a manager. I have no regrets about the team I picked because I don't think you can.
"You pick what you think is right and if that works then great, but if it doesn't then you go back to the drawing board and try something else."