Shrimpers substitute Barry Corr salvaged a share of the spoils for the Shrimpers when he headed home five minutes from time at Roots Hall.
But Brown was still left frustrated by his side's display.
"I was disappointed with how we played and too many of the team wanted to pass it on and let someone else make things happen," said Brown.
"Against Bradford on Friday we had seven or eight players who put themselves forward for a starting slot at Wembley in this weekend's JPT final with Crewe but there were far fewer than that out there.
"We started off with a 4-3-3 formation and I decided to include Freddy Eastwood as he's a talisman here at Southend.
"However, it didn't really work and I thought we lacked width.
"It was only when Barry Corr and Marc Laird came on that we started to play well but we were still reliant on our goalkeeper having to make a couple of good saves."