The Shrimps have now triumphed in five and drawn one of the six meetings between the two teams. And it was a first half goal from Andrew Fleming which handed them their latest victory against the Shrimpers.
The midfielder won the ball back from Southend's player of the year Sean Clohessy before firing past goalkeeper Paul Smith.
"A mistake by our most consistent player and player of the year has cost us the game in the end and that's what life is like in this league," said Brown.
"This is a reality check for these players and I think there's a lack of character in the team which is something we need to sort out.
"Two seasons running they've been close to the top for about 75 per cent of the games before fading away at the end and we need to change that.
"It will be a busy few weeks and there's plenty of work and talking to be done. We certainly need to improve our home form but we're coming back in for pre-season training on June 10 and that signals our intentions for next season."