The Shrimpers were second best to Crewe and fell behind to Luke Murphy's impressive effort as early as the sixth minute before Max Clayton added a second shortly after half-time, with the League One side going on to run out 2-0 winners.
Southend have won just two of their nine league fixtures since reaching their first-ever Wembley cup final back in February, and they must now turn that stuttering form around if they are to close the six-point gap to the top seven, starting with Wednesday's game in hand at Rochdale.
Speaking after Sunday's defeat, defender Barker said: "You've got a lot of disappointed people (in the dressing room). It's a big, big build-up. We've been through six rounds, I think, to get here so it is a big, big build-up and to lose at Wembley is obviously going to be a very disappointing thing.
"We've got a game again in three days. We're going to be all in again on Monday morning for a warm-down and training.
"We've got to keep on in the league. I think we've got to win five out of five and that's our major point now."
Barker added: "The aim at the start of the season was to get promoted, it's as simple as that.
"But we've obviously fallen off over the last month or so. We need to win five out of five and results to go for us. But we'll give it our all from Monday morning onwards.
"We know we've got to win every game to get into the play-offs and hopefully we can do it."