Gavin Tomlin's double and goals from Ryan Cresswell and Barry Corr saw Southend make it six games unbeaten and took their tally to 10 away goals in their last three matches.
After the Kingsmeadow thrashing in League Two, Ardley said: "Too many players were playing for themselves. I need to sit down with the players and get them to respond - I've told them they need to look at themselves. What happened in the second half was unacceptable.
"We worked hard on set-pieces in the week, talking about how their defenders were a threat from them. As manager I take the rap for that, but I don't think the boys did what I asked of them.
"We had 10 on the injury table, but I'm not going to hide behind that because the performance was unacceptable - too many players underperformed. We didn't show enough character in adversity and it wasn't good enough. It's a character thing - sadly we were lacking, and we need to roll up our sleeves.
"We need to freshen them up as best as we can, and produce a performance that puts this one to shame. It's how you bounce back that matters. The staff will roll our sleeves up, we just the need the disciples to follow our lead."