Following their stunning victory against Tottenham a week before, the Reds arrived in buoyant mood at St Mary's on Saturday but were second best throughout.
It was a dismal result against a team battling for Premier League survival and left the visitors seven points behind fourth-placed Chelsea.
"If you can beat Tottenham at home, you should come here and win," Downing told the club's official website. "If you want to be in the top four, you have to come to places like this and win.
"That's easier said than done. It's difficult with them being in a relegation fight and playing for the new manager as well. I still think we had chances in the game considering we didn't play well. We started too late and the game was over."
Early goals by Morgan Schneiderlin and Rickie Lambert left Liverpool rocking and only a Philippe Coutinho goal earned them minor consolation with Jay Rodriguez slotting in a cool third for the impressive hosts.
"We just didn't start brightly enough," Downing added. "The manager said we didn't start playing until we were 2-0 down. He stressed from the start that this was a difficult game, probably one of the hardest of the season.
"They are scrapping for their lives and, to be fair, they were the better team on the day. You've got to give them credit for that. We started too late."