The Eagles were undone by two goals in two minutes from Rickie Lambert and Pablo Osvaldo at the start of the second half, condemning them to a third straight Premier League defeat.
Saturday's result leaves the south Londoners second bottom in the table with just one win this season and tipped for a long, hard battle against relegation.
"For large parts of the first half, I thought our game plan was working," Holloway told Sky Sports after the game.
"We were frustrating them and we had a couple of breakaways, not as many as I would have want, but a couple that could have led to things.
"But at the moment, things aren't going for us. This game is really harsh; does that flatter them a little bit? Yes, I think it does.
"The chances you get, you have to take them and if we do, the atmosphere in that dressing room will feel a lot better and a lot stronger.
"Everybody has bad runs, we got a win the other week and had a two-week break and lost our momentum.
"We've got to make sure we start picking up some points and get this group, who are newly formed, working together."
The main talking point from a Palace perspective came on 28 minutes when striker Marouane Chamakh went down under a challenge from goalkeeper Artur Boruc in the box.
The Eagles appealed for a penalty but referee Martin Atkinson instead gave Saints the free kick and booked Chamakh for diving - a decision Holloway grudgingly agreed with.
"He (Chamakh) still believes there is some contact that made him do that, but I can't understand why he didn't shoot," Holloway added. "I think he tried to lure the 'keeper and go round him and stick it in an empty net, but it didn't work out that way.
"The ref saw it that way and I have to say looking at that he is right; for me he (Chamakh) started to fall, so we will just get on with the next one."