The three points ensured Palace reached the play-offs - where they will play rivals Brighton - and condemned Peterborough to relegation.
Lee Tomlin put Posh ahead but Glenn Murray levelled from the spot before half-time.
Nathaniel Mendez-Laing put the visitors back in front to stun Selhurst Park, but with Palace needing a point to ensure a top-six spot, sub Kevin Phillips finished superbly on 83.
At that stage, a point would have been enough to save Posh but Mile Jedinak's header ended a nine-match run without a win for Palace and sent Posh down.
"The way that game went is about as ridiculous as the last few weeks have been," said Holloway.
"What I'm finding out really quickly about this group is the confidence they have.
"We got back into the game twice when we needed to after we had got ourselves into a pickle and it worked.
"Kevin Phillips has been doing that all his career and hopefully he can help us. We've come through it now and I think we'll be stronger for that against Brighton.
"I'm not celebrating because now we have to come up with a good performance to get past our arch-rivals."