The Seagulls were reduced to 10 men in the eighth minute when Lewis Dunk saw red for bringing Yannick Bolasie down when he was through on goal.
The Eagles then took advantage of their extra man as Glenn Murray scored twice against his former club before Owen Garvan sealed the win.
Poyet said: "You come here to play a nice game of football against your biggest rival away from home and the least you want is a normal game. Going down to 10 so early - it put a big task ahead of us, but that's football I suppose.
"It was an absolutely perfect decision, it was a red card.
"It was a key moment when Craig Mackail-Smith missed.
"Crystal Palace scored before half-time and it changed completely our approach for the second half.
"We had to reorganise and go back out with a different mentality because we were losing. We knew we would have one or two chances and what a chance.
"If we scored you have got a different game but it was not a day for anything to come for us, not even a bit of luck or mistake from one of them or the referee."