Libor Kozak had the best chance of a first period of few opportunities but saw his injury-time header from a corner put out by Adrian Mariappa.
The action improved after the break and Jason Puncheon forced Brad Guzan into a save early on, before the Villa keeper made an excellent save to push Barry Bannan's low shot onto the post.
The Villa keeper then denied Puncheon again, while substitute Jordan Bowery had his header pushed over late on at the other end.
But Gayle won it in injury-time after being played in one-on-one with Fabian Delph by fellow substitute Dean Moxey.
The Palace record signing made no mistake from 20 yards, curling an effort past Guzan, much to the delight of the travelling Palace fans.
The second away win of the campaign was a massive victory for Tony Pulis' side, who climbed out of the drop zone on goal difference, while Villa slumped to a fourth straight loss and were booed off the pitch.
Home boss Paul Lambert was still without injured pair Christian Benteke and Ron Vlaar for the game but Gabriel Agbonlahor returned from suspension to captain the side.
Palace were without suspended top scorer Marouane Chamakh so Yannick Bolasie came into Pulis' team.
Andreas Weimann and Kozak both kept their places in an attacking Villa line-up, but the first half was a story of frustration for the hosts.
They had plenty of possession but could not turn that into attempts on Julian Speroni's goal, with the Palace keeper more than equal to a couple of dangerous crosses.
It was not until the very end of the half that Villa really threatened to break the deadlock.
Karim El Ahmadi's volley was deflected on its way over the bar and from the corner Kozak's header looked destined for the top corner until Mariappa got his head in the way.
Palace were content to soak up pressure a lot of the time but when they did break forward they caused Villa problems.
Headers from Bolasie and Puncheon were held by Guzan, while on more than one occasion a lack of precision meant the visitors were unable to capitalise on promising positions.
The tempo increased at the start of the second half, and it was Palace who probably should have taken the lead in the 48th minute.
The lively Bolasie raced down the right wing and pulled the ball back for Puncheon, who was in lots of space on the edge of the box, but the forward got the ball stuck under his feet and could only poke a shot towards goal that was beaten away by Guzan.
Villa responded and in the 54th minute a spell of pressure ended with Delph picking out Weimann with an excellent ball, but the Austrian chose to shoot first time and it was straight at Speroni.
Palace broke straight down the other end and Puncheon picked out Bannan just inside the box. The Scot turned and struck a shot towards the bottom corner that was superbly pushed onto a post by Guzan.
Villa were having the majority of the play, though, and, after an Agbonlahor run ended with the striker shooting straight at Speroni, Weimann shot over when well placed and smacked the ground in frustration.
Bolasie was a real dangerman for Palace and in the 64th minute he got to the byline before pulling the ball back across the box for Puncheon, who was again denied by Guzan.
That was the forward's last action as he was replaced by Jonathan Williams, with Pulis also sending on Gayle for Cameron Jerome.
Villa switched to a back three as Leandro Bacuna, Bowery and Alexander Tonev entered the fray, and it was Bowery who looked to have come closest in the closing moments with an 89th-minute header that was tipped over by Speroni before Gayle's late, great intervention.