It took just six minutes for Arsenal to open the scoring as Olivier Giroud swept home Alex Oxlade-Chamberlain's cross from close range, but Aston Villa drew level midway through the first half after being awarded a penalty.
Benteke's weak shot was kept out by Wojciech Szczesny, who was only shown a yellow card after bringing down Gabriel Agbonlahor, but the ball looped up kindly for the Belgian striker to head in the rebound.
Villa were given a second, more controversial, spot-kick just after the hour mark when Laurent Koscielny appeared to get a touch on the ball as he tackled Agbonlahor but was penalised by referee Anthony Taylor, and Benteke stepped up to send Szczesny the wrong way.
Koscielny was sent off moments later after being given a second yellow card following a late tackle on Andreas Weimann, but Arsenal threatened to fight back until Antonio Luna's late breakaway goal left the Gunners faithful to vent their frustrations with a chorus of boos at the final whistle.
Arsenal had started brightly and it was no surprise when they swept into the lead on six minutes. Oxlade-Chamberlain broke down the left and the England midfielder cut the ball back across the six-yard box where Giroud guided it into the far corner.
Villa, though, were level on 22 minutes. Agbonlahor cut through the centre of the Arsenal defence and darted in on goal. Szczesny was out quickly, but could only pull down the Villa forward as they went for the loose ball, which dropped to Weimann and he could only fire it into the side-netting.
Play was correctly called back by referee Taylor for a penalty, with the Arsenal keeper fortunate to have only been shown only a yellow card. Benteke struck the penalty low to the left where Szczesny made a superb save, only for the ball to bounce back up and the Belgian striker headed in.
Suddenly the atmosphere of the match changed, with plenty of heavy tackles from both sides to test the referee's patience. Arsenal had to make a substitution as Kieran Gibbs suffered a cut above his eye, while Villa captain Ron Vlaar and Jack Wilshere squared up after a clash on the half-way line, with both men booked.
Szczesny had to dart back into his goal after a rush forward to produce a full-length diving save as Ashley Westwood's 25-yard drive looked to be floating in.
Tomas Rosicky blasted over at the start of the second half after a neat one-two with Giroud on the edge of the 18-yard box.
Villa were almost ahead on 52 minutes when, following a quick break, Delph's low drive beat Szczesny, but bounced back off the inside of his right-hand post and across goal.
The Premier League's new goal decision system, GDS, came into play as the TV replays showed the referee had been notified the ball struck the base of the post and was not over the line before it rolled away.
The match officials, though, could have done with the benefit of a second look at the award of a second penalty for Villa on the hour.
Agbonlahor sprinted into the Arsenal area but Koscielny looked to have made a fine recovering tackle, getting a touch on the ball before the Villa man. However, referee Taylor decided otherwise and pointed to the spot.
This time, Benteke made no mistake, crashing the ball past Szczesny.
It got worse for Arsenal on 67 minutes when Koscielny, who had been booked for the penalty, was shown a second yellow card when he clattered into Weimann.
To their credit, Arsenal rallied, and Rosicky was denied by a good reaction save from Brad Guzan, who deflected substitute Santi Cazorla's half-volley up onto the crossbar.
But with Arsenal pressing forward, they were hit on the break as Villa's summer signing Luna darted clear to slot past Szczesny with five minutes remaining.