The Blues had Willian sent off midway through the second half with the score at 0-0 and lost Ramires to another red card in the closing stages, while their misery was compounded in stoppage time when manager Jose Mourinho was sent to the stands.
Delph converted in the 82nd minute with a masterful flick, denting Chelsea's title surge.
The midfielder won possession in midfield and passed to Marc Albrighton, who returned the ball for the former Leeds midfielder to back-heel superbly past Petr Cech.
And the man of the match told Sky Sports 1: "I thought we were the better team from the start and we looked like we wanted it more.
"We were well organised, we attacked when we had the chance and we defended well.
"We've been scoring goals like that in training and it's nice for it to come off in a game.
"Marc Albrighton's been superb ever since he's come back.
"I kept going at it, managed to get the breakthrough, gave it to Marc and he was going to cut it back.
"It was my best goal so far in the Premier League. We've been working hard since the last game. No-one gave us a chance. We needed to get a job done here, and luckily we've done it."
Delph missed the chance to double Villa's lead late on, and admitted: "I was a bit disappointed, to be honest. I thought I should have put it away. That'll haunt me a little bit but, at the end of the day, the three points was the more important thing and we're going home happy."