David James saved John Carew's penalty before the break at Fratton Park after the Norwegian had restored parity following Michael Brown's early opener for the hosts.
The game looked to be heading towards stalemate but Delfouneso tapped home Emile Heskey's flick on to move Villa above Liverpool into sixth place in the Premier League.
Avram Grant decided to hand 18-year-old Leonard Sowah his first Premier League start in a Pompey team that contained six changes from Wednesday's 0-0 draw with Wigan.
That meant Sowah became the first player who was born after the Premier League was established to start a top-flight game.
James, Aaron Mokoena, Hassan Yebda, Kanu and FA Cup semi-final hero Kevin Prince Boateng also came in but striker Frederic Piquionne did not make the match-day squad.
Villa boss Martin O'Neill opted to stick with the same team that drew 2-2 with Everton on Wednesday.
Both sides went into the game having suffered differing fortunes in their respective games at Wembley last weekend.
Villa crashed out of the FA Cup with a 3-0 defeat against Chelsea while Pompey pulled off a surprise win over Champions League-chasers Tottenham Hotspur.
O'Neill still harboured hopes of European qualification via the league though - and he knew a win would lift the Midlanders above Liverpool.
Pompey's first chance came four minutes in, when Brad Friedel spilled a John Utaka cross, the ball fell to Kanu but the Nigerian's shot was blocked by James Collins.
Fratton Park went into uproar in the ninth minute when Anthony Vanden Borre raced down the right, out-muscled Stephen Warnock, and laid the ball off to Michael Brown who carefully placed the ball past Friedel to put Pompey 1-0 up.
James then reacted well to deny Stewart Downing and Carew but the Norwegian beat the England stopper minutes later to draw Pompey level soon after.
Marc Wilson's attempts play Carew offside from Warnock's through-ball failed miserably and the Villa striker raced through before cutting back across the defender to beat James.
Wilson then almost put the ball into his own net to make it 2-1 to Villa but his deflected clearance rebounded off a post.
The near capacity crowd were being entertained by some end to end football as both sides looked to take the advantage.
Gabriel Agbonlahor thought he should have been awarded a penalty when he appeared to be dragged down by Vanden Borre in the box but referee Lee Probert waved away his protests.
Papa Bouba Diop then gifted Villa a spot kick by bringing down Carew after losing possession inside his own box.
The Norwegian chose to power his drive down the middle but James saved to deny Villa the lead.
Villa came out of the blocks quickest in the second half, with Young proving the main threat.
The 24-year-old burst down the right for the visitors but young Sowah showed maturity beyond his years to stifle the quick winger's mazy run five minutes into the second period.
James Milner then beat two Pompey defenders before seeing his 25-yard strike comfortably saved by former Villa goalkeeper James.
Young then switched to the left flank where he crossed for Agbonlahor, but James was there again to stop the striker's header from five yards.
Villa thought they should have had a second penalty when James appeared to pull down captain Stiliyan Petrov in the box.
O'Neill threw on Delfouneso for Agbonlahor and that proved to be a masterstroke as he put Villa ahead with his first touch of the game.
Milner crossed for Heskey, who headed the ball on before his fellow substitute tapped home from five yards to pick up his first league goal with eight minutes left.
Portsmouth pushed for an equaliser but Friedel saved their best effort from Ward at the death and O'Neill ended the game with a vital win.