Callum Wilson opened the scoring with his first since his return from a long-term shoulder injury.
Malvind Benning equalised for the Saddlers before Delfouneso's first for the Sky Blues since his arrival from Aston Villa on loan made sure City moved six points away from the drop zone.
The Sky Blues took the lead after 22 minutes, when after some good link-up play with Chuba Akpom, Wilson raced through unopposed to slot the ball past Richard O'Donnoll for his 16th of the season.
But the Saddlers got the equaliser after 53 minutes when Febian Brandy's effort inside the area was parried by Joe Murphy and Benning tapped in the rebound from three yards for his first of the campaign.
Delfouneso came on for Akpom after 62 minutes, and the striker scored the winner in the 84th minute when he diverted Michael Petrasso's cross in from eight yards out.
The result leaves Walsall five points off the play-offs and winless in their last six outings.