James Bree netted the winner for the Tykes in sudden death after Sean McAllister had lifted his spot-kick over the crossbar for Scunthorpe.
Earlier, Ben Smith had the chance to win it with his fifth penalty for the visitors but his effort was saved by goalkeeper Joe Anyon.
There was penalty drama all the way through this evenly-balanced tie.
Two spot-kicks in the opening seven minutes of the second half brought the game to life after a goalless opening period.
It was Barnsley who struck first in the 47th minute when Anyon brought down striker Conor Wilkinson in the box as he tried to recover a mistake by centre-back David Mirfin.
Anyon had already marked his debut for the home side with a stunning 14th-minute penalty save to keep out a spot kick from Marley Watkins after Josh Scowan had been brought down in the box by Jake King.
But this time it was Scowan who made no mistake from the spot to put the Tykes in front.
However, their lead was short-lived as five minutes later referee Graham Salisbury pointed to the spot for a third time as he spotted a holding offence by defender Marc Roberts in a crowded box as Barnsley tried to defend a free-kick.
It was last season’s leading marksman Paddy Madden who opened his account for Scunthorpe as he confidently swept the penalty past keeper Nick Townsend.
Both sides went close to a winner before the tie went to penalties.
Madden, Tom Hopper, Hakeeb Adelukan, Scott Laird, Neal Bishop and Mirfin were all successful for Scunthorpe. But Kevin Van Veen and McAllister missed their attempts.
Sam Winnall, Conor Hourihane, Scowan, Alfie Mawson, Roberts and Lewin Nyatanga were all successful for Barnsley before Smith took the glory with the final penalty.