Australian defender Jason Davidson scored the winning spot-kick after Oxford captain Jake Wright had seen his effort saved by Boaz Myhill.
The Baggies, with seven changes from the side that started against Southampton at the weekend, looked lively early on and made the breakthrough just before the half hour mark through Jonny Mullins’ own goal.
However, their performance lacked urgency thereafter as they toiled to find the second goal that would have made for a more comfortable evening.
Oxford tried to take advantage of labouring Albion and improved after half-time. But it was no surprise to see a team who have lost all four of their League Two matches this season, scoring only one goal, struggle to create a clear-cut opportunity.
It looked like Mullins’ misfortune would be enough to see West Brom sneak through but Danny Hylton headed the equalising goal with four minutes remaining.
Oxford could have nicked it in extra-time before they found themselves hanging on after Junior Brown’s red card.
Oxford almost made the worst possible start when Chris Brunt got across his marker at the near post to chest the ball goalwards. Max Crocombe nervously fumbled but kept it out and then gathered at the second attempt.
Shortly afterwards Joe Riley’s last-ditch tackle denied Saido Berahino after recent signing Brown Ideye, making his full debut, had put the England Under-21 international through on goal with a nicely weighted pass.
There was a scare for West Brom when Josh Ruffles’ shot deflected narrowly wide, but the Baggies were soon putting the visitors under pressure again.
Crocombe spilled Brunt’s cross but Ideye flicked the ball over his shoulder – and the crossbar. The young keeper got away with another moment of nerves midway through the first half when Stephane Sessegnon’s 25-yarder bounced from his grasp but Oxford’s good fortune was about to run out.
Andre Wisdom’s low cross from the right looked harmless but the ball struck Mullins on the leg eight yards out and bounced into the bottom corner of the net.
As like the start of the match, Oxford were nearly caught out at the beginning of the second half. However, Ideye failed to connect with Wisdom’s low cross with the goal gaping and the League Two side escaped.
Claudio Yacob then spurned a chance to extend the lead but he headed straight at Crocombe when unmarked at the far post.
Oxford enjoyed their first successful spell of the game as the midway-point of the second-half approached.
Gareth McAuley made a sliding block inside his penalty area to prevent Ruffles from having a clear sight of goal, before West Brom were carved open down the right and Carlton Morris shot wide of the far post. Wisdom then had to block Hylton’s goalbound effort.
It was beginning to look like Oxford’s chance had passed them by but with four minutes remaining a long, diagonal ball into the box was headed back across goal by substitute Will Hoskins and Hylton nodded it over the line from a yard out to send the tie into extra time.
Yet again West Brom had an early chance when play restarted as James Morrison’s 15-yard shot was deflected just wide.
But Oxford, tails up after the equaliser, almost went ahead when Myhill spilled a cross but Craig Dawson did brilliantly to scramble the ball away. Myhill then made amends with a good block to deny Brown.
Brown was sent off midway through the second period of extra-time when he hacked down Dawson but Oxford held on to take it to penalties, surviving a late scare as Dawson headed what he thought was the winning goal only for the referee to rule it out for a push.
West Brom must have feared the worst when Morrison and Ideye missed but Myhill saved Michael Collins’ and Tom Newey’s spot-kicks as it stood at 3-3 after five penalties each.
Each side netted three in a row before Myhill saved Wright’s effort and Davidson held his nerve to send Albion into round three for the first time since 2011.