Albion had the best chances of the first half, with a Craig Dawson header being pushed over by Lukasz Fabianski.
There was also Steven Reid's free-kick, seemingly sent over by a slight touch of Fabianski's glove, although the goalkeeper argued he had not made contact, hoping to prevent a corner that was given nonetheless.
In terms of Arsenal's impact in the opening 45 minutes, perhaps the most notable was that of Nicklas Bendtner - making his first Gunners appearance since August 2011 - with a post when he failed to connect with an early cross, and debutant Isaac Hayden on Youssouf Mulumbu with a sliding challenge that saw the 18-year-old Gunner shown a yellow card which could have been red.
There was no question that Fabianski had produced a save, and a good one, to keep out Mulumbu's strike on the hour mark.
But a minute later Arsenal had the lead as Mulumbu lost possession, Bendtner slipped in Thomas Eisfeld and the 20-year-old German midfielder slotted in what was his first Gunners goal on his maiden start.
West Brom equalised in the 71st minute when Saido Berahino, scorer of a hat-trick in round two, headed past Fabianski from close range.
Baggies substitute Morgan Amalfitano hit the bar in extra time but the teams could not be separated over 120 minutes.
In the shoot-out, West Brom initially had the advantage after Luke Daniels saved Serge Gnabry's effort, but misses from Dawson and Amalfitano allowed Nacho Monreal to seal victory for the away side.
Arsenal, who have now won eight successive matches in all competitions, will host Chelsea in round four.
West Brom created an opening early on as Shane Long got his head to skipper Reid's cross, the ball then striking Carl Jenkinson and diverting to Fabianski.
Five minutes later Bendtner missed Ryo Miyaichi's delivery and collided with the post, continuing afterwards having briefly received treatment.
Albion began to settle and after Graham Dorrans shanked an effort, his corner was headed goalwards by Dawson and Fabianski tipped over the bar.
Hayden might have felt lucky to escape with only a booking for his tackle on Mulumbu in the 33rd minute, and Monreal also received a yellow card moments later for fouling Berahino, with Dorrans then blasting the resulting free-kick wide.
The visitors looked to apply some pressure towards the end of the half and after a Monreal shot was blocked, there was a scare for West Brom when Daniels slipped as a corner came into the box, but they managed to clear the ball.
The Baggies went close just prior to the interval as Reid's free-kick whistled over - via a touch from Fabianski according to the match officials.
Eisfeld looped a shot over the bar shortly into the second half, and Thomas Vermaelen then became the third Arsenal player to receive a yellow card, having felled Long.
Albion stepped things up a gear, with Fabianski then diving to excellently palm behind Mulumbu's drive from the edge of the box.
Moments later, though, Mulumbu gave the ball away and Arsenal took advantage as Eisfeld latched on to Bendtner's through-ball to finish.
Berahino came to West Brom's rescue, nodding in from Long's tee-up to draw things level.
Berahino then flashed a shot over, before Diego Lugano picked up the hosts' first booking for a two-footed challenge on Mertesacker.
Extra-time was confirmed after no Arsenal player was able to get a killer touch on Jenkinson's late delivery into the danger zone - and it was the Gunners who progressed to the fourth round.