Arsene Wenger's side dominated for much of the game and created several clear-cut chances, only to be denied by fine saves from Stoke's stubborn goalkeeper.
However, the Bosnian was left helpless when Lukas Podolski's free-kick 12 minutes from time deflected off Geoff Cameron and into the back of the net.
The result sees Arsenal close to within a point of fourth-placed Tottenham Hotspur, who play on Sunday, while Stoke remain tenth.
Arsenal made a high-tempo start, with Theo Walcott's pace proving an early thorn in Stoke's side and Alex Oxlade-Chamberlain trying his luck with a shot that flashed wide.
The visitors took 20 minutes to get into the game and eventually had half-chances of their own through Peter Crouch, Ryan Shawcross and Glenn Whelan.
But Arsenal soon returned to the ascendancy and would have led at half-time had it not been for a trio of excellent saves from Begovic.
First, the Bosnian stopped a close-range snap-shot from Oxlade-Chamberlain, before palming a point-blank Laurent Koscielny header to safety.
However, his best stop came five minutes from the break, when Jack Wilshere released Oxlade-Chamberlain, who opened up his body and sent a right-foot shot towards the far corner, only for Begovic to divert it wide with a strong hand.
The break came at the wrong time for Arsenal and they struggled to regain their momentum for the first 20 minutes of the second period.
However, the introduction of the rested Santi Cazorla and Podolski from the substitutes' bench gave them fresh impetus and they soon set up camp in and around Stoke's box.
It looked like the breakthrough would not come, but after Andy Wilkinson brought down Walcott on the edge of the box, Podolski produced the winner courtesy of a helping hand from Cameron.