A drab first half gave little indication of the entertainment which was to follow, with substitute Michu making an instant impact as he fired Swansea into the lead in the 58th minute seconds after joining the fray.
The Gunners were sparked into life after falling behind and levelled through their own super-sub in the shape of Lukas Podolski (81) before Kieran Gibbs' superb volley looked to have won it for Arsene Wenger's men two minutes later.
But Swansea hit back, with the visitors switching off at a corner and allowing striker Graham to conjure a clever far-post finish, having been found by Ki Sung-yueng.
In a first half which was dominated by plenty of passing but a real paucity of chances, Graham had one of the best on 13 minutes when he got away from Per Mertesacker and drilled in a powerful shot from outside the box which Wojciech Szczesny did well to parry.
The best opportunity of the opening 45 minutes came shortly before the interval when former Arsenal man Kyle Bartley climbed above Mertesacker to meet a good free-kick from Jonathan De Guzman only to see his header hit the top of the crossbar.
After such a miserable opening period, the game came to life after the break with Olivier Giroud twice going close and Aaron Ramsey seeing penalty appeals waved away by Howard Webb, with replays showing he had tripped himself up in the box with Dwight Tiendalli in pursuit.
Swansea boss Michael Laudrup had opted to leave Michu on the bench, but turned to his star forward in the 56th minute and was rewarded with an instant strike, with the Spaniard getting away from a static Arsenal backline to calmly finish beyond Szczesny.
The Gunners were in danger of failing to make the fourth round for the first time in 17 years, with Wenger sending on Podolski for his FA Cup debut in the 72nd minute and the Germany international beating Michel Vorm with a low shot on the turn nine minutes from time.
Arsenal were now in the ascendancy and they seized the lead thanks to a sumptuous volley from Gibbs, with the full-back playing a one-two with Giroud before advancing onto the Frenchman's return pass to smash the ball beyond the helpless Vorm.
However, with progress in sight, Arsenal switched off at the back as Kemy Agustien's corner went all the way through to Ki at the far post, with his ball to Graham turned home by the Swansea striker, who has been linked with a move away from the Welsh club in the January transfer window.
There was still time for the Gunners to go close to snatching victory, with Vorm producing a stunning one-handed save to keep out Laurent Koscielny's powerful header from Theo Walcott's corner.