The match came only three days after the Gunners' crushing 4-0 Champions League defeat against AC Milan and a week after their last-gasp Premier League win at the Stadium of Light.
Again it was Sunderland who took the lead through Kieran Richardson's volley five minutes before half-time but this time there was no way back for Arsene Wenger's side with the result sealed by an Alex Oxlade-Chamberlain own goal in the 78th minute.
Wenger threw Gervinho straight back into the team on his return from the African Nations Cup while Oxlade-Chamberlain and Lukasz Fabianski were also among five changes from the Milan drubbing.
What Wenger did not need after losing Laurent Koscielny and Per Mertesacker in the past week was more defensive injuries but he was forced into a change after less than 10 minutes when Francis Coquelin appeared to suffer a hamstring problem and Sebastien Squillaci came on.
The Gunners had already gone close through Mikel Arteta's free-kick that curled narrowly wide, while Sunderland forward Stephane Sessegnon also drilled a shot just off target.
Last weekend's loss, inflicted by Thierry Henry's injury-time goal, was Martin O'Neill's first at home, and the Black Cats boss had goalkeeper Simon Mignolet to thank for a smart stop to deny Gervinho in the 28th minute.
Arsenal wanted a penalty when Robin van Persie went down under a challenge from John O'Shea, and the striker certainly had a case with O'Shea getting no more than the faintest of touches on the ball.
While the Gunners looked dangerous in attack, their defence again was not standing up very well to pressure and Johan Djourou became the first man in the book when he hauled down Craig Gardner after a poor touch had let the midfielder in.
Sebastian Larsson's free-kick was headed away by Thomas Vermaelen but only as far as Richardson, whose well-struck volley arrowed into the corner of the net with the aid of a slight deflection off Squillaci.
And it almost got worse for Arsenal before half-time as Jack Colback did well to keep in a deep cross from Larsson but James McClean could only poke his effort wide from close range under pressure from Fabianski.
Only the crudest of challenges from Bacary Sagna ended a break from McClean in the opening moments of the second half and the full-back was swiftly shown a yellow card.
Larsson has been deadly with free-kicks from just outside the box this season but this time his shot was just too high.
Eight minutes after the break Wenger made a double change, substitute Squillaci also appearing to pick up an injury and being replaced by Tomas Rosicky, while Theo Walcott came on for Ramsey.
Alex Song dropped back into the centre of defence and Walcott took up a forward position but it was Sunderland doing almost all the attacking.
Having survived that spell, the Gunners responded with a good bit of possession and Arteta won a free-kick 25 yards out for a foul by Larsson but Van Persie's effort was deflected behind and the corner came to nothing.
Sunderland's workrate really was phenomenal as they hassled and harried their opponents all over the pitch.
Vermaelen headed a free-kick from Arteta straight into the hands of Mignolet but the tie was decided in the 78th minute when Sunderland got their second.
Sessegnon created the goal with a run from inside his own half to the edge of the Arsenal area after more great Black Cats pressure had set up the break.
The Benin forward played in Larsson on the right and his shot rebounded off the inside of the near post onto to Oxlade-Chamberlain, who got his feet in a muddle and could only help the ball over the line.
Van Persie, who had been quiet all afternoon, stabbed a shot over the bar as Arsenal desperately sought a way back in but it was emphatically Sunderland's day.