Sunderland boss Gus Poyet endured an afternoon to forget in his first match in charge, while Steven Gerrard scored a landmark goal as Liverpool drew at Newcastle United.
Manchester City claimed a first away league win of the season after proving too good for West Ham United.
Chelsea striker Samuel Eto'o finally got off the mark in a home win over Cardiff City, Everton edged past Hull City and Stoke City were held at home by West Bromwich Albion.
Arsenal claimed the outright lead in the Premier League as Mesut Ozil scored a second-half brace in a 4-1 win over Norwich at the Emirates Stadium.
Quick passing and clever improvisation brought Arsenal's opener on 18 minutes as Olivier Giroud and Jack Wilshere combined for the England midfielder to side-foot past John Ruddy.
Another excellent move led to Arsenal's second goal on 58 minutes when the leaders broke from outside their own penalty area before Giroud crossed for Ozil to head beyond Ruddy.
Norwich pulled a goal back with 20 minutes to go when Arsenal could only clear a cross to the edge of the box where Jonny Howson collected the ball and drilled a low shot past Wojciech Szczesny.
But any hope of an equaliser was snuffed out seven minutes from time when substitute Aaron Ramsey bamboozled two Norwich defenders before guiding his shot past Ruddy.
Ramsey was involved in Arsenal's fourth goal two minutes from the end when he laid Tomas Rosicky's cross back to Ozil to finish first time.
Manchester United's frustrating start to the season continued after Southampton scored a late equaliser to earn a deserved 1-1 draw at Old Trafford.
United went ahead on 26 minutes when Adnan Januzaj's superb pass sent Wayne Rooney through on goal. Rooney's shot was saved by Artur Boruc, but the ball bounced out to Robin van Persie and he skipped past the stranded Saints keeper before rolling his shot beyond a sliding defender.
Rooney struck the crossbar with a thumping volley moments later and Van Persie headed against the bar in the second half before Southampton snatched a deserved equaliser a minute from time.
A corner was curled in from the left which United failed to deal with and Dejan Lovren got some sort of touch before the ball nicked off Adam Lallana and trickled past David de Gea.
Sergio Aguero scored a goal in each half to hand Manchester City boss Manuel Pellegrini his first away league win following a 3-1 victory at West Ham United.
Argentine striker Aguero opened the scoring on 16 minutes with a cool finish after Alvaro Negredo had cleverly allowed Fernandinho's slide-rule pass to run through the Hammers defence.
Aguero helped himself to a second on 51 minutes when he was left unmarked at the near post to head David Silva's clipped free-kick beyond a flailing Jussi Jaaskelainen.
West Ham were back in the game on 58 minutes when Stewart Downing's knock-down was hooked into the air by Ravel Morrison and Ricardo Vaz Te produced an acrobatic overhead kick.
But, after Negredo had thundered a shot against the bar, City got a third 10 minutes from time when Aguero's cute back-heel teed up Silva to curl a shot past Jaaskelainen.
Cameroon striker Eto'o scored his first Chelsea goal and Eden Hazard notched a brace in a 4-1 home victory over promoted Cardiff.
Cardiff stunned Stamford Bridge after 10 minutes when David Luiz left Ramires' back-pass and Jordon Mutch nipped in to dink the ball over the exposed Petr Cech.
Chelsea equalised in controversial circumstances on 33 minutes when Eto'o kicked the ball away from David Marshall as the Cardiff keeper bounced it.
Hazard fed possession back to Eto'o and, after the former Barcelona forward had dithered six yards out, the ball broke to his Belgian team-mate to score.
Eto'o then finally broke his duck for Chelsea on 66 minutes as he controlled Hazard's pass before firing a low shot past Marshall.
Jose Mourinho's side put the game beyond Cardiff with two goals in four minutes as Oscar hammered a 25-yard effort beyond Marshall before the visiting keeper failed to keep out Hazard's low shot.
Liverpool captain Gerrard scored his 100th Premier League goal as his side twice came from behind to draw 2-2 with 10-man Newcastle United at St James' Park.
Newcastle took the lead midway through the first half when Yohan Cabaye's dipping shot from 35 yards bounced beyond Simon Mignolet's dive.
But Liverpool equalised three minutes before half-time when Gerrard scored from the penalty spot after Newcastle defender Mapou Yanga-Mbiwa was sent off for pulling back Luis Suarez.
Alan Pardew's side regained the lead on 56 minutes when substitute Paul Dummett took advantage of poor defending to side-foot home Cabaye's free-kick at the far post for his first Newcastle goal.
Liverpool hit back for a second time on 72 minutes when Daniel Sturridge headed home Luis Suarez's cross before the Uruguayan clipped the crossbar with a volley three minutes later.
Steven Pienaar made an instant impact as a substitute to hand Everton a 2-1 victory over Hull City at Goodison Park.
Everton enjoyed a flying start by taking the lead on eight minutes when Kevin Mirallas' low shot from outside the area crept inside the post.
Hull responded with the equaliser on the half-hour mark when Sone Aluko pulled the ball back from the right-hand side for Yannick Sagbo to slam home his first goal for the club.
The home side regained the lead on 57 minutes when Pienaar steered the ball home with his first touch from Mirallas' cross into the box.
Poyet suffered a miserable start to his Sunderland tenure as the recalled Phil Bardsley's own goal set Swansea on the way to a 4-0 win at the Liberty Stadium.
A flurry of three goals in seven minutes started on 57 minutes when Jonathan de Guzman's corner from the right was diverted past Sunderland keeper Keiren Westwood by defender Bardsley, who was making his first appearance of the season.
Within a minute, Swansea had taken a 2-0 lead as De Guzman received the ball 25 yards from goal and curled a brilliant shot beyond the reach of Westwood.
And on 64 minutes, Swansea had a third goal to celebrate when Wilfried Bony scored from the penalty spot after Leon Britton had been fouled by Craig Gardner.
Swansea made Poyet's afternoon even worse 10 minutes from time when Chico Flores' header from another De Guzman corner bounced past Westwood off Sunderland striker Steven Fletcher.
Stephane Sessegnon missed golden opportunities in each half as West Brom drew 0-0 with Stoke at the Britannia Stadium.