Crystal Palace are still without a point since winning promotion after throwing away the lead at Stoke City and Roberto Martinez's first home match as Everton boss ended in a draw with West Bromwich Albion.
North East duo Newcastle United and Sunderland picked up their first points of the season following draws with West Ham United and Southampton, respectively.
Liverpool made it back-to-back 1-0 wins to start the season after Daniel Sturridge's first-half goal secured the points at Aston Villa.
Sturridge struck on 21 minutes when he finished from close range after rounding Villa keeper Brad Guzan after Philippe Coutinho had cleverly left Jose Enrique's pass into the box.
Villa created chances to equalise and came close to claiming a point four minutes from time when Christian Benteke's powerful volley was superbly saved by Simon Mignolet.
Lukas Podolski scored a goal in each half as Arsenal made light of conditions at a rain-soaked Craven Cottage to claim a 3-1 derby victory over Fulham.
Arsenal took the lead on 14 minutes when a hopeful shot from Aaron Ramsey ricocheted kindly into the path of Olivier Giroud and he showed a cool head to clip the ball over David Stockdale and into the bottom corner.
The visitors doubled their lead on 41 minutes when Theo Walcott's cross-shot was beaten into the path of Podolski 18 yards out and the German forward made no mistake as he caressed a powerful drive into the roof of the net.
The third goal arrived on 68 minutes when Santi Cazorla cut the ball back for Podolski to take one touch to steady himself and crash the ball through a maze of legs into the corner of the net.
With 13 minutes remaining, Fulham substitute Darren Bent marked his debut by bundling home from close range after Wojciech Szczesny failed to get a strong enough hand on a Dimitar Berbatov shot from a tight angle.
Robbie Brady's first-half penalty earned Hull a 1-0 win over Norwich at the KC Stadium as the visitors failed to take advantage of playing against 10 men for over an hour.
The home side scored the decisive goal on 22 minutes when former Hull defender Michael Turner was adjudged to have fouled Yannick Sagbo and Brady confidently dispatched the penalty.
But five minutes later Sagbo was sent off after the summer signing appeared to butt Russell Martin following a tangle after a Norwich corner, yet Chris Hughton's side rarely threatened to make their hosts pay.
Manchester United targets Leighton Baines and Marouane Fellaini both started for Everton in a 0-0 draw with West Brom at Goodison Park.
Everton were the better team for long periods as Ben Foster made several smart saves before the West Brom keeper was forced off through injury 12 minutes from time.
The hosts came closest to pinching the points late on when Fellaini chested down Phil Jagielka's diagonal ball into the box, but his flicked shot came back off the post.
Mark Hughes enjoyed his first win as Stoke City manager as his side came from behind to win 2-1 against Crystal Palace at the Britannia Stadium.
Palace were in front on 31 minutes when Ryan Shawcross failed to deal with a long ball and Marouane Chamakh took advantage to drill a low shot past Asmir Begovic.
Stoke almost equalised immediately when Peter Crouch was denied by the crossbar, but the towering striker was involved when the home side did draw level on 58 minutes.
It was Palace's turn to make a hash of coping with a bouncing ball as Crouch helped it on in the penalty area to Jon Walters, who fed Charlie Adam to steer his shot past Julian Speroni.
And four minutes later Stoke were in front after Palace failed to clear a long throw-in. Robert Huth showed the greatest desire to win a loose ball ahead of Chamakh and it broke for Shawcross to swivel and score the winner.
Jose Fonte headed home a late equaliser as Southampton denied Sunderland a first win of the season following a 1-1 draw at St Mary's.
Sunderland enjoyed a stunning start and were in front after just three minutes when Emanuele Giaccherini was left unmarked six yards out to head home Sebastian Larsson's corner.
Southampton thought they had equalised two minutes later when Jay Rodriguez finished clinically from Rickie Lambert's pass, but the striker was flagged offside.
The Saints' second-half pressure brought an equaliser two minutes from time when James Ward-Prowse curled a free-kick into the box and Fonte powered a header inside the upright.
Sam Allardyce maintained his record of never losing at St James' Park since being sacked by Newcastle United after West Ham United secured a 0-0 draw on Tyneside.
Kevin Nolan missed a good chance to score against his former club in the second half and Hammers striker Modibo Maiga had a late goal rightly disallowed for offside.
Newcastle almost nicked it in fortunate circumstances when Sammy Ameobi's cross from the left flew over Jussi Jaaskelainen and bounced off the post.