Everton edged out Chelsea in a feisty game at Goodison Park, Newcastle United got the better of Aston Villa, while Mark Hughes picked up a point against his former club as Stoke City drew with Manchester City.
West Bromwich Albion finally scored their first goal of the season to earn a point at Fulham, while promoted duo Hull City and Cardiff City drew at the KC Stadium.
Olivier Giroud and Aaron Ramsey continued their impressive starts to the season as Arsenal beat Sunderland 3-1 at the Stadium of Light as Ozil caught the eye on his debut.
Ozil had an early impact when he created the opening goal on 11 minutes as he beautifully pulled Kieran Gibbs' long ball under control before rolling a pass across the edge of the penalty area for Olivier Giroud to finish low inside the near post.
Modibo Diakite hit the top of the crossbar for Sunderland before the home side equalised three minutes after half-time when Craig Gardner scored from the penalty spot after Adam Johnson had tumbled under a challenge from Laurent Koscielny.
Sunderland had a goal disallowed correctly for offside before Arsenal regained the lead on 67 minutes when Carl Jenkinson pulled a cross back to the edge of the area and Aaron Ramsey struck a superb first-time shot past Keiren Westwood.
The hosts thought they had equalised again when Jozy Altidore's shot just crept over the line, but referee Martin Atkinson had already stopped played for a foul on the American striker, much to Sunderland's anger.
Ramsey scored his second of the game nine minutes later when he rolled the ball under Westwood after being played in by Giroud following a patient build-up from Arsenal.
Jose Mourinho suffered his first Premier League defeat since returning as Chelsea manager after Steven Naismith's goal handed Roberto Martinez his first victory as Everton boss.
Samuel Eto'o was close to marking his Chelsea debut with a goal, but was denied by a superb block from Gareth Barry, while Tim Howard produced a flying save to thwart Ramires.
The only goal arrived in first-half stoppage-time when Leon Osman's cross was headed back across goal by Nikica Jelavic for Naismith to nod in from close range.
Both teams had chances in the second half, with Andre Schurrle dinking just wide for Chelsea and Leighton Baines striking the angle of post and bar for Everton late on.
Robin van Persie and Wayne Rooney fired Manchester United to an uninspiring 2-0 victory over 10-man Crystal Palace at Old Trafford.
Referee Jon Moss, who had waved away three previous penalty appeals and booked Ashley Young for diving, dismissed Kagisho Dikgacoi for bringing down the England man on the edge of the box in the 44th minute.
Van Persie stepped up to convert from the spot but it was not until the 81st minute that Rooney, who started the match sporting a headband to protect the gash which kept him out of England's World Cup qualifiers, curled home an excellent free-kick.
Gylfi Sigurdsson scored a goal in each half as Tottenham produced a controlled performance to beat Norwich 2-0 at White Hart Lane.
Christian Eriksen and Roberto Soldado were heavily involved for Spurs, with the Spain international's impudent back-heel coming back off the post after the Dane's run into the box.
Those two players combined to create Spurs' opening goal on 28 minutes as Soldado laid the ball off for Eriksen and his perfect pass played in Sigurdsson to finish.
Spurs increased their lead three minutes into the second half when Paulinho's low cross from the right wasn't cut out by Norwich keeper John Ruddy and Sigurdsson finished from close range at the far post.
Manchester City again looked far from fluent away from home as Stoke created the better chances in a largely forgettable 0-0 draw at the Britannia Stadium.
Aston Villa suffered a third defeat in a row after Hatem Ben Arfa was the inspiration again for a 2-1 win in the Midlands.
Loic Remy, making his first start for Newcastle, set up the opening goal on 18 minutes when he drove into the box and his low cross deflected into the path of Ben Arfa to sweep home from six yards out.
Villa equalised midway through the second half when the in-form Christian Benteke rose powerfully to head home a corner to take advantage of poor goalkeeping from Tim Krul.
But Newcastle were back in front six minutes later when substitute Yoan Gouffran side-footed home after Brad Guzan parried Ben Arfa's well-struck shot.
Gareth McAuley left it late to score West Brom's first goal of the season as his side avoided defeat at Fulham for only the second time at Craven Cottage in the Premier League after a 1-1 draw.
Fulham took a 22nd-minute lead when a corner dropped to Steve Sidwell about 12 yards out and he drilled a volley beyond Boaz Myhill.
The Cottagers thought they had scored a second in the final 10 minutes after Bryan Ruiz's shot came back off the crossbar and then Hugo Rodallega volleyed home, but Sidwell had been offside before he set up the Colombian.
And in the final minute West Brom delivered the sucker punch as Chris Brunt chipped a corner towards the far post where McAuley steamed in to head past David Stockdale.
Last season's top two in the Championship played out a 1-1 draw as Cardiff City came from behind to take a point at Hull City.
The Tigers went ahead five minutes before half-time when Tom Huddlestone's whipped cross was watched by a static Cardiff defence and Curtis Davies stole in at the far post to head past Joe Lewis.
Cardiff levelled just before the hour mark following a well-constructed team move which ended with Don Cowie crossing from the right and Peter Whittingham timing his run to finish first time left footed.