Chelsea drew at Tottenham but finished with 10 men as Fernando Torres was sent off, and Southampton joined the two London clubs in the top four by beating Crystal Palace.
Hull City held firm to take all three points at home to West Ham, while Jordon Mutch hit a later screamer to give Cardiff victory at Fulham.
David Moyes suffered his first competitive home defeat as Manchester United manager as West Brom won 2-1 at Old Trafford.
In the first half, Anderson's header was denied by the crossbar but nine minutes after the break, Morgan Amalfitano coolly lifted the ball over David De Gea after a surging run to give Albion the lead - although United soon levelled as Wayne Rooney's free-kick came through a crowd of players.
Early on, West Brom had to replace midfielder Scott Sinclair because of injury, with Berahino coming on. The England Under-21 striker rifled a shot through Rio Ferdinand's legs on 67 minutes and that proved to be the winner as Albion hit the top half of the table, while United slipped to 12th.
Aston Villa came from behind to beat Manchester City 3-2 at Villa Park.
Yaya Toure saw a header cleared off the line in the first half, but was not to be denied when he slotted home the opener following a corner just before the break.
Karim El Ahmadi looked just offside when he equalised on 51 minutes, but Edin Dzeko restored City's advantage within five minutes.
Back came Villa as Leandro Bacuna curled home a sweet free-kick and two minutes later, City's defence was nowhere to be seen as Andreas Weimann ran onto Libor Kozak's flick-on and prodded the ball past Joe Hart to secure all three points. Villa are ninth in the table, while City are five points off the pace in fifth.
Arsenal started the day top of the table and ended in pole position again as they ground out a 2-1 win at Swansea.
After a sterile first half at the Liberty Stadium, Serge Gnabry put the Gunners in front on 58 minutes and man-of-the-match Aaron Ramsey doubled their advantage four minutes later.
Ben Davies brought the Swans back into it, but Arsene Wenger's men held on to make it five straight victories since their opening-day defeat to Villa.
Chelsea fought back to deny Tottenham as John Terry's header earned a 1-1 draw at White Hart Lane, where Fernando Torres was controversially sent off.
Rival managers Jose Mourinho and Andre Villas-Boas exchanged the weakest of handshakes before the game, from which both will feel their side did enough to take all three points.
Tottenham dominated the first half and took the lead on 19 minutes through Gylfi Sigurdsson, but Terry powered in a header on 65 minutes to set up a thrilling finish.
Torres was lucky to escape with only a booking shortly after the break, when he appeared to scratch Jan Vertonghen's face. With nine minutes left, referee Mike Dean showed the lively Spaniard a second yellow card for jumping into Vertonghen with his elbow when replays showed there was little, if any, contact.
Spurs are two points behind Arsenal after six games, with Chelsea a further two points back in third.
Southampton are up to fourth after recording a 2-0 victory at home to struggling Crystal Palace.
Palace, who remain 19th, felt they should have been given a penalty on the half hour when Maroune Chamakh went clear through and tumbled under a challenge from goalkeeper Artur Boruc - only to be shown a yellow card for diving.
The second half was barely two minutes old when Dani Osvaldo held off two Eagles defenders to score his first Saints goal and Rickie Lambert followed that up almost immediately with a 25-yard free-kick.
Hull City beat West Ham 1-0 at the KC Stadium as Robbie Brady converted a penalty on 12 minutes after what looked minimal contact from Irons defender Joey O'Brien.
James Tomkins saw a header cleared off the line, while Curtis Davies' header was hacked away by Modibo Maiga onto the woodwork. The Tigers are seventh in the standings, with the Hammers a point above the drop zone in 16th.
Cardiff heaped more pressure on Fulham manager Martin Jol by winning 2-1 at Craven Cottage.
Steven Caulker headed the visitors into the lead from a Peter Whittingham corner on 12 minutes.
However, Jol's men were level when Bryan Ruiz, who replaced Alex Kacaniklic in the 41st minute, curled in Sascha Riether's pass from 25 yards.
But Mutch hit a screamer in the dying minutes to secure a priceless away victory for the Welsh club.