United - thumped 4-1 by rivals Manchester City last weekend - had hit the crossbar through a first-half header from Anderson, but Old Trafford was left in stunned silence when Morgan Amalfitano scored a superb solo effort after 54 minutes.
The on-loan Marseille midfielder collected the ball on the half-way line, surged past both Michael Carrick and Rio Ferdinand before coolly chipping goalkeeper David de Gea as the travelling Baggies fans dreamed of a first away win over Manchester United since 1978.
A curling free-kick from Wayne Rooney which floated through the penalty area and into the far corner three minutes later levelled things up, only for the visitors to again go on the offensive as Jonas Olsson's header hit the crossbar.
De Gea saved a 25-yard free-kick from Amalfitano, before he was beaten again when England Under-21 international Saido Berahino converted the midfielder's through ball in the 67th minute.
Substitute Marouane Fellaini's conversion from Nani's cross was correctly ruled out for offside and Boaz Myhill saved from Carrick as United suffered a shock home defeat.
Manchester City also came unstuck as they went down 3-2 at Aston Villa following a second-half rally by Paul Lambert's men.
Yaya Toure slotted home following a corner just before half-time to put City ahead.
Villa - thrashed 4-0 at home by Spurs in the Capital One Cup on Tuesday night - levelled through Karim El Ahmadi on 51 minutes, despite replays suggesting the Morocco midfielder was offside.
City, though, were quick to respond and lead again when Edin Dzeko bundled in a corner five minutes later.
However, Villa were level when Leandro Bacuna clipped in a 20-yard free-kick in the 73rd minute before going ahead seconds later with another quick goal as Andreas Weimann got on the end of a long ball to beat Joe Hart and complete a miserable afternoon for the Manchester clubs.
In the lunchtime kick-off, Chelsea fought back to deny Tottenham the chance to go clear at the top of the Premier League as John Terry's header earned the visitors a 1-1 draw at White Hart Lane, where Fernando Torres was controversially shown a red card.
Rival managers Jose Mourinho and Andre Villas-Boas exchanged the weakest of handshakes before the game.
Tottenham dominated the first half and took the lead through Gylfi Sigurdsson, but Terry - one of the personalities with whom Villas-Boas clashed when in charge at Stamford Bridge - powered in a header on 65 minutes set up what looked to be a thrilling finish, which was overshadowed somewhat when, with nine minutes left, referee Mike Dean showed the lively Torres a second yellow card for jumping into Jan Vertonghen with his elbow when replays showed there was no contact.
Meanwhile, Cardiff heaped more pressure on Fulham manager Martin Jol as a stoppage-time strike from Jordon Mutch earned the Welshmen a 2-1 win at Craven Cottage.
Steven Caulker headed the visitors into a 12th minute lead, but Fulham levelled through a 25-yard effort from substitute Bryan Ruiz on the stroke of half-time.
Just when the match seemed to be heading for a draw, Mutch cracked in a superb winner from 25 yards as the boos rang out on the banks of the Thames following a fourth league defeat in the opening six matches.
Hull beat West Ham 1-0 at the KC Stadium.
Robbie Brady converted a penalty on 12 minutes after what looked minimal contact from Irons defender Joey O'Brien.
Both sides hit the woodwork in what was an open game, but West Ham manager Sam Allardyce was again left fuming when two claims for handball were waived away by referee Kevin Friend in the second half.
The Eagles, with just one league win so far, felt they should have been given a penalty in the first half when Maroune Chamakh went clear through and tumbled under a challenge from goalkeeper Artur Boruc - only to be shown a yellow card for diving.
Saints, who won at Anfield last weekend, took hold of the match with two goals in as many minutes following the restart.
Club-record signing Dani Osvaldo opened his account two minutes into the second half when he lashed in Adam Lallana's pass, before England forward Rickie Lambert crashed in a 25-yard free-kick off the woodwork as as Mauricio Pochettino's well-organised outfit moved up into the top four.
Very funny. There is much rejoicing. Good news for the world. Dogs are happy. Cats have cream. Birds with worms. Early days. Beer in fridge. You are unhappy. I am happy. It's a good life.- popularpete