City took a first-half lead on Sunday through Natasha Flint but Katie Chapman’s second-half header, followed by a late goal from substitute Jackie Groenen, earned Emma Hayes’ side all three points.
England striker Toni Duggan had the game’s first clear opportunity early on when Chelsea failed to clear a corner but her shot was saved magnificently by Marie Hourihan.
But Hourihan could do nothing when Flint dispossessed Drew Spence just shy of the half-hour mark before looking up and lobbing the Chelsea goalkeeper.
Houghton then picked up what seemed to be a serious injury just before half-time and although she did not need a stretcher, her participation in England’s forthcoming matches against Sweden and Wales looks in jeopardy.
Chelsea drew level in the 62nd minute when a pinpoint cross from Rachel Williams was met by Chapman’s diving header into the left corner.
And substitute Groenen also scored when she broke away towards goal in the 90th minute, taking the ball round Bardsley before placing it into an empty net.
Birmingham City maintained their lead at the top of the FA WSL after Jo Potter’s second-half strike condemned Notts County to their second defeat in a row.
The home side started well with Jess Clarke curling inches wide after nine minutes. And the Notts County forward was in the thick of it 10 minutes later when she almost pressured Aoife Mannion into diverting into her own net.
However, the league leaders needed only three minutes after the restart to take the lead when Potter, who has just been called up to the England squad, scored direct from a corner.
County tried desperately to get back on level terms but failed to cause Rebecca Spencer any serious problems in the Birmingham goal.
Elsewhere, Everton were made to pay for not converting a late penalty after Alex Scott’s late winner gave Arsenal a 2-1 win on Merseyside.
Danielle Carter gave Arsenal an early lead before Nikita Parris equalised for Everton in first half stoppage time.
Emma Byrne then produced an excellent penalty save to deny Everton’s Michelle Hinnigan after Japanese midfielder Yukari Kinga handled on 64 minutes.
And Hinnigan’s inability to convert from 12 yards proved costly as right-back Scott popped up six minutes before time to tap in a winner giving Arsenal all three points.