Adrian Mariappa had headed the Hornets in front but two quick goals from David Nugent turned the Championship game in Leicester's favour.
Sean Murray's deflected free-kick hauled Watford level, and although the visitors had dominated the second half, Forsyth's strike stole the points.
Watford were forced to make one change from the side which drew with Nottingham Forest on Saturday as Troy Deeney missed out with a hamstring strain. Chris Iwelumo came in.
Leicester named an unchanged line-up following an impressive win against Cardiff.
The Hornets took the lead from the first attack of the game. Murray sent over a free-kick from the left
and defender Mariappa - a transfer target for a host of Premier League clubs in January - powered a header into the roof of the net.
Joe Garner had the chance to make it two but his effort was well saved by Kasper Schmeichel and Leicester levelled from the breakaway.
Jermaine Beckford played the ball out to Nugent on the left and he cut inside before firing into the bottom corner.
The same combination put the Foxes in front in the 18th minute. Beckford laid the ball off to his strike partner and Nugent's blistering drive gave Scott Loach no chance.
The frantic pace continued as Watford equalised after 33 minutes. Matt Mills brought down Alex Kacaniklic just outside the area and youngster Murray stepped up to score his first senior goal, albeit with the aid of a deflection which deceived Schmeichel.
Hornets defender Lee Hodson brought a decent save from the keeper and Murray was inches away from his second before the break.
Leicester defender Wes Morgan should have done better with a free header at the start of the second half and Beckford spurned a golden chance to restore the Foxes lead as he somehow headed Sean St Ledger's cross wide from four yards.
The visitors were on top as Beckford did well to turn and get a shot in. Loach spilled his shot but Nugent was not quick enough to react and the danger was cleared.
Watford's response came from Jonathan Hogg but his shot from a corner cleared the crossbar, while at the other end Paul Gallagher did well to get in a shot from a tight angle but he was denied by Loach.
John Eustace brought a fine save from Schmeichel with a fierce drive from the edge of the area in the 78th minute and shortly afterwards they were ahead.
Eustace picked out substitute Forsyth with a raking pass and he rifled the ball past the goalkeeper.
Leicester's best chance of an equaliser fell to Richie Wellens but he blazed over after good work from Gallagher.
Watford full-back Lloyd Doyley was sent off in injury time for bringing down Beckford as he bore down on goal.