Calvin Zola cancelled out Adam Virgo's first-half opener with five minutes to go, but Forster steamed through to clinch the three points for the visitors in the 90th minute.
Albion took the lead in the 35th minute when Virgo outjumped team-mate Glenn Murray to loop Kevin McLeod's cross over Steve Collis.
Brighton continued to dominate after the break, with Andy Whing curling a shot wide, before Steve Thomson had Collis at full stretch to tip away a low drive.
Colin Hawkins beat Collis in the 63rd minute, but his header crashed back off the upright and minutes later James Bailey was on hand to clear Murray's header off the line.
But Ben Rix's perfect pass found Zola, who kept his cool to level with five minutes to go, before Forster restored the Gulls' lead after latching onto a long ball.