A header from Jonathan Douglas, which goalkeeper Paul Jones got a hand to but could not keep out, condemned Town to a sixth straight defeat.
The Sussex side, who not long ago were eyeing a late play-off charge, are now just three points above the drop zone although they have two games in hand on most of the teams below them.
Gregory was left bemoaning Jones' error for the goal and said: "He got his hand to it but it slipped past him, but he's a top guy and made some excellent saves to keep us in the game.
"We came here with a game plan and it was working until then. We now need to knuckle down, work hard and not dwell on it. The amount of games we have coming up is criminal, but we need four or five points to be safe."