Graham Westley guided the Hertfordshire club into the Football League for the first time in their history last season and his men now stand 90 minutes from reaching the third tier of English football.
Having arrived at the Crown Ground with a 2-0 first-leg advantage courtesy of goals from midfielders Stacy Long and Joel Byrom, Stevenage snatched a late winner through substitute Chris Beardsley.
The visitors defended superbly throughout to restrict John Coleman's side - who had left-back Joe Jacobson and midfielder Sean McConville sent off in the final quarter of the match - to few genuine goalscoring opportunities.
And Beardsley fired home from close range in the dying embers of a game marred by a niggling undercurrent both on the pitch and on the touchline.
The hosts, backed by a vociferous home crowd, fashioned a series of half-chances but rarely looked like making them pay.
In the ninth minute Luke Joyce crossed dangerously for Terry Gornell to shoot goalwards and four minutes later, Jimmy Ryan saw a fierce right-footed shot direct from a corner blocked.
Accrington continued to probe and in the 24th minute Ian Craney shot wide when well placed.
A series of late challenges and minor flare-ups marred proceedings but Stevenage began to signal their own threat as the interval approached.
Four minutes before the break Long showed delightful skill to outfox an Accrington defender before firing a left-footed shot wide from 14 yards.
Two minutes later, skipper Mark Roberts hit a volley from distance which flashed just past Alex Cisak's right-hand post.
In the 58th minute, McConville's diving header from close range drew a smart stop from Stevenage stopper Chris Day.
Midfielder Darren Murphy went close twice in quick succession with long-range strikes and in the 69th minute Jacobson was shown a straight red for a heavy challenge on Laurie Wilson.
McConville saw red for violent conduct a minute later after becoming embroiled in a fracas.
And Stevenage went on to claim a winner in the last minute when Beardsley swept home Wilson's low cross with a low right-footed shot.