Belford, 19, saved from Luke Freeman and Filipe Morais while Ben Chorley blazed over the crossbar as Town triumphed 3-1 in the shootout.
Stevenage striker Francois Zoko had earlier taken the game to spot-kicks with a dramatic 70th-minute equaliser as the tie finished 1-1 in normal time.
Zoko headed home at the near post to cancel out Nicky Ajose's first-half opener.
Ajose scored his ninth goal of the season to fire Swindon into a 23rd-minute lead.
Stevenage should have had a penalty when Zoko was tripped by Yaser Kasim but the visitors' appeals were waved away.
Alex Pritchard, Mohammed El Gabnbas and Kasim all scored for the hosts in the shootout while Zoko was on target for Graham Westley's side.
Swindon will now play Peterborough over two legs for a place in the final at Wembley against either Fleetwood or Chesterfield, who will contest the northern section final.