Ryan Dickson lashed home the winning spot-kick after goalkeeper Seb Brown had saved from Andre McCollin and Aaron Brown had blazed his effort over the bar.
It took a late penalty from Gavin Tomlin to take the game to spot-kicks in the first place as the visitors fought back from 2-0 down.
Glenn Poole gave the hosts the lead, also from the spot, after Charlie MacDonald had been fouled and Kevin O'Connor doubled the lead when he finished off a fine move.
Marc Bircham pulled one back before half-time and it took a series of fine saves by Josh Wagenaar to keep Yeovil in it. The best was from Ross Montague while Moses Ademola lashed an effort across the face of goal and Mark Phillips headed narrowly wide.
Tomlin almost won it at the death for the visitors but mis-kicked in front of goal and was made to pay the price when his team-mates missed their penalties.