The Fifers were in charge from the off and Scott McBride, Paul McManus and Collin Samuel were all denied.
The visitors were eventually rewarded for their dominance in the 44th minute when David Muir headed home Craig Johnstone's cross from the right byline.
The second half was a different story, though, as the Borderers chased a leveller and Steven Notman headed over a good chance from a corner.
Darren Lavery also shot wide, before he made the vital breakthrough when he converted Scott Dalziel's centre from close range with 18 minutes remaining.
Neither side could force a winner, leaving the contest evenly poised ahead of Saturday's second leg.