The decisive moment arrived after just 21 minutes, when Boyd kept his cool on the edge of the six-yard box to convert Alexei Eremenko’s cross at the second attempt – after keeper Chris Dilo had blocked his initial effort.
However, the hosts had to survive the final four minutes with 10 men after James Fowler was shown a second yellow card.
In an end-to-end contest both sides created chances, with Vitalijs Maksimenko going close on a number of occasions for the hosts, as did Lee Ashcroft.
At the other end, Marc McAusland missed a close range header after half-an hour while Kenny McLean fired inches wide from 35 yards after the break.
The same player had another decent effort on goal, while Gregg Wylde, Steven Thompson and Josh Magennis all missed opportunities to force an equaliser.
The result leaves Kilmarnock needing one point from their final match of the season on Saturday – against Hibernian at Easter Road – to spare themselves the tension of playing in the relegation play-off.
The win also ended Kilmarnock’s run of four straight losses.