The former Town favourite took charge of his first match since being appointed Robins manager but, despite overseeing an improved display, Feeney's brilliant strike proved the difference in the League Two encounter on Boxing Day.
The winger grabbed only his second goal of the season with a curling shot past Scott Brown on 57 minutes.
Minutes earlier Jeff Goulding had gone close for the visitors, heading a Warren Cummings cross from the right flank against the post.
Barry Hayles wasted a glorious chance to equalise for the home side, skewing a left-foot shot wide in the 71st minute after good work by David Pipe.
Joss Labadie, one of five changes made by Yates, whistled a shot narrowly wide after 80 minutes with the keeper beaten.