Kay crashed a shot off the woodwork in the 24th minute after a Roberts corner was cleared to the edge of the box.
Roberts then hit the bar in the second half with an 18-yard volley, with Jordan Rhodes forcing County keeper Owain Fon Williams into a good save when the ball was lofted back into the box by Theo Robinson.
But that was as good as it got for a home side whose usual attacking style was stifled by a disciplined Stockport defence expertly marshalled by on-loan Leeds defender Paul Huntington.
Stockport's Nicholas Bignall raced on to a punt over the top of Town's defence in only the second minute but blazed over.
County's David Poole finally forced Alex Smithies into a save in stoppage time while Huddersfield had a flurry of half chances but were unable to unlock Stockport's defence.