Victory came as a result of a much-improved second-half display from the Latics as they won their second league match in row to leapfrog Colchester in the table.
Stevenage went ahead in the ninth minute when James Dunne's first-time effort from the edge of the box nestled in the bottom corner after Luke Freeman's shot was blocked.
Boro nearly doubled their lead four minutes later when captain Ben Chorley met Freeman's corner with a firm header, only to be denied by Oldham goalkeeper Alex Cisak.
The visitors drew level 57 seconds into the second half when Matt Smith headed down a long ball into the path of Dean Furman and the skipper found the net from the edge of the box.
And the Latics grabbed what could prove a vital win on the hour as Tarkowski stole in to power in a diving header from Jonathan Grounds' corner.
