Baxter grabbed his first moments after James Tarkowski had cancelled out Darius Charles' first-half opener for Stevenage with Christian Montano soon making it 3-1.
Boro fought back though and were level thanks to efforts from Greg Tansey and Filipe Morais, however Baxter slotted in the winner from the spot four minutes from time.
After a pulsating start, it was Stevenage who broke the deadlock in the 44th minute when Charles pounced on a loose ball after Oumare Tounkara hit the post.
Tarkowski saw a header ruled out for an apparent push on keeper Chris Day but was not to be denied five minutes after the restart as he crashed home a sweet volley.
Baxter was the provider with a free-kick before netting himself on the rebound after Day saved his spot-kick in the 66th minute and then Montano put Oldham 3-1 moments later.
Tansey gave Stevenage hope by making it 3-2 in the 75th minute and Morais equalised seven minutes from time but Baxter's successful spot-kick sealed the win for Oldham.