The Latics have achieved three straight league victories under caretaker boss Tony Philliskirk - as well as drawing with Everton in the FA Cup - but basement side Pompey are now without a win in 22 games.
Portsmouth came under early pressure, with Dan Butler forced into a goalline clearance before goalkeeper Simon Eastwood denied Chris Iwelumo.
Oldham broke through after 11 minutes when top scorer Baxter took possession 25 yards out and drilled a shot past Eastwood for his 12th goal of the campaign.
Both sides wasted chances before half-time as Pompey's Liam Walker shot off target and Oldham went close through Lee Croft and Lee Barnard.
Portsmouth had Sam Sodje sent off in the 50th minute when the defender had an off-the-ball altercation with Barnard and received a straight red card for raising his hands.
Oldham hit the woodwork twice in their search for a second goal - a Jonathan Grounds free-kick struck the crossbar and Jordan Obita's overhead kick came back off a post.
Portsmouth only threatened to equalise when Jed Wallace fired over 12 minutes from time.