Philliskirk restored the Latics' lead with 13 minutes remaining after a James Tarkowski own goal early in the second half had brought the Robins level, cancelling out Carl Winchester's first-half opener for the hosts.
Oldham went close midway through the first half when Winchester's excellent cross was headed wide by Kirk Millar.
And it was full-back Winchester who gave Lee Johnson's team a spectacular lead after 32 minutes, winning possession on the halfway line before firing a shot into the top corner of the net from 25 yards out.
Swindon needed a huge slice of luck to equalise in the 56th minute. Oldham defender Tarkowski tried to block Nathan Thompson's cross inside the box and could only send a looping deflection into his own net.
Oldham continued to create chances with substitute Jonson Clarke-Harris sending an effort inches wide and Winchester being denied by visiting goalkeeper Wes Foderingham.
The winner came after 77 minutes as Jonathan Grounds crossed to the far post and Philliskirk planted home a diving header from eight yards.
Swindon almost snatched a point in stoppage time but Dany N'Guessan's effort was blocked on the line.