Posh led 4-2 with less than half an hour remaining, but were left stunned by Genseric Kusunga's injury-time winner.
The visitors led after five minutes when Mark Little's right-wing cross was headed in from six yards by Britt Assombalonga.
Peterborough struck again in the 25th minute, with Tommy Rowe's near-post header following Lee Tomlin's corner.
Tomlin made it 3-0 to the visitors on the stroke of half-time as he pounced on a loose ball and swept home from 14 yards.
Oldham gained hope with goals in the 48th and 54th minutes, Gary Harkins forcing in from close range before David Worrall found the top corner from 16 yards.
Peterborough were gifted their fourth in the 63rd minute when Jonathan Grounds' error was punished by Nicky Ajose's easy finish.
Oldham put the match back on a knife-edge five minutes later through James Wesolowski's scrambled effort.
The home side then levelled in the 89th minute when Tomlin handled James Tarkowski's header on the goal-line - Tomlin was shown a straight red card - and Harkins converted the subsequent penalty.
Oldham's amazing fightback was complete in the fourth minute of time added on when Kusunga blasted home from two yards.