Bottom-of-the-table Posh went into the game at London Road bidding for their third consecutive victory at this level for the first time since 1993.
They enjoyed the better of the first half and deserved took the lead 10 minutes into the second half when Charlie Lee, whose only previous goal this season came in the dramatic 4-4 draw at home to Cardiff in December, headed home.
But Rovers, who are now unbeaten in five games and closing in on the top six, levelled 12 minutes later.
Martin Woods, making his 100th league start, saw his cross partially cleared by Craig Morgan and Gareth Roberts thumped home a 20-yard volley for only his third goal of the season.
Rovers then won it at the death - securing their first success at London Road for 20 years - when Oster tapped home Billy Sharp's cross from close range to score for the first time since February 17 2009.
Posh flew out of the traps, with Mark Little firing over from the edge of the box and Scott Griffiths having his volley blocked by Shelton Martis inside eight minutes.
Lee then forced Rovers goalkeeper Neil Sullivan into a save to keep out his 25-yard shot before the visitors finally came to life.
But Dean Shiels, restored to the starting line-up, could not keep his shot from 20 yards on target.
The home side came close to breaking the deadlock in the 23rd minute. A free-kick from Sergio Torres found Craig Mackail-Smith but his header from close range went over.
Rovers were still to find their range, however, as Oster fired another effort wide with Posh then creating two excellent openings.
First Gabriel Zakuani headed a Torres corner wide before the latter was denied by Sullivan from the edge of the box.
Doncaster finished the half strongly and Sharp, with 16 goals this season, finally forced Posh goalkeeper Joe Lewis into action before Woods blasted over from 25 yards.
The visitors continued where they left off with Sam Hird heading a James Coppinger cross wide within a minute of the restart.
Peterborough came close yet again to opening the scoring in the 50th minute but Sullivan pulled off another save to keep out Morgan's header from a Tommy Rowe free-kick.
Moments later Sullivan was called into action again, this time saving from Lee's header.
But Lee was not to be denied shortly afterwards when he finally made the breakthrough when he headed home Rowe's inswinging corner for his second goal of the season.
Mackail-Smith was denied by Sullivan while Rowe and Exodus Geohaghon failed to hit the target before Roberts' stunner.
Gabriel Zakuani (Posh) and Jay Emmanuel-Thomas and Woods (Doncaster) came close to late winners before Oster had the final word.