After a dismal start to the campaign, Albert Adomah poured more misery on Sean O'Driscoll's beleaguered Rovers by netting on the stroke of half-time.
And, to the horror of the home fans, Hayter then blasted wide from the spot after Kyle Bennett was fouled in the box.
But, after pressing for an equaliser, Doncaster got it as Hayter turned the hero when he rose to head home Milan Lalkovic's cross.
Doncaster headed into the game as the Championship's bottom side, pointless and having lost five of their last six games in all competitions.
The Robins had also endured a disappointing start to their campaign preceding the trip to the Keepmoat, winning only one of their opening four league games.
Rovers restored Mark Wilson and James O'Connor to the substitutes' bench after injury, with former England international David James replacing sidelined Robins goalkeeper Dean Gerken.
City threatened first, Nicky Maynard almost adding to his three-goal early-season haul with a fizzing shot that just missed Gary Woods' left-hand post after three minutes.
The hosts came close to an opener in the 24th minute when Richard Naylor connected with Giles Barnes' inswinging corner, but his header scraped over the crossbar.
Just before the break, Mustapha Dumbuya warmed James' palms with a long-distance drive, before Adomah directed a similar strike wide of Woods' goal.
Hayter might have given Rovers the advantage going into half-time with a right-footed volley, but James was equal to his attempt.
Yet it was the Robins that drew first blood in the 45th minute, Adomah manoeuvring into space in the penalty area and blasting home.
Hayter then spurned the chance to equalise from the penalty spot with 56 minutes on the clock. Cole Skuse brought down the onrushing Bennett, only for Hayter to power his spot-kick wide of the post.
But he was redeemed 10 minutes later.
Chelsea loanee Lalkovic stormed down the right wing to send a pinpoint cross in for Hayter, which he gratefully headed home.
Rovers, buoyed by their equaliser, turned up the heat on Keith Millen's side.
Barnes forced James to clutch at the ball with a shot from distance, with Lalkovic scooped the ball over the crossbar after finding good space on the edge of the box.
Jon Stead nearly summoned up a late blow to the resurgent Rovers in the 78th minute, angling his body to strike but miscuing his shot agonisingly over Woods' crossbar.
Reece Brown had an opportunity to produce a dramatic victory for Doncaster deep into injury time, but he clipped his header over the crossbar before the final whistle.