Swindon got off to the perfect start in new manager Kevin MacDonald's first home game in charge when striker Andy Williams opened the scoring after just 11 minutes.
Williams' shot looped off Walsall captain Andy Butler as the defender tried to block, wrong-footing goalkeeper Aaron McCarey.
Walsall had won eight of their previous 11 games to propel themselves towards the play-off places.
And the Saddlers were gifted the chance to level after Febian Brandy was tripped by Lee Miller, Will Grigg coolly slotting home the resulting spot-kick for his 17th goal of the season.
Swindon retook the lead 11 minutes from time when Adam Rooney's shot was spilled by McCarey, allowing Simon Ferry to tap in the rebound.
At that stage it looked as if MacDonald would maintain his 100% winning record since replacing Paolo Di Canio.
But Paterson rescued a point in the dying seconds by bundling an Ashley Hemmings cross past Swindon goalkeeper Wes Foderingham at the far post.
