Three points looked to be going to Saints after Billy Sharp poked home late on, but the game changed in the final minute of added time when the Pompey midfielder lashed home.
Southampton originally took the lead midway through the first half through Sharp but Chris Maguire's rasping effort levelled for Portsmouth.
Jamie Ashdown's fantastic performance in the Pompey goal seemed to secure the visitors a point until Sharp put Southampton back ahead in the 89th minute.
However, Norris struck deep into stoppage time to break home fans' hearts - even though they return to the top of the Championship on goal difference.
St Mary's was rocking as the match began and the intensity translated onto the pitch, with referee Neil Swarbrick issuing the game's first yellow card after 69 seconds.
Morgan Schneiderlin was booked for hauling back David Norris and was soon forced off the field through injury after a seventh minute collision with Luke Varney.
Jack Cork came on in his place but not until after Maguire had picked up the second booking of a frantic opening.
Both sides were struggling to get a foothold, but Pompey came close to breaking the deadlock when Kelvin Davis was forced to parry a Kelvin Etuhu drive following a defensive mishap.
It was Saints, though, that were looking the more assured side and, after Danny Fox saw a long-range effort go close and Ricardo Rocha was booked, they broke the deadlock.
Pompey failed to clear a corner and Sharp collected a pass from Rickie Lambert in the box, before rifling home in front of the raucous Northam End.
The former Doncaster striker was booked for putting his finger to his lips in front of the home support after the 27th minute strike and card-happy referee Swarbrick soon issued his fifth yellow card of the half to Scott Allan.
Southampton were looking comfortable but a thunderbolt out of the blue drew Pompey level in the 36th minute.
Maguire found time on the edge of the box to hit a venomous shot that beat Davis and flew into the top corner.
The goal sparked the 3,034 visiting supporters into voice, but Saints dominated until the end of period.
Pompey began the second half brightly and should have gone ahead inside a minute of the restart, but Maguire inexplicably scuffed wide with the goal gaping.
Saints soon jolted back into life and the visitors had goalkeeper Ashdown to thank for keeping the scores level.
The Portsmouth goalkeeper first thwarted Adam Lallana's expertly-struck volley and then tipped over Jose Fonte's goalbound header from the resulting corner.
Lallana fizzed an effort wide after neat play with Sharp, before Portsmouth enjoyed a spell in the ascendancy at the other end.
As the game entered the final 10 minutes, Portsmouth were penned back in their own half.
Fonte came close and Lallana toppled over after rounding the goalkeeper, before Rocha almost put into his own net.
Southampton continued to press and got the goal they deserved in controversial fashion in the 89th minute.
Sharp poked home at the far post and saw his goal disallowed for offside, but the decision was reversed after a chat between the referee and linesman.
However, things changed again in stoppage time when Norris struck home to the delight of the visiting fans.
