Ryan Bird's headed double and a freak goal from Ricky Holmes put Pompey in command at Fratton Park only for a late rally from the Grecians to leave the home side hanging on.
Holmes put Pompey ahead in first-half injury time when his cross from 40 yards out near the left touchline caught the wind and embarrassed Exeter goalkeeper Artur Krysiak.
Exeter were enraged by referee Oliver Langford as the first of Bird's headed brace doubled Pompey's lead four minutes into the second half.
The official waved play on as Grecians defender Jordan Moore-Taylor lay injured and Yassin Moutaouakil crossed to the far post where Bird's powerful downward header bounced into the net.
Pompey looked to have wrapped up the points when substitute Andy Barcham's first contribution was to whip in a near-post cross which Bird bulleted home.
But Exeter threw themselves a lifeline when Danny Coles headed in from Arron Davies' inswinging corner.
Substitute John O'Flynn piled on even more pressure as Pompey failed to clear and the midfielder poked home from close range after 86 minutes.
And Pompey keeper Trevor Carson preserved victory for his team with a stunning full-length save in injury time from Tom Nichols' header.