The visitors stole a march after just four minutes when Coulson lashed home a sensational low volley from the left-hand side of the box to seal a fifth straight win.
Pompey goalkeeper Trevor Carson was forced to deny them a second three minutes later, when he smothered John McCombe's close-range effort.
Jed Wallace charged down the left and picked out Ryan Taylor after 16 minutes, but his tame header was straight at York stopper Nick Pope.
Wallace then forced Pope to palm away a strike after racing forward straight from the second-half kick-off.
But Portsmouth's hopes of getting back on terms suffered a crushing blow when skipper Ben Chorley was given a straight red card for a high challenge on Wes Fletcher in the 50th minute.
York substitute Ryan Jarvis wasted a golden opportunity to make it two with 11 minutes to go, when he scuffed Ben Davies' cross from the left embarrassingly wide with the goal gaping.