Kitson slotted home from six yards in the 73rd minute of Saturday's Championship clash after excellent play from strike partner David Nugent, who then won a penalty which Halford converted in injury time after he was fouled by Leon Barnett.
The Norwich defender, who this week agreed a permanent deal on a three-and-a-half-year contract when the January transfer window opens, was sent off for a second booking as a result to cap a miserable day for the home side.
Portsmouth were also indebted to goalkeeper Jamie Ashdown - who had a loan spell at Norwich in 2006 - after he pulled off several fine saves as City's six-game unbeaten run came to an end.
Norwich had to make one change with captain Grant Holt suspended after picking up his fifth booking in the 2-1 win at Derby last Saturday, with Simeon Jackson recalled to the starting line-up.
Portsmouth were unchanged from the team which beat Swansea 2-1 on November 26, the last time they played after their home clash with Burnley was postponed.
The visitors made a bright start and came close to taking the lead in the third minute.
Barnett was penalised for a foul on Nugent and from the resulting free-kick on the edge of the box, Liam Lawrence's effort curled just wide.
Norwich should have taken the lead in the 18th minute. Chris Martin, making his 100th appearance for the club, flicked the ball on to Jackson, who was through one-on-one but could only shoot straight at Ashdown with his weaker foot.
Jackson headed a corner from on-loan midfielder Henri Lansbury over the crossbar moments later as the Canaries' confidence grew.
From then on, it was all Norwich but they failed to turn their dominance into goals.
Barnett headed a David Fox corner over the crossbar before the latter fired high and wide from 30 yards.
Ashdown produced an excellent save to deny Martin in the 33rd minute and then held a stinging drive from Lansbury while Andrew Crofts failed to find the target moments before the break.
Portsmouth nearly made Norwich pay six minutes into the second half but Lawrence's 30-yard effort hit the post.
The home side responded and Ashdown pulled off another superb stop to keep out Wes Hoolahan's 18-yard drive before the Irish midfielder fired another effort from further out wide of the post.
Portsmouth took the lead 17 minutes from time following great play from Nugent, who eventually left Kitson with a simple opportunity from six yards - his first goal in nine games.
City responded immediately and Fox was foiled by Ashdown from 20 yards with the goalkeeper then in the right position to catch a header from Barnett from Fox's corner before Lansbury drilled just over from range.
Portsmouth wrapped up victory in injury time when Barnett saw red for a second booking after bringing down Nugent in the box and Halford made no mistake.