Pompey, 18th in League Two, needed a penalty shoot-out following a 0-0 draw at Plainmoor.
Whittingham said: "I think we deserved to win it after 90 minutes so I am glad the penalties went our way."
Home skipper Lee Mansell fired Torquay's third penalty over the bar and the result was decided when Pompey right-back Yassin Moutaouakil successfully converted the visitors' fifth.
During the game home keeper Martin Rice made a brilliant late save to deny Pompey dangerman Andy Barcham.
Whittingham added: "This should give us confidence now. I certainly don't think our points tally is value for how we have performed really so I am pleased that we have come away with a win following yet another decent performance."